Being widely adopted in the avionics industry and navigation  already, monitoring and assuring systems’ functional performance based on multiple error sources plays an increasingly important role for IVs and is generally termed Safety of the Intended Functionality (SOTIF) in the automotive context. Cooperative information can increase the efficiency of motion planning for intelligent vehicles (IVs). Relying on wrong information, however, may lead to emergency braking or even crashes. In order to reach SOTIF, measures have to be taken to avoid such incidences. Consider, for example, an IV approaching an intersection with occlusion of the crossing road for on-board sensors. In order to resolve the occlusion, infrastructure sensor modules are posted at the intersection. From the sensor data, an environment model is computed. This model then is reported to the IV by a road side unit (RSU).
While classical probabilistic approaches can easily model uncertainty in the object states, often described by covariance matrices, they lack the ability to explicitly model the evidence-based statistical uncertainty within the estimated probabilities. For the example from above, intuitively, the reliability of the incoming data depends on the amount of evidence supporting them. While classical probabilistic approaches can easily model the probability for the hypothesis that a road user is at a given position at a given time, they lack the ability to express how much evidence actually supports the hypothesis and what is the influence of the assumed prior.
In this work, a Subjective-Logic-based (SL-based) online estimation mechanism is presented that holistically estimates the reliability of cooperative information reported by other agents such as the RSU of the example above. Through the use of SL calculus, IVs are not only able to calculate the SOTIF-related measures, but also determine their corresponding certainty in terms of statistics. Testing for different aspects of functionality results in multiple information sources within the IV, termed opinions . By SL-based fusion of these opinions, we are able to monitor and incorporate multiple areas of functional errors. In this work, we propose the use of four opinions, each providing evidence for the correctness of the overall system. The four tests are 1) the prediction test, in which the RSU’s past behavior prediction of individual objects within its field of view is compared to the current situation, 2) the map test, checking the reported objects for consistency with the IV’s digital map, 3) the ego perception test, which compares detections from the IV’s ego perception with the reported object list, and 4) the ego localization test performing a comparison between the estimated ego position and the position of the corresponding reported object with its uncertainty. The system overview is sketched in Fig. 1.
In distributed systems theory, integrity monitoring and reliability estimation is generally termed Fault Detection and Isolation (FDI) . However, while being a long investigated topic, most developed mechanisms still suffer from exhaustive resource consumption and are centralized by nature, i.e. require an overall monitoring entity and are thus unable to scale up to large installations ,. Furthermore, isolation of (identified) faulty nodes is an open issue . The scheme described within this work shows an online capable and thus low resource consuming, decentralized approach that is able to detect faulty behavior before the fault takes effect.
In the automotive domain, integrity monitoring for IVs is used to monitor digital map reliability using a data-driven classifier approach and to estimate and assure lane information quality , including probabilistic fusion from multiple sources using the Dempster-Shafer (DS) theory of evidence ,. While being effective in practice for their specific use-case, these approaches are tailored towards use of data that is generated from on-board sensors, i.e. from within an IV. We, on the other hand, propose a solution that uses information from the outside and, additionally, is simple in terms of computational complexity. Lastly, our approach relies on the theory of SL , thus does not suffer from the drawback of inconsistent system behavior in some cases due to weaknesses in the underlying DS-based fusion ,. In comparison to , our system takes into account multiple information sources and yields continuous reliabilities as well as their estimation uncertainties rather than a binary classification as part of the fusion process.
Recent works on misbehavior detection in vehicular ad-hoc networks (VANETS) proposed the use of neural networks or SL . While also being designed for large scale and distributed systems, both approaches focus on the security aspects of VANETS. While we acknowledge the importance of security for intra-vehicle communication, we focus on the safety of vehicular communication and the corresponding SOTIF challenges, i.e. assuring functionality safety of a cooperatively perceived environmental model.
In light of previous research, we propose a novel framework for online reliability estimation of cooperative information received from RSUs using the theory of Subjective Logic. The contributions of this paper are twofold. First, we show how SL can be used as a general framework for IVs to assess and fuse individual, orthogonal information cues into one holistic overall reliability estimate and guarantee SOTIF protection levels. Second, we propose four exemplary tests in the context of IVs receiving cooperative information from an RSU. These tests are designed to enable a SOTIF-aware functionality for IVs and are tested in a real-world experiment, which shows that the proposed framework delivers reliable and good results.
Ii Subjective Logic Basics
Subjective Logic is an extension of classical probability theory that explicitly models statistical uncertainty. It unifies several evidence-based extensions of classical probability theory, such as Dempster-Shafer theory, and is linked to classical probability theory by a bijective mapping to Dirichlet distributions. One of the key features of SL is a set of fusion operators that make it possible to combine different pieces of evidence, termed opinions. The Aleatory Cumulative Belief Fusion operator is used to combine independent pieces of evidence to reduce the statistical uncertainty. In turn, the Aleatory Belief Fusion operator is used to average over statistical dependent pieces of evidence. Finally, the Uncertainty Weighted Average Belief Fusion operator does a weighted averaging between two opinions, while the result tends to the more certain opinion. In this section, we briefly summarize the SL basics used in this paper. The definitions and theorems are taken from , where further details can be found.
Definition 1 (Subjective Logic opinion): Let be a domain and , where is the cardinality. Let
further be a random variable in. A SL opinion (opinion in short) is an ordered triple with
Hereby, is the belief mass distribution over , is the uncertainty mass representing a lack of evidence, and is the base rate distribution over representing the prior.
Definition 2 (Dirichlet Distribution): Let be a domain of mutually disjoint values, be the evidence for outcome , a prior distribution over , and
the probability distribution ofover
. Then, the probability density function (PDF)
Remark: Dirichlet PDFs, in essence, are probability distributions over discrete probability distributions, stating the probability that according to the available evidence, the assumed discrete probability distribution is correct. Thus, Dirichlet PDFs and -PDFs in particular are used within this work to visualize SL opinions as well as reliabilities. The higher the value of a -PDF at a given probability, the more evidence supports the probability.
Definition 3 (Aleatory Cumulative Belief Fusion): Let and be sources A’s and B’s respective opinions over the same variable on domain . Then, the operator in
where and , is called Aleatory Cumulative Belief Fusion.
Definition 4 (Aleatory Average Belief Fusion): Let and be sources A’s and B’s respective opinions over the same variable on domain . Then, the operator in
where and , is called Aleatory Average Belief Fusion. When multiple opinions are fused, we use the shorthand
Definition 5 (Uncertainty Weighted Average Belief Fusion): Let and be sources A’s and B’s respective opinions over the same variable on domain . Then, the operator in
with , is called Uncertainty Weighted Average Belief Fusion.
Theorem 1 (Equivalent mapping): Let be an opinion and a Dirichlet distribution over the same . Then, the mapping
transforms the Dirichlet PDF into the opinion and vice versa. Proof: See .
Iii Reliability Estimation Mechanism
In this section, the reliability estimation mechanism is described in detail. First, the four single tests are described. Then, the fusion process of all test results is explained.
Iii-a Consistency Check of the Prediction
The basic idea of the consistency check of the RSU’s behavior prediction is to accumulate predictions as well as the actual measurements and then compare whether or not the former predictions fit to the later measurements. Therefore, upper and lower bounds of each prediction point are calculated. Mathematically, this forms a set of tuples , where is the lower bound of a prediction point and is the upper bound, respectively. In the second step, the set of actual measurements
is linearly interpolated resulting in the polygon line. The polygon line then splits the measurement space into an upper and a lower subspace and , respectively. The set of prediction tuples then is partitioned into a correct and incorrect subset:
With the evidence vector
and the equivalent mapping (7), the partitioned subsets are mapped to the opinion . Finally, a probability sensitive trust discounting
of the test opinion accounting for the statistical dependence of the prediction samples is performed. Herby, can be interpreted as the probability vector that the current piece of evidence is statistically independent of the evidence already accounted for and, thus, truly brings new information into the fusion result. Simply spoken, a tuple is classified as correct, if the polygon line intersects the corresponding interval. Otherwise it is classified as incorrect. Note that the trust discounting step basically has the same effect as applying the partially dependent source cumulative belief fusion operator from SL described in .
As we focus on reliability analysis rather than object prediction in this work, for simplicity, a Kalman filter with a constant velocity model is used to predict the objects’ future positions along their lanes. This then is used as behavior prediction of the RSU. Thus, the upper and lower bounds can simply be chosen as thebounds of the individual prediction points. Note that more sophisticated methods for behavior prediction, e.g. , exist and can be integrated as well. The upper and lower bounds then have to be chosen accordingly.
Iii-B Consistency Check Between Map and Received Object Lists
The idea of this test is to detect inconsistency between the digital map of the IV and the object list reported by the RSU. Therefore, at first, the typical positions of the object centers according to the RSU perception are determined during commissioning, where the correct functionality of the RSU is guaranteed. This is done using several 1D histograms along the lanes spread over the RSU’s field of view (FOV), where the 1D histograms only resolve the lateral position of the object centers and are simply stacked along the lanes (see Fig. 2). The individual bin length is chosen to compromise between fine-grained approximation of the corresponding continuous density and a reasonable value of object probes that accumulates within each bin. Note that we do not expect the exact number of bins to have a strong influence on the overall result because of the Aleatoric Average Belief Fusion performed over all individual bins. The reference histograms then are mapped to reference opinions using the equivalence mapping (7) and pooled according to
Thus, only one average opinion per lane is computed. These are stored in the digital map of the vehicle.
When the IV approaches the FOV of the RSU, it starts to create a set of histograms using the same bins as for the creation of the reference opinion. The sample opinion
then is determined online the same way as , but with only the online data available from the reported object lists . If the RSU data is consistent with the map, the distance between the opinions and should be small, i.e. below a threshold . A measure for the distance between opinions is the degree of conflict
where and , respectively, are the maximum-likelihood mappings of the opinions and to probabilities .
If , the incoming RSU data is considered to match the vehicle’s map and, hence, the estimated reliability within the map test opinion is increased according to
where is the opinion generated from the new evidence, and is the probability sensitive trust discounted opinion  about the new evidence. can be interpreted as the probability that a single measurement is meaningful. Here, is used. Otherwise, the trust into the RSU is revised according to the probability sensitive trust revision :
Iii-C Comparison Between Ego Perception and Received Object Lists
The RSU’s main purpose is to extend the environmental model of the IV into areas that are occluded for the IV’s ego perception. Therefore, it can be expected that the mutual FOV of the IV and the RSU is small and mostly in corner regions, where the IV’s ego perception reliability is reduced. Furthermore, objects may move quickly through the narrow mutual FOV. Thus, it can be expected that the IV’s ego perception will only receive very view measurements to confirm the existence of an object reported by the RSU. As a first step, we perform nearest neighbor association with the Euclidean distance between object received from the RSU and the objects detected with the IV’s on-board perception FOV. For gating, a simple thresholding with a fixed threshold before association is used. Note that more sophisticated mechanisms for both gating and data association are available  and could be integrated as well. We, however, want to focus on the overall idea of SL-based fusion and reliability estimation in this work and thus use this simple, yet effective technique for gating and association. If a matching object is found, the trust into the RSU is increased, else a missing detection is assumed and the trust into the RSU is strongly reduced. Mathematically, this mechanism can be described as
In (16), describes the opinion resulting from an object within the ego perception that confirms an object from the RSU’s object list , while is the opinion resulting from a missing detection, i.e. when the ego perception detects an object that has no match in . To account for the fact that missing detection might have severe implications, they are up-weighted compared to positive tests with the parameter , which ensures that missing objects decrease confidence in the RSU significantly.
The new input opinion then is fused with the last perception test opinion resulting in the new perception test opinion . This time-recursive structure incorporates the fact that both the ego and the RSU perception are expected to have recursive estimators to create their individual object lists.
Iii-D Comparison Between Ego Localization and Received Object Lists
While the comparison between ego perception and received object list can detect if missing detections in the RSU occur, it does not make a statement on the measurement uncertainties reported by the RSU. This is due to the fact that the measured objects are extended by nature. Thus, the IV may perceive only a part of an object, while the RSU perceives the whole object resulting in differing object center estimates. A check on the uncertainties thus would not always be meaningful for the ego perception test. Hence, an additional test is needed. By comparing the object corresponding to the IV with the ego localization, a very precise measurement is available to evaluate the uncertainties reported by the RSU. Furthermore, even if no other road user is approaching the intersection, the ego localization test is always possible.
The reported uncertainties are checked by comparing the Euclidean distance between ego localization and object center of the corresponding RSU object with the total uncertainty consisting of the ego localization uncertainty and the reported uncertainty . If
, the opinions are considered consistent and the trust into the RSU is increased. Otherwise, assuming a Gaussian distributed error, the probability that the data is consistent is below. Thus, in this case, the trust into the RSU is reduced. If no matching object can be found within , a missing detection in the RSU’s perception is detected and the incidence is passed to the ego perception test. Mathematically, the ego localization test can be formulated as
where is the opinion confirming the RSU, is the opinion resulting from an underestimated uncertainty and is the new input opinion the ego localization test opinion is updated with.
Iii-E Fusion of the Test Results
So far, four tests have been proposed, each of which focuses on a different property of valid RSU data. While the map test and the prediction test completely rely on the data reported by the RSU, the ego perception test and the ego localization test include the IV’s perception as additional source of information. Thus, given the same amount of evidence, the tests including the IV’s perception are far more important for the overall estimation of the RSU’s reliability. In turn, as the ego perception test as well as the ego localization test need the IV be already very close to the intersection, these test will usually operate on a far smaller amount of evidence as the tests independent of the IV’s perception. Both effects are accounted for by combining the uncertainty weighted average belief fusion operator with a simple importance weighted average belief fusion. Mathematically, this can be expressed as
In (18), describes the fused opinion on the ego perception independent tests, while describes the fusion of the ego-perception-based tests. The opinions and then are mapped to the Dirichlet space using the mapping (7), weighted there with and , respectively, and mapped back to the overall fusion result . Thus, (18) defines a new SL fusion operator that does an belief and importance weighted average belief fusion.
Iv Experimental Setup
The experiment is set up at a mid-size city crossing with occlusion due to close-to-road buildings. Besides others, four SICK LD-MRS 8 layer laserscanners are mounted on poles in height, observing the intersection. Their data is fused and the resulting object list then is reported to the IV by the RSU. Fig. 2 sketches the FOV covered by the LiDAR subsystem of the RSU. As can be seen, the laserscanners show a blind spot where sensor reliability is reduced111The blind spot is observed by the other sensors, not used here.. Thus, real examples of missing detections are expected to be found within the recorded data. The IV approaches this intersection, while the RSU sends object lists of its perception.
For the IV, an automated vehicle as described in  is used. For the ego perception, the front IBEO 4 layer laserscanner is used. The IV is localized using a high precision real time kinematic (RTK) system.
For evaluation, sequences where recorded on two different days, where the IV approaches the intersection on different entrance lanes. The sequences have been manually labeled with respect to malfunctions of the RSU such as missing detections. In sequences the RSU has been manually classified as correctly behaving, while in the other , the RSU shows misbehavior. Only the map test could not be evaluated without modifying the recorded data. In order to evaluate the map test, thus, the original data reported by the RSU are shifted in east direction. This corresponds to a differently chosen reference point or a calibration error due to a GPS offset during recalibration.
V-a Consistency Check Evaluation of the Prediction
Figure 3 exemplary visualizes the result of the prediction test. In the given example, the observed vehicle decelerates. Hence, some of the prediction tuples are classified as incorrect. This leads to a reduction of the estimated reliability. Thus, it can be seen that the proposed mechanism behaves as expected.
V-B Evaluation of the Consistency Check Between Map and Received Object Lists
To evaluate the consistency test between the map and the received object lists, the objects from the RSU are shifted by a constant offset to simulate an RSU calibration or mapping error. As the object centers are highly concentrated within a small band in the northern branch of the intersection (see Fig. 2) the probability gradient along east direction is big. Figure 4 shows the effect of a shift in east direction on the estimated reliability. It can be seen that the test reacts quite sensitive to a shift in east direction due to the big probability gradient along east direction. If the shift is big enough, the RSU reports objects at positions that are physically unfeasible, i.e. objects moving over houses. Thus, the estimated reliability then is reduced to zero all of a sudden (see ).
V-C Evaluation of the Comparison between Ego Perception and Received Object Lists
Figure 5 exemplary shows the effect of a missing detection in the RSU. First of all, the ego perception objects match the objects reported by the RSU. Hence, the estimated reliability of the RSU is high as can be seen from the blue Dirichlet PDF in Fig. 5LABEL:sub@fig:PerceptionOpinion and the mapped reliability in Fig. 5LABEL:sub@fig:PerceptionReliability . Then, the missing detection occurs as can be seen in Fig. 5LABEL:sub@fig:PerceptionVisualization. As consequence, the estimated reliability is significantly decreased, as can be seen in Fig. 5LABEL:sub@fig:PerceptionReliability. The -PDF corresponding to the respective perception test opinion at that instance of time is depicted in red in Fig. 5LABEL:sub@fig:PerceptionOpinion. After the missing detections, the estimated reliability slowly recovers due to the following correctly reported information.
V-D Evaluation of the Comparison between Ego Localization and Received Object Lists
In Fig. 6, the effect of an underestimated uncertainty is demonstrated by example, wherethe overall uncertinty was chosen too small. Thus, the estimated reliability is reduced from the prior estimation, because the geometrical center of the ego vehicle according to the RTK measurement is outside the uncertainty ellipse estimated by the RSU.
V-E Evaluation of the Overall Reliability Estimation
In order to evaluate the overall reliability estimation, the reliability estimations are mapped to -distributions and visualized all in Fig. 7. The -distributions resulting from the sequences with correctly performing RSU are marked in blue, while the -distributions resulting from a faulty RSU are marked in red. It can be seen that the two classes are easily separable using the proposed reliability estimation mechanism. Furthermore it shows that the reliability is estimated to be beyond whenever the RSU is reliable, while the estimated reliability is below whenever the RSU is faulty. By integrating over the -distributions it can be confirmed that for all true positives a confidence level supports the hypothesis that the correct function probability is at least . In turn, whenever the RSU is faulty, there is at least a confidence that the correct function probability is at most . This demonstrates that the reliability estimation mechanism works well and shows robustness with respect to intra-class variations.
In this work, a general framework to estimate the reliability of cooperative information received from an Road Side Unit (RSU) for use in intelligent vehicles (IVs) was presented. By the use of the theory of Subjective Logic (SL), we showed how orthogonal information sources (termed opinions in SL) can be fused in an natural and mathematically sound and easily extendable way. To prove the applicability, four exemplary tests were proposed, which individually only provide mild cues towards an RSU’s reliability of information. However, after the proposed scheme for probabilistic fusion, IVs are able to separate faulty from correct data samples with a large margin of safety. Real-world experiments showed the applicability and effectiveness of our approach.
In future, the possible faulty effects of wireless data communication will be examined as well as the effects of sudden and short-lived sources of error like sensor blockage.
-  S. Pullen, “Augmented GNSS: Fundamentals and Keys to Integrity and Continuity,” in Proceedings of the ION GNSS, 2011.
-  A. Jøsang, Subjective Logic. Springer International Publishing, 2016.
-  A. Zidan, M. Khairalla, A. M. Abdrabou, T. Khalifa, K. Shaban, A. Abdrabou, R. E. Shatshat, and A. M. Gaouda, “Fault Detection, Isolation, and Service Restoration in Distribution Systems: State-of-the-Art and Future Trends,” IEEE Transactions on Smart Grid, vol. 8, no. 5, pp. 2170–2185, Sep. 2017.
-  A. Dorri, S. S. Kanhere, and R. Jurdak, “Multi-Agent Systems: A Survey,” IEEE Access, vol. 6, pp. 28 573–28 593, 2018.
-  O. Hartmann, M. Gabb, F. Schüle, R. Schweiger, and K. Dietmayer, “Robust and Real-Time Multi-Cue Map Verification for the Road Ahead,” in Proc. 17th Int. IEEE Conf. Intelligent Transportation Systems (ITSC), Oct. 2014, pp. 894–899.
-  T. T. Nguyen, J. Spehr, M. Uhlemann, S. Zug, and R. Kruse, “Learning of Lane Information Reliability for Intelligent Vehicles,” in Proc. IEEE Int. Conf. Multisensor Fusion and Integration for Intelligent Systems (MFI), Sep. 2016, pp. 142–147.
-  T. T. Nguyen, J. Spehr, J. Xiong, M. Baum, S. Zug, and R. Kruse, “Online Reliability Assessment and Reliability-Aware Fusion for Ego-Lane Detection Using Influence Diagram and Bayes Filter,” in Proc. IEEE Int. Conf. Multisensor Fusion and Integration for Intelligent Systems (MFI), Nov. 2017, pp. 7–14.
T. T. Nguyen, J. Spehr, D. Vock, M. Baum, S. Zug, and R. Kruse, “A General Reliability-Aware Fusion Concept Using DST and Supervised Learning with Its Applications in Multi-Source Road Estimation,” inProc. IEEE Intelligent Vehicles Symp. (IV), Jun. 2018, pp. 597–604.
-  M. Obst, L. Hobert, and P. Reisdorf, “Multi-Sensor Data Fusion for Checking Plausibility of V2V Communications by Vision-based Multiple-Object Tracking,” in 2014 IEEE Vehicular Networking Conference (VNC), Dec 2014, pp. 143–150.
-  F. A. Ghaleb, A. Zainal, M. A. Rassam, and F. Mohammed, “An Effective Misbehavior Detection Model Using Artificial Neural Network for Vehicular Ad Hoc Network Applications,” in Proc. Information and Network Security (AINS) 2017 IEEE Conf. Application, Nov. 2017, pp. 13–18.
-  S. Dietzel, R. van der Heijden, H. Decke, and F. Kargl, “A Flexible, Subjective Logic-Based Framework for Misbehavior Detection in V2V Networks,” in Proc. Proceeding of IEEE Int. Symp. a World of Wireless Mobile and Multimedia Networks 2014, Jun. 2014, pp. 1–6.
-  I. Bronshtein, K. Semendyayev, G. Musiol, and H. Mühlig, Handbook of Mathematics. Springer Berlin Heidelberg, 2007.
-  A. Jøsang, S. Marsh, and S. Pope, “Exploring Different Types of Trust Propagation,” in Trust Management, K. Stølen, W. H. Winsborough, F. Martinelli, and F. Massacci, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 179–192.
J. Wiest, M. Höffken, U. Kreßel, and K. Dietmayer, “Probabilistic Trajectory Prediction with Gaussian Mixture Models,” in2012 IEEE Intelligent Vehicles Symposium, June 2012, pp. 141–146.
-  S. Reuter, B. Vo, B. Vo, and K. Dietmayer, “The Labeled Multi-Bernoulli Filter,” IEEE Transactions on Signal Processing, vol. 62, no. 12, pp. 3246–3260, June 2014.
-  F. Kunz, D. Nuss, J. Wiest, H. Deusch, S. Reuter, F. Gritschneder, A. Scheel, M. Stübler, M. Bach, P. Hatzelmann, C. Wild, and K. Dietmayer, “Autonomous Driving at Ulm University: A Modular, Robust, and Sensor-Independent Fusion Approach,” in 2015 IEEE Intelligent Vehicles Symposium (IV), June 2015, pp. 666–673.