I Introduction
The theory of belief functions (also called Dempster–Shafer Theory, DST) provides effective tools to modele the uncertain information and to combine them using a combination rule. One of the classical combination rule in the belief function framework is the Dempster’s rule [1]. But this rule has been criticized because of its unexpected behavior under some situations enlightened by the famous Zadeh’s example. Smets proposed a modification of Dempster’s rule, often called “conjunctive rule”, where the empty set can be assigned with a nonnull mass under the Transferable Belief Model (TBM) [2]. In fact the conjunctive rule is equivalent to the Dempster rule without the normalization process. It has a fast and clear convergence towards a solution.
Smets supports that the mass on the empty set, called also global conflict, can play a role of alarm [2]. When the global conflict is high, it indicates that there is strong disagreement among the sources of mass functions to be fused. However, as observed in [3, 4], the mass on the empty set is not sufficient to exactly describe the conflict since it includes an amount of autoconflict [5]. Even if the sources are reliable, they can be of high conflict in sense of the mass assigned to the empty set. When using the conjunctive rule, even if there is only a small amount of concordant evidence, the total conflict mass function, i.e. will be an absorbing element. Consequently, when combining a large number of (incompatible) mass functions using the conjunctive rule, the global conflict may tend to 1. This makes it impossible to reveal the cause of high global conflict. We do not know whether it is due to the sources to fuse or caused by the absorption power of the empty set [3]. In other words, even the combined mass function by the conjunctive rule is , the proposition that the sources are highly conflicting may be incorrect.
In order to rectify the drawbacks of the classical Dempster’s rule and conjunctive rule, many approaches have been made through the modification of the combination rule. However, most of them are not efficient when applied on a large number of sources either due to the ineffective way to handle conflict or the high complexity of the computation. Moreover, the major opinion among all the participants is not easy to found. In some applications such as crowdsourcing, there are usually a large number of sources. An important problem for crowdsourcing systems is to identify the experts who tend to answer the questions correctly among participants. Finding the reliable workers in the system can improve the quality of knowledge one can extract from crowds. One of the most commonly used assumptions in crowdsourcing systems is that the majority of participants can give correct answers. Thus if the major opinion among the participants could be elicited, the reliable workers can be easily found.
We propose in this paper, a conjunctivebased combination rule, named (stands for Large Number of Sources), in order to aggregate a large number of mass functions. This rule is based on the idea that combining mass functions from different sources is similar to combining opinions from multiple stakeholders in group decisionmaking. Hence, the more one’s opinion is consistent with the other experts, the more reliable the source is. To build our rule, we assume that all the mass functions available are separable mass functions. A separable mass function can be expressed by a group of simple support mass functions, such as nondogmatic mass functions. In many applications, the mass assignments are directly in the form of Simple Support Functions (SSF) [6]. Hence, we can group the SSFs in such a way that sources in the same group have the same focal elements or have focal element without conflict. Mass functions in each small group are first fused and then discounted according to the proportions of SSFs in each group. After that the number of mass functions to fuse is the number of groups which is independent of the number of sources. Therefore, the problem brought by the absorbing element (the empty set) using the conjunctive rule can be avoided.
The rest of this paper is organized as follows. In Section 2, some basic knowledge of belief function theory is briefly introduced. The proposed evidence combination approach is presented in detail in Section 3. Numerical examples are employed to compare different combination rules and show the effectiveness of rule in Section 4. Finally, Section 5 concludes the paper.
Ii Background
Iia Basic knowledge on the theory of belief functions
We consider such as the discernment frame. A mass function is defined on the power set . The mass function is said to be a Basic Belief Assignment (BBA) on , if it satisfies:
(1) 
If , with , is called a focal element, and the set of focal elements is denoted by .
The frame of discernment can also be a focal element. If is a focal element, the mass function is called nondogmatic. The mass assigned to the frame of discernment, , is interpreted as a degree of ignorance. In the case of total ignorance, . This mass function is also called a vacuous mass function. If there is only one focal element, i.e. , the mass function is categorical. Another special case of assignment is named consonant mass functions, where the focal elements include each other as a subset, i.e. if .
In order to combine information sources assumed reliable and cognitively independent, the conjunctive combination is usually used, given by:
(2) 
where is the mass allocated to by expert . Another kind of conjunctive combination is Dempster’s rule [7] given by:
(3) 
The item is generally called Dempster’s degree of conflict of the combination or the inconsistency of the combination.
The disjunctive rule [8] can be used if we only assume that at least one of the sources is reliable.
If information sources are dependent the cautious rule [9] can be applied. Cautious combination of nondogmatic mass functions is defined by the BBA with the following weight function:
(4) 
We thus have
(5) 
where is the simple support function focused on with weight function issued from the canonical decomposition of . Note also that is the min operator. Moreover, in the case of dependant sources, the average combination rule can be choosen.
In order to manage the value by redistributing it on partial ignorance, the Dubois and Prade rule ( rule) [10], can be applied. It can be seen as a mixed conjunctive and disjunctive rule.
Moreover the proposed by [5] is one of the most popular rule to combine hight conflicting sources.
IiB Discounting process based on sourcereliability
When the sources of evidence are not completely reliable, a discounting operation proposed by [6] can be applied. Denote the reliability degree of mass function by , then the discounting operation can be defined as:
(6) 
If , the evidence is completely reliable and the BBA will remain unchanged. On the contrary, if , the evidence is completely unreliable. In this case the socalled vacuous belief function, can be got. It describes our total ignorance.
Before evoking the discounting process, the reliability of each sources should be known. One possible way to estimate the reliability is to use confusion matrices
[11]. Generally, the goal of discounting is to reduce global conflict before combination. One can assume that the conflict comes from the unreliability of the sources. Therefore, the source reliability estimation is to some extent linked to the estimation of conflict between sources.Hence, Martin et al. proposed to use a conflict measure to evaluate the relative reliability of experts [3]. Once the degree of conflict is computed, the relative reliability of the source can be computed accordingly. Suppose there are sources, , the reliability discounting factor of source can be defined as follows:
(7) 
where quantifies the degree that source conflicts with the other sources in , and is a decreasing function. The following function is suggested by the authors:
(8) 
where .
IiC Simple support functions
Suppose is a BBA defined on the frame of discernment . If there exists a subset such that could be expressed in the following form:
(9) 
where , then the belief function related to BBA is called a Simple Support Function (SSF) (also called simple mass function) [6] focused on . Such a SSF can be denoted by where the exponent of the focal element is the basic belief mass (bbm) given to the frame of discernment , . The complement of to 1, i.e. , is the bbm allocated to [12]. If the mass function represents the total ignorance, if the mass function is a categorical BBA on .
A belief function is separable if it is a SSF or if it is the conjunctive combination of some SSFs [13]. In the work of [13], this kind of separable masses is called useparable where “u” stands for “unnormalized”, indicating the conjunctive rule is the unnormalized version of DempsterShafer rule. The set of separable mass functions is not obvious to obtain. It is easy to see consonant mass functions (the focal elements are nested) are separable.Smets [12] defined the Generalized Simple Support Function (GSSF) by relaxing the weight to . Those GSSFs with are called Inverse Simple Support Functions (ISSF). He proved all nondogmatic mass functions are separable if one uses GSSFs.
Iii A combination rule for a large number of mass functions
The conjunctive combination rule tries to reinforce the belief on the focal elements with which most of the sources agree. However, in this rule, the empty set is an absorbing element. When combing inconsistent BBAs, the mass assigned to the empty set tends quickly to 1 with the increasing number of sources [3]. Consequently, when using Dempster rule, the gap between
and 1 may rapidly exceed machine precision, even if the combination is valid theoretically. In that case the fused BBAs by the conjunctive rules (normalized or not) and the pignistic probability are inefficient due to the limitation of machine precision. Moreover, the conjunctive combination rule assumes that all the sources are reliable, which is difficult to reach or to verify in real applications.
In the theory of belief functions, the idea to the reinforce belief and the alarm role of the empty set in the conjunctive rule are essential. In order to propose a rule which can be applicable when the number of mass functions to combine is large and keep the previous behavior, the following assumptions are made:

The majority of sources are reliable;

The larger extent one source is consistent with others, the more reliable the source is;

The sources are cognitively independent [8].
Based on these assumptions, the proposed rule will discount the mass functions according to the number of sources providing BBAs with the same focal elements. The discounting factor is directly given by the proportion of mass functions with the same focal elements. As a result, the rule will give more importance to the groups of mass functions that are in a domain, and it is free of autoconflict [5]. This procedure can be used to elicit of the majority opinion.
The simple support mass functions are considered here. In this case, the mass functions can be grouped in the light of their focal elements (except the frame ). To make the rule applicable on separable mass functions, the decomposition process should be performed to decompose each BBA into simple support mass functions. In most of applications, the basic belief can be defined using separable mass functions, such as simple support functions [14] and consonant mass functions [15].
Hereafter we describe the proposed rule for simple support functions, and then an approximation calculation method of rule is suggested.
Iiia Combination of many simple support functions
Suppose that each evidence is represented by a SSF. Then all the BBAs can be divided into at most groups (where ). It is easy to see that there is no conflict at all in each group because of consistency. The focal elements of the SSF are singletons and itself. For the combination of BBAs inside each group, the conjunctive rule can be employed directly. Then the fused BBAs are discounted according to the number of mass functions in each group. Finally, the global combination of the BBAs of different groups is preformed also using the conjunctive rule. Suppose that all BBAs are defined on the frame of discernment
and denoted by
where . The detailed process of the combination is listed as follows. Our proposed rule called for Large Number of Sources rule is composed of the four following steps:

Cluster the simple BBAs into groups based on their focal element . For the convenience, each class is labeled by its corresponding focal element.

Combine the BBAs in the same group. Denote the combined BBA in group by SSF , . For the conjunctive combination rule we have:
(10) where the number of BBAs in group is . In order to consider the total ignorance as a neutral element of the rule, if we allow .

Reliabilitybased discounting. Suppose the fused BBA of all the mass functions in be . At this time, each group can be regarded as a source, and there are sources in total. The reliability of one source can be estimated by comparing to the group of sources. In our opinion, the reliability of source is related to the proportion of BBAs in this group. The larger the number of BBAs in group is, the more reliable is. Then the reliability discounting factor of , denoted by , can be defined as:
(11) Another version of the discounting factor can be determined by a factor taking into account the precision of the group:
(12) where
(13) Parameter can be used to adjust the precision of the combination results. The larger the value of is, the less imprecise the resulting BBA is. The discounted BBA of can be denoted by SSF with . As we can see, when the number of BBAs in one group is larger, is closer to 1. That is to say, the fused mass in this group is more reliable.

Global combine the fused BBAs in different groups using the conjunctive rule:
(14)
The previous mentioned methods in Section IIB to estimate reliability are much more complex than the proposed method here. Indeed, usually the distance between BBAs should be calculated or a special learning process is required. In rule, to evaluate the reliability discounting factor, we only need to count the number of SSFs in each group. But other reliability estimation methods can also be used.
IiiB LNS properties
The proposed rule is commutative, but not associative. The rule is not idempotent, but there is no absorbing element. The vacuous mass function is a neutral element of the rule.
There are four steps when applying rule^{1}^{1}1The source code for rule could be found in R package ibelief [16].: decomposition (not necessary for simple support mass functions), innergroup combination, discounting and global combination. The rule has the same memory complexity as some other rules such as conjunctive, and cautious rules if all the rules are combined globally using FMT method. Only and rules have higher memory complexity because of the partial conflict to manage. Suppose the number of mass functions to combine is , and the number of elements in the frame of discernment is . The complexity for decomposing^{2}^{2}2In the decomposing process, the Fast Möbuis Transform method is used. mass functions to SSFs is . For combining the mass functions in each group, due to the structure of the simple support mass functions, we only need to calculate the product of the masses on only one focal element . Thus the complexity is . The complexity of the discounting is . In the process of global combination, the BBAs are all SSFs. If we use the Fast Möbuis Transform method, the complexity is . Moreover there are at most mass functions participating the following discounting and global conjunctive combination processes. Since in most application cases with a large number of mass functions, we have , the last two steps are not very timeconsuming. The total complexity of is and so is equivalent to .
We remark here that one of the assumptions of rule is that the majority of sources are reliable. However, this condition is not always satisfied in every applicative context. Consider here an example with two sensor technologies: TA and TB. The system has two TAsensors ( and ), and one TBsensor . Suppose also a parasite signal causes TA sensors to malfunction. In this situation, the majority of sensors are unreliable, and we could not get a good result if the rule is used directly as at this time. Actually there is an underlying hierarchy in the sources of information, rule could be evoked according to the hierarchy, such as . We will study that more in the future work.
Iv Experiments
To illustrate the behavior of the proposed combination rule and to compare with other classical rules, several experiments will be conducted here. Some different types of randomly generated mass functions will be used. The function RandomMass in R package ibelief [16] is adopted to generate random mass functions.
Experiment 1. In the crowdsourcing applications, all the users can provide some imprecise and uncertain answers. But only a few are trusty. The elicitation of the majority opinion is very important to identify the experts. Assume that the answers by different uses are in the form of the mass functions over the same discernment frame denoted by . The assignments of all the mass functions are as follows:
As can be seen from the above equations, there are five out of six mass functions () assigning a large value on , while delivers a function strongly committed to another solution. It indicates that the first source is obviously different from the other five sources.
The combination results by conjunctive rule, rule, disjunctive rule, rule, rule, cautious rule, average rule and the proposed rule^{3}^{3}3As the focal elements are singletons except , parameter has no effects on the final results by the proposed combination rule. are depicted in Table I. As can be observed, the conjunctive rule assigns most of the belief to the empty set, regarding the sources as highly conflictual. rule, rule, rule and average rule redistribute all the global conflict to other focal elements. Disjunctive rule gives the total ignorance mass functions. Cautious rule and the proposed rule keep some of the conflict and redistribute the remaining. From the original six BBAs, we can see that there are five mass functions supporting , while only one supporting . The six mass functions are not conflicting, because the majority of evidence shows the preference of . We consider here source 1 is not reliable since it contradicts with all the other sources. But the belief given to is more than that to when using , , , and the cautious rule, which indicates that these rules are not robust to the unreliable evidence. The result by the average rule gives equal evidence to and . The obtained fused BBA by the proposed rule assigns the largest mass to focal element , which is consistent with the intuition. It keeps a certain level of global conflict, and at the same time reflects the superiority of compared with . From the results we can see that only the rule can correctly elicit the major opinion.
Conjunctive  DS  Disjunctive  DP  PCR6  Cautious  Average  

0.57341  0.00000  0.00000  0.00000  0.00000  0.27000  0.00000  0.07964  
0.06371  0.14935  0.00000  0.06371  0.10644  0.03000  0.15000  0.45129  
0.32659  0.76558  0.00000  0.32659  0.45139  0.63000  0.15000  0.07036  
0.00000  0.00000  0.00011  0.08165  0.00000  0.00000  0.00000  0.00000  
0.00000  0.00000  0.00000  0.00000  0.00000  0.00000  0.00000  0.00000  
0.00000  0.00000  0.00000  0.00000  0.00000  0.00000  0.00000  0.00000  
0.00000  0.00000  0.00000  0.00000  0.00000  0.00000  0.00000  0.00000  
0.03629  0.08506  0.99989  0.03629  0.44217  0.07000  0.70000  0.39871 
Experiment 2. We test here the influence of parameters and in rule. Simple support mass functions are utilized in this experiment. Suppose that the discernment frame under consideration is . Three types of SSFs are adopted. First and SSFs with focal element and respectively (the other focal element is ) are uniformly generated, and then SSFs with focal element are generated. The value of masses are randomly generated. Different values of ranging from 0 to 6 are used to test. The mass values in the fused BBA by varying with are displayed in Figure 1, and the corresponding pignistic probabilities are shown in Figure 2.
From these figures, we can see that can have some effects on the final decision. Figure 1 shows that with the increasing of , the mass function assigned to the singleton focal elements increases. On the contrary, the mass given to the focal element whose cardinality is bigger than one decreases. In fact parameter in aims at weakening the imprecise evidence which gives only positive mass to focal elements with high cardinality, and the exponent allows to control the degree of discounting. If is larger, we give more weights to the sources of evidence whose focal elements are more specific, and more discount will be committed to the imprecise evidence. As a result, in the experiment when is larger than 1.2, (Figure 2). At this time the mass functions with focal element make little contribution to the fusion process, while the final decision mainly depends on the other two types of simple support mass functions with singletons as focal elements.
In real applications, could be determined based on specific requirement. This work is not specially focusing on how to determine , thus in the following experiment we will set as default.
Experiment 3. The goal of this experiment is to show how Dempster’s degree of conflict is dealt with by most of rules when combining a large number of conflicting sources.
In this experiment, the frame of discernment is set to . Assume that there are only 2 focal elements on each BBA. One is the whole frame , and the other is any of the singletons ( or ). The number of BBAs which have the focal element is set to , while that with is . We fix the value of , and let , with a positive integer. We generate such kind of BBAs randomly, but only withholding the BBAs for which the mass value assigned to or is greater than 0.5.
Four values of are considered here: . If , . If , the number of mass functions supporting is two times of that supporting , and so on. The global conflict (mass given to the empty set) after the combination with different values of for the four cases is displayed in Figures 3–6 respectively. The mass assigned to the focal element with different combination approaches is shown in Figures 7–10.
It is intuitive that when becomes larger, the global conflict should be smaller and we should give more belief to the focal element . From Figures 3–10 we can see that only the results by rule are in accordance with this common sense. The simple average rule assigns larger BBA to , but it does not keep any conflict. rule could not work at all when is larger than 20^{4}^{4}4In Figures 7–10, the mass given to by rule is not displayed when is large (and also for some small ), because in these cases the global conflict is 1 and the normalization could not be processed., as it regards these BBAs as highly conflict. Although the conjunctive rule and cautious rule could work when combining a larger number of mass functions, the obtained fused mass function is , which is useless for decision in practical.
From Figures 3–6, we can see a kind of limit of the global conflict for the rule. In fact, the mass on the empty set for this rule is also depending on the size of the frame of discernment and more directly on the number of groups created in the first step of the rule. The limit value of the global conflict will tend to 1 with the increase of the size of discernment when considering only categorical BBAs on different singletons.
V Conclusion
There is usually a lot of uncertain information in big data applications. The theory of belief functions is a flexile framework to deal with imprecise and uncertain information, especially it provides many ways for the task of information fusion. However, although lots of combination rules have been designed in recent years in this framework, most of them are not applicable when the number of source to combine is quite large due to the complexity or the existing absorbing element.
We propose here a new combination rule, named rule, preserving the principle of the conjunctive rule. This rule first groups the mass functions according to their set of focal elements (without autoconflict). After the inner combination, the mass functions in each group can be summarized by one mass function. The reliability of the source is estimated by the proportion of BBAs in one group. Therefore, after discounting the mass function of each group by the reliability factor, the final combination can be proceeded by the conjunctive rule (or another rule according to the application).
The rule is able to combine a large number of mass functions. The only existing method applicable for a large number of sources is the average rule. However, that rule may give more importance to few sources with a high belief (even if the source is not reliable). It cannot capture the conflict between the sources. The proposed rule has a reasonable complexity (lower than the and rules). Moreover, it can provide reasonable combination results and can be used to elicit the major opinion. This is of practical value in the crowdsourcing system.
Overall, this work provides a perspective for the application of belief functions on big data. We will study how to apply rule on the problems of social network and crowdsourcing in the future research work.
Acknowledgements
This work was supported by the National Natural Science Foundation of China (Nos.61135001, 61403310, 61672431) and the Fundamental Research Funds for the Central Universities of China (No.3102016QD088).
References
 [1] P. Smets, “Analyzing the combination of conflicting belief functions,” Information Fusion, vol. 8, pp. 387–412, 2007.
 [2] ——, “The combination of evidence in the transferable belief model,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 12, no. 5, pp. 447–458, 1990.
 [3] A. Martin, A.L. Jousselme, and C. Osswald, “Conflict measure for the discounting operation on belief functions,” in Information Fusion, 2008 11th International Conference on. IEEE, 2008, pp. 1–8.
 [4] S. Destercke and T. Burger, “Toward an axiomatic definition of conflict between belief functions,” Cybernetics, IEEE Transactions on, vol. 43, no. 2, pp. 585–596, 2013.
 [5] A. Martin and C. Osswald, “Human experts fusion for image classification,” Information & Security: An International Journal, Special issue on Fusing Uncertain, Imprecise and Paradoxist Information (DSmT), vol. 20, pp. 122–143, 2006.
 [6] G. Shafer, A mathematical theory of evidence. Princeton University Press, 1976.
 [7] A. P. Dempster, “Upper and lower probabilities induced by a multivalued mapping,” The annals of mathematical statistics, pp. 325–339, 1967.
 [8] P. Smets, “Belief functions: the disjunctive rule of combination and the generalized bayesian theorem,” International Journal of approximate reasoning, vol. 9, no. 1, pp. 1–35, 1993.
 [9] T. Denœux, “The cautious rule of combination for belief functions and some extensions,” in Information Fusion, 2006 9th International Conference on. IEEE, 2006, pp. 1–8.
 [10] D. Dubois and H. Prade, “Representation and combination of uncertainty with belief functions and possibility measures,” Computational Intelligence, vol. 4, no. 3, pp. 244–264, 1988.
 [11] A. Martin, “Comparative study of information fusion methods for sonar images classification,” in Information Fusion, 2005 8th International Conference on, vol. 2. IEEE, 2005, pp. 7–pp.

[12]
P. Smets, “The canonical decomposition of a weighted belief,” in
14th International Joint Conference on Artificial Intelligence
, vol. 95, 1995, pp. 1896–1901.  [13] T. Denœux, “Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of evidence,” Artificial Intelligence, vol. 172, no. 2, pp. 234–264, 2008.
 [14] ——, “A –nearest neighbor classification rule based on dempstershafer theory,” Systems, Man and Cybernetics, IEEE Transactions on, vol. 25, no. 5, pp. 804–813, 1995.
 [15] D. Dubois and H. Prade, “Consonant approximation of belief functions,” International Journal of Approximate Reasoning, vol. 4, no. 56, pp. 419–449, 1990.
 [16] K. Zhou and A. Martin, ibelief: Belief Function Implementation, 2015, R package version 1.2. [Online]. Available: http://CRAN.Rproject.org/package=ibelief