Filtering ASVs/OTUs via Mutual Information-Based Microbiome Network Analysis

Microbial communities are widely studied using high-throughput sequencing techniques, such as 16S rRNA gene sequencing. These techniques have attracted biologists as they offer powerful tools to explore microbial communities and investigate their patterns of diversity in biological and biomedical samples at remarkable resolution. However, the accuracy of these methods can negatively affected by the presence of contamination. Several studies have recognized that contamination is a common problem in microbial studies and have offered promising computational and laboratory-based approaches to assess and remove contaminants. Here we propose a novel strategy, MI-based (mutual information based) filtering method, which uses information theoretic functionals and graph theory to identify and remove contaminants. We applied MI-based filtering method to a mock community data set and evaluated the amount of information loss due to filtering taxa. We also compared our method to commonly practice traditional filtering methods. In a mock community data set, MI-based filtering approach maintained the true bacteria in the community without significant loss of information. Our results indicate that MI-based filtering method effectively identifies and removes contaminants in microbial communities and hence it can be beneficial as a filtering method to microbiome studies. We believe our filtering method has two advantages over traditional filtering methods. First, it does not required an arbitrary choice of threshold and second, it is able to detect true taxa with low abundance.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 19

04/28/2018

Fuzzy logic based approaches for gene regulatory network inference

The rapid advancement in high-throughput techniques has fueled the gener...
09/03/2018

Community detection analysis in wind speed-monitoring systems using mutual information-based complex network

A mutual information-based weighted network representation of a wide win...
10/20/2019

Towards Further Understanding of Sparse Filtering via Information Bottleneck

In this paper we examine a formalization of feature distribution learnin...
01/15/2015

Evaluating accuracy of community detection using the relative normalized mutual information

The Normalized Mutual Information (NMI) has been widely used to evaluate...
04/30/2015

Detecting and ordering adjectival scalemates

This paper presents a pattern-based method that can be used to infer adj...
04/03/2020

Characterizing Halloumi cheese bacterial communities through metagenomic analysis

Halloumi is a semi hard cheese produced in Cyprus for centuries and its ...
07/04/2019

The Geometry of Community Detection via the MMSE Matrix

The information-theoretic limits of community detection have been studie...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

High-throughput sequencing approaches are some of the most powerful tools for studying and characterizing microbial communities. Bacterial phylogeny and taxonomy can be characterized using marker genes, such as 16S rRNA gene sequences which are present in all bacteria, and it is sufficiently large for informatics and analysis purposes [20, 30]

. However, the potential for contamination which is defined as non-intended introduction of bacteria during sample collection, DNA extraction, and PCR amplification into the sample of interest is high; thus a low signal-to-noise ratio poses a major problem in analyses of such data

[41, 7, 33]. Contamination is particularly problematic when studying low yield samples because of significant impacts on results [33, 41]

. Therefore, it is necessary to identify, minimize, and filter contaminants as a potential source of bias that leads to skew data analysis.

Attempts to experimentally control or eliminate sources of contamination can be challenging if not impossible. To minimize or identify contamination, strategies such as inclusion of negative controls or blanks for every batch of samples and use them through entire extraction, amplification, or library preparations have been suggested [5, 33]. One of the advantages of sequencing the blanks is the ability to detect and quantify the levels of contamination as well as their the sources. [27, 9, 31, 5, 33]. However, including an appropriate negative control is not always easy and in the majority of microbiome published studies controls have not been included [18]. [41] and [33] recommended keeping records of kits and other reagents, performing technical replicates, and using sample randomization across kits and PCR runs to control measurement error. Some researchers have proposed using mock communities as a positive control during extraction, amplification, and sequencing alongside experimental samples [7]. Positive controls are commercially available in the form of defined communities, however their validity for a particular microbiome research is not guaranteed and standardized protocols for designing positive controls might not be available [18].

None of the above experimental methods are capable of eliminating existing contaminants completely, easily, and reliably in all cases. Therefore, strategies that use the power of bioinformatics and statistical methods to clean sequencing data must be introduced. For example, [21] identified and removed Operational Taxonomic Units (OTUs) as potential contaminants if they have strong negative correlation with amplicon counts after 16S library preparation. However, in many cases, contaminant OTUs might occur on the host as well as being present as contamination and therefore, this leads to a higher than desired false positive rate. Ad hoc methods such as removing genes or taxa with total read count or percentage smaller than or below an empirical threshold across all samples [32, 2, 23, 38, 42] are easy to implement and relatively common among microbiome studies. However, choosing an appropriate filtering threshold is a complex problem by itself and an arbitrary choice can bias the results. In addition, the impact of taxa or genes is not directly proportional to their numeric abundance and there might be biological signal among rare taxa—or genes—that is of interest; thus removing low abundance taxa could lead to loss of important information.

The decontam package in R introduced by [8] has been developed to identify contaminants using statistical models. [8] demonstrated the accuracy of their method to remove contaminants from a data set generated by [33]. However, a major limitation of decontam is that it assumes contaminants and true signals are distinct from one another, and this assumption is violated in the case of cross-contamination due to sequences from pooled samples. [26] developed the R package microDecon which is based on proportions of contaminant OTUs or Amplicon Sequence Variants (ASVs) in blank samples to identify and remove contaminant reads from meta-barcoding data. They demonstrated that their method is robust to both high and low contamination levels. They also showed that their approach can recover the real community from the contaminant community even with a large overlap between the two. However, similar to [8], in case of the existence of cross-contamination, this method is not effective as it assumes a common source of contamination. Recently, [35] introduced the R package PERFect for microbiome filtering using covariance matrices and compared them to traditional filtering procedures. They showed that for a very strong signal, PERFect provides a more effective contaminant reduction when the signal-to-noise ratio is high. A limitation of their methods is that it is skewed toward retaining dominant taxa, however, this is a common limitation among any filtering methods that does not take into account other types of information such as knowledge about blanks or negative controls.

Here, we propose and validate a method to identify and remove non-bacterial signals that are observed due to contamination or sequencing errors in microbiome data. We use the fact that bacteria live in communities where they rely on one another, and their interactions or coexistence are major drivers of microbial community and function. We utilize a graph model to represent and characterize these interactions and/or coexistence by assuming each taxon is a node and pairwise-bacterial associations are edges in this biological network. We use an information theoretic functional to estimate the strength of these interactions and remove isolated taxa that are not informative to the network as potential noise. We apply permutation and bootstrap based hypothesis testing to measure the probability of increase in information loss due to taxa removal is random. We validate our method using the

[7] mock community data set. Next, we compare the performance of commonly used ad hoc filtering methods with our proposed method. Finally, we apply our method to a gut microbiome data set.

The rest of this paper is organized as follows. In Section 2, we introduce our filtering method using graph models and information loss measurement. Statistical inference based on bootstrap and permutation hypothesis testing is presented in Section 2. Method validation and comparison with traditional filtering methods using [7] data set are provided in Section 3 and 4, respectively. Finally we conclude the paper on Section 5.

2 Materials and Methods

We propose a method to identify and remove contaminated sequence reads from data sets, while accounting for the amount of information loss due to this removal. Note that the proposed method can be applied to both OTU and ASV count tables.

2.1 Mathematical Definition

Here, we define notations which will be frequently used in the following sections. Consider a high-dimensional count matrix where each input represents the count of sequence reads of a taxon, which, for simplicity, we will assume to be a bacterial species or strain. Let be a microbial abundance matrix. For each and , let be the observed count of the -th taxon in the -th sample and denotes the abundance of the -th taxon across all samples. Generally, the number of samples is considerably less than the number of taxa, that is .

2.2 The Proposed Method: Network-based Contaminant Identification

Graph theory is an important concept in statistics and can be used to describe the relationships between random variables

[39, 24]. A network (or a graph) is defined as a set of nodes connected by edges [28]. Microbial interactions can be represented as a connectivity network, where nodes correspond to taxa and the edges represent the associations between taxa [40]. One potential association measure is mutual information (MI) which is a non-directional connectivity measure. MI was introduced by Shannon in 1948 [34] as a measure of statistical dependence between two random variables. Unlike Pearson or Spearman correlation coefficients, the most widely used association measures, that quantify linear and monotonic relationships, respectively, MI can be used to estimate non-linear relationships [36, 10].

MI measures the expected reduction in uncertainty about that results from learning , or vice versa. This quantity can be formulated as

(1)

where , known as “entropy,” is the average amount of information, or surprise, a variable has. It is defined to be

(2)

where is the probability of observing the -th value of the bin measurement data using partition-based methods such as histograms. The conditional entropy is the uncertainty of given and it is formulated as

(3)

where is the joint probability density of measurements and .
From equation (3) we can derive the following identity

(4)

where is the joint entropy which measures the amount of uncertainty in the two random variables and taken together.

MI possesses the following desirable properties.

  1. It is symmetric: ,

  2. , equality holds if and only if the two variables are independent,

  3. .

In situations where is uniquely determined by , knowledge of dictates a single possible value of . It then follows that the conditional entropy satisfies and therefore MI has the maximum value of . Moreover, the stronger the relationship between two variables, the greater is the MI. Kinney and Atwal [22] proved that MI places the same importance on linear and nonlinear dependence.

Here, we use MI as an association measure and transform it into network adjacencies. A network adjacency satisfies the following conditions:

  1. ,

  2. ,

  3. .

For taxa an adjacency matrix is a by matrix where each entry is the amount of information shared between each pair of taxa. We construct our adjacency matrix based on MI by satisfying three above conditions: 1) transformation to ; 2) symmetrization; and, 3) setting diagonal values to 1. It can be easily seen that MI is bounded below by 0 and it is symmetric. However, it is not bounded above by 1 and the diagonals are not equal to but rather are the entropy of the variable, . To satisfy the above conditions, we divide each entry of the mutual information matrix by one of its upper bound which is a joint entropy between each pair of taxa, resulting in adjusted adjacency matrix .

Therefore, for each pair of taxa and , the adjusted mutual information is calculated as

(5)

The result of this transformation is by matrix where each entry varies between and . Also, if , then and so . Thus our transformation (5) satisfies the conditions of a network adjacency.

In the following subsection, we describe an approach that results in an unweighted adjacency matrix based on adjusted mutual information measure we defined above.

2.2.1 Filtering Using Unweighted Network Adjacency

A filtered unweighted network adjacency between taxa and can be defined by hard thresholding the adjusted mutual information-based adjacency matrix using signum function.

(6)

where

is the hard threshold parameter. Hard thresholding leads to the intuitive concept of taxa connectivity (i.e., a binary variable indicating whether two species do or do not interact), and it is commonly used to construct a sparse covariance matrices

[43, 37].

Choosing The Threshold

In many biological networks, hard thresholding of the association adjacency matrix is based on the scale-free criteria (defined below) of a graph and often applied when [3, 1, 44]. In other words, it is assumed that the probability that a node is connected with other nodes (the degree distribution of a network) is characterized by a power-law distribution

(7)

where is the node degree, and is some exponent reported in some biological graphs to be [4]. We choose the threshold by fitting a linear function to the empirical degree distribution in log space and estimating the coefficient of variation, (), of the fit. We choose the threshold that results in the highest value . In addition to high values, it is recommended [43, 37] to have a high mean connectivity so that the network contains enough information. We compute the mean degree for each threshold , by taking the average over the degree of all nodes. It is expressed as follows

We use mean connectivity as a tie breaker for thresholds that could produce the same value. Choosing an appropriate threshold which provides us with the highest and a high , we build our network based on and remove isolated nodes (taxa), i.e., nodes that have connectivity degree of 0. Because isolated nodes do not share information with other taxa, we assume they are potential contaminants, and we may remove them without significant loss of information. Conversely, nodes (taxa) that create non-trivial subgraphs (i.e., subgraphs having more than one node) are assumed to be true taxa.

2.3 Subnetworks With Minimal Information Loss

Hidaka et al. [17] proposed a method of searching graph partitions (separations of the vertex set) which leads to the minimal information loss. In another work, Smirnova et al. [35] proposed a filtering loss measure to remove taxa with insignificant contribution to the total covariance. Inspired by the these ideas, we propose a method to filter taxa in a network based on total mutual information.

To do this, first we define the connectivity degree of the -th node for in the weighted graph; this is the sum of the weights on all edges adjacent to node . The formula for connectivity degree is

(8)

where we take to be the weight on the edge connecting nodes and . Next, we sort the connectivity degree

in an increasing order. Following this, we remove nodes (taxa) based on sample quantile values of sorted connectivity degrees for all taxa

. Finally, we compute the information loss according to the following formula:

(9)

where is the Frobenius norm, sometimes also called the Euclidean norm, is the adjusted mutual information matrix after removing all taxa below the quantile. Here, represents the total information shared between taxa after removing certain number of taxa.

2.4 Statistical Inference: Hypothesis Testing

2.4.1 Hypothesis Testing Using a Permutation Test

In this subsection we present an algorithm based on permutation testing, described in Algorithm 1, inspired by François et al. [14] to compare the difference in information loss due to various quantile removal. Let be the quantile values. We are interested in testing if the information loss by removing the taxa with degree less than is significantly different from removing taxa with degree less than , i.e., . Permutation test is a non-parametric hypothesis test [16]

and is commonly used to assess the statistical significance when the distribution of the test statistic is not known and needs to be empirically derived. Here, we introduce essential notations for Algorithm 1.

For all , define to be the after removing taxa with degree less than , and let be the number of taxa removed. Let . If is any subset of the columns of the full OTU table, define as the adjusted mutual information matrix of .

Input: significance level , number of permutations

, quantile vector

.
1 Define : .
2 Calculate .
3 Calculate .
4 For permutation
  1. Randomly shuffle columns of taxonomy count, call this matrix .

  2. For each k:

    1. Remove the first columns from .

    2. Compute using .

  3. Calculate the -th column of , such that

For each , compute , where is the -th row of .
5 Calculate p-values: .
6 Find the index of the first entry of that is less than or equal to . Call this index .
7 Remove all taxa with degree less than .
Algorithm 1 Algorithm 1

2.4.2 Hypothesis Testing Using Bootstrap

In previous subsection we described a permutation test as a useful hypothesis testing tool. Here we use bootstrap methods [13], Algorithm 2, to test the same hypothesis. Again, we specifically wish to test . Similar to permutation tests, a bootstrap hypothesis test is based on a test statistic. Here, we introduce essential notations for Algorithm 2. Let be quantile values. Let for and be the taxa count matrix. For all , define to be the columns of after removing taxa with degree less than . Define to be the subset of the columns of corresponding to the columns of . Let be the covariance matrix of and be the number of taxa in . Consider the test statistic

(10)

We next describe our bootstrap process and bootstrap test statistic . For each and , sample columns with replacement from and name this matrix . In addition, We denote the first columns of , and the remaining columns . Let () and () be the covariance (adjusted mutual information) matrices of and , respectively. Let be the adjusted mutual information matrix of and define,

(11)

Lastly, define our bootstrap test statistic to be

(12)
1 Input: significance level , the number of bootstrap samples , quantile vector
For each k:
  1. Calculate (Eq. 10).

  2. for each :

    1. Generate , , , and Calculate (Eq. 12).

  3. Compute ,

3 ().
4 Find the index of the first entry of that is less than or equal to . Call this index .
5 Remove all taxa with total abundance less than .
Algorithm 2 Algorithm 2

3 Evaluating Filtering Method

3.1 Mock Microbial Community

To test our method, we used a publicly available mock community data set given in Brooks et al. [7] where the ground-truth was known. These data consist of prescribed proportions of cells from seven vaginally-relevant bacterial strains: Atopobium vaginae, Gardnerella vaginalis, Lactobacillus crispatus, Lactobacillus iners, Prevotella bivia, Sneathia amnii, and Streptococcus agalactiae to quantify and characterize bias introduced in the sample processing pipeline such as DNA extraction, PCR amplification, and sequencing classification.

We start by constructing an unweighted network of vaginal microbiome data. Table 1 and Figure 1 report the results for varying the threshold parameter for the mock community data. It can be seen that the coefficient of determination clearly favors . Based on these results, we use to construct the unweighted network.

In Figure 2 we have established the adjusted mutual information unweighted network of this dataset. It can be seen that can reflect the true connection between the microbiome as a subnetwork and majority of noise taxa are indicated as isolated nodes. In addition, we can define the weighted network where weights are adjusted mutual information (), this is shown in Figure 3. It can be seen that in these types of networks all the nodes are connected to all other nodes. Here, edges are colored based on the strength of the connectivity between adjacent nodes from very weak (light grey), moderate (grey), strong (black). Notice that the weight between majority of true taxa is strong, however we can see three subnetworks of noise that strongly share information.

3.1.1 Receiver Operator characteristic (ROC)

Here we use ROC curve to evaluate the classification accuracy of each taxon in this data set using a thresholding parameter in reference to the binary outcome , which takes (noise taxon) or (true taxon). In order to do this, we measure the degree of each node (taxon) in our unweighted network obtained by different hard thresholding parameter . For each taxon, convention dictates that a true taxon is defined as

. The classification accuracy of each taxon is then evaluated by considering a confusion matrix. It cross-classifies the predicted outcome for taxon with

versus the true outcome . For the fixed cutoff , the true positive fraction is the probability of identifying a taxon as a true signal, when it is truly a taxon.

In general, ROC analysis assesses the trade-offs between the test’s fraction of true positives versus the false positives as varies over the range of to .

and the false positive fraction is the probability of identifying a taxon as a true signal, while it is a noise taxon.

Since is not fixed in advance, one can plot TPF (sensitivity) y-axis against FPF(1-specificity) x-axis for all possible values of . If , for all , it is a useless test regards to the binary prediction. A perfect test that is completely informative about the signal status has and for at least one value . In other words, an excellent model has area under the ROC curve near which indicates a good measure of separability. A model with area near indicates a good measure of separability but a poor classification accuracy. An area under the ROC curve of means model has no class separation ability and is considered to be a random classifier. Figure 4 shows the ROC curve, assessing true versus false positive rate with AUC = 0.86 demonstrates the good performance of the method.

m=46 Predicted: True Taxa Predicted: Noise Taxa
Actual True True Positive False Negative

We use [7] data set to estimate the amount of information loss by removing different percentages of taxa and investigate if the difference in information loss by removing percentages of taxa with degree less than versus is significant. Figure 5 illustrates the results for this mock community data set.The left panel displays the information loss and the right panel displays the difference in information loss. The data set was sorted according to the increasing connectivity degree of taxa using adjusted mutual information adjacency matrix. For example, a cutoff assignment of removes of the Taxa with the lowest connectivity degree. It is clear that applying percentile based filtering changes the amount of information loss, Figure 5. For example, information loss has a drastic increase from to filtering threshold, while there is no or minimal change in information loss between removing and of taxa. This provides us with the intuition that of taxa can be removed from the further analysis without loosing significant amount of information and these taxa could be the result of sequencing or PCR error, especially in high-throughput sequencing data sets.

Figure 6 shows information loss versus the number of taxa that are removed based on the lowest connectivity degree one at a time. We can see that after removing true signals (indicated by taxonomic name) the information loss values increase dramatically. From the Figures 5 and 6, it is clear that information loss increases after removing a certain number or percentage of taxa. However, we need to investigate whether this rise of information loss is due to random errors or a real effect. In other word, we want to determine whether the information loss after removing less than of taxa is significantly different from information loss after removing of taxa, where is the

quantile value of connectivity degree. To do this, we use permutation and bootstrapping approaches to test the null hypothesis which indicates that removing taxa with degree less than

versus does not make any difference in information loss and hence we can remove them from further analysis. We follow the Algorithm 1 and 2 by setting , , , and . The results of permutation and bootstrapping tests are shown in Table 2, second column shows that there is a significant loss of information after removing of taxa (p-value) at significance level. As an alternative, we follow Algorithm 2 to apply bootstrap method for hypothesis testing to approximate p-value. Table 2, third column shows that the bootstrap test gives similar results to permutation tests which also indicates that there is strong evidence that removing of taxa will result in loosing significant amount of information.

4 Comparison Study on Mock Community Data [7]

Here, we use data set in [7] to assess the performance of our method and compare results to alternative methods. More specifically, we consider four traditional methods which have been commonly employed for filtering of microbiome data: (1) Traditional 1: we retain taxa with more than , , and relative abundance in at least one sample [11, 25, 29, 15]. (2) Traditional 2: we retain taxa with at least 5 reads in at least 3 samples [19]. (3) Traditional 3: we retain taxa presented in more than 5 samples [6]. (4) Traditional 4: we remove samples with fewer than 100 reads and taxa with fewer than 10 reads, as well as taxa which present in fewer than of samples [12].

Results presented in Figure 7 and Table 3 indicated that MI-based filtering method performs better than Traditional 2, 3, and 4 as well as Traditional 1 for the choice of . In particular, we can see than MI-based method removed of taxa with minimum loss of information and preserved of taxa which were all true signals (). Traditional 1 filtering method with retain threshold of and performed as good as MI-based method, however, when retain threshold was this method retained of contamination. In Traditional 2 filtering method, following filtering of taxa, this method retained noise signals, Similarly in Traditional 3 and 4, they retained around and non-bacterial signals, respectively. Therefore, proposed method was successfully able to identify and remove contaminants from this data which result in dimensionality reduction of the data that can reduce computation time and improve interpretation of the downstream analysis. The proposed method has two advantages in addition to its superior performance in comparison with above mentioned traditional methods. First, it does not required a choice of threshold which is critical and not easy to obtain. Second, it is able to detect true taxa with low abundance. Most of the traditional methods have subjective predetermined thresholding value that might have adverse effects on the analysis due to loss of important information within filtered taxa. Our proposed method choose filtering threshold based on hypothesis testing and information loss. As mentioned earlier, these traditional methods remove taxa with low abundance and hence any important taxa with low abundance is removed leading to significant loss of information. However, MI-based filtering method, removes taxa based on their interactions with other taxa and therefore it can preserve low abundance taxa in case of their strong association with other taxa. This allows us to study significant taxa that occur in low abundance.

5 Conclusion

Removing contaminants prior to any downstream analysis is an essential step in metagenomic sequencing data research. Host associated contaminants significantly complicate analysis, particularly in low microbial biomass body sites. Contamination can cause analysis of sequencing reads to result in false positive or false negative and hence decreasing the reliability of the analysis. Here, we developed a simple method that uses combination of graph theory and information theoretic functionals to identify and remove contaminants in metagenomic data sets. Our results suggest that mutual information based filtering method can improve the accuracy of detecting contaminants, especially in comparison with the commonly used traditional filtering methods.

To fully explore the strengths and weaknesses of our proposed filtering method, evaluation on different labeled mock community data sets are necessary. Unfortunately, labeled data sets are expensive and difficult to obtain and hindered our ability to test our method further. We believe that it is possible to improve the threshold selection in the unweighted graph given improved (ground-truthed) data with which to work. Looking solely for isolated nodes is not sufficient as it is unable to filter out random interaction between contaminants nodes. When the number of nodes increases we can expect more random interaction between contaminants, making the isolated node approach even less powerful. We believe more advanced methods from graph theory could remedy this short coming. Future work could include but not limited to examination the efficiency of techniques such as dense community detection, dense subgraph selection, and vertex selection based on vertex centrality. These sophisticated node selection methods could provide a more powerful filtering method in the unweighted graph.

References

  • Albert [2005] Albert, R., 2005: Scale-free networks in cell biology. Journal of Cell Science, 118(21), 4947–4957.
  • Anders and Huber [2010] Anders, S. and W. Huber, 2010: Differential expression analysis for sequence count data. Genome Biology, 11(10), R106.
  • Barabási and Albert [1999] Barabási, A.-L. and R. Albert, 1999: Emergence of scaling in random networks. Science, 286(5439), 509–512.
  • Barabási and Oltvai [2004] Barabási, A.-L. and Z. N. Oltvai, 2004: Network biology: understanding the cell’s functional organization. Nature Reviews Genetics, 5(2), 101–113.
  • Barton et al. [2006] Barton, H. A., N. M. Taylor, B. R. Lubbers, and A. C. Pemberton, 2006: DNA extraction from low-biomass carbonate rock: An improved method with reduced contamination and the low-biomass contaminant database. Journal of Microbiological Methods, 66(1), 21–31.
  • Brigham et al. [2001] Brigham, A., E. Sadorf, and G. S. (U.S.), 2001: Benthic Invertebrate Assemblages and Their Relation to Physical and Chemical Characteristics of Streams in the Eastern Iowa Basins, 1996-98. Water-resources investigations report, U.S. Department of the Interior, U.S. Geological Survey.
  • Brooks et al. [2015] Brooks, J. P., D. J. Edwards, M. D. Harwich Jr, M. C. Rivera, J. M. Fettweis, M. G. Serrano, R. A. Reris, N. U. Sheth, B. Huang, P. Girerd, V. M. Consortium, J. F. Strauss 3rd, K. K. Jefferson, and G. A. Buck, 2015: The truth about metagenomics: quantifying and counteracting bias in 16S rRNA studies. BMC microbiology, 15, 66.
  • Davis et al. [2018] Davis, N. M., D. M. Proctor, S. P. Holmes, D. A. Relman, and B. J. Callahan, 2018: Simple statistical identification and removal of contaminant sequences in marker-gene and metagenomics data. Microbiome, 6(1), 226.
  • de Goffau et al. [2018] de Goffau, M. C., S. Lager, S. J. Salter, J. Wagner, A. Kronbichler, D. S. Charnock-Jones, S. J. Peacock, G. C. S. Smith, and J. Parkhill, 2018: Recognizing the reagent microbiome. Nature Microbiology, 3(8), 851–853.
  • Dionisio et al. [2004] Dionisio, A., R. Menezes, and D. A. Mendes, 2004: Mutual information: a measure of dependency for nonlinear time series. Physica A: Statistical Mechanics and its Applications, 344(1), 326–329, applications of Physics in Financial Analysis 4 (APFA4).
  • Dobbler et al. [2019] Dobbler, P., V. Mai, R. S. Procianoy, R. C. Silveira, A. L. Corso, and L. F. W. Roesch, 2019: The vaginal microbial communities of healthy expectant Brazilian mothers and its correlation with the newborn’s gut colonization. World journal of microbiology & biotechnology, 35(10), 159.
  • Duvallet et al. [2017] Duvallet, C., S. M. Gibbons, T. Gurry, R. A. Irizarry, and E. J. Alm, 2017: Meta-analysis of gut microbiome studies identifies disease-specific and shared responses. Nature Communications, 8(1), 1784.
  • Efron and Tibshirani [1994] Efron, B. and R. Tibshirani, 1994: An Introduction to the Bootstrap. Chapman & Hall/CRC Monographs on Statistics & Applied Probability, CRC Press, ISBN 9781000064988.
  • François et al. [2006] François, D., V. Wertz, and M. Verleysen, 2006:

    The permutation test for feature selection by mutual information

    .
  • Gliniewicz et al. [2019] Gliniewicz, K., G. M. Schneider, B. J. Ridenhour, C. J. Williams, Y. Song, M. A. Farage, K. Miller, and L. J. Forney, 2019: Comparison of the vaginal microbiomes of premenopausal and postmenopausal women. Frontiers in Microbiology, 10, 193.
  • Good [1994] Good, P., 1994: Permutation tests: a practical guide to resampling methods for testing hypotheses. Springer series in statistics, Springer-Verlag, ISBN 9780387940977.
  • Hidaka and Oizumi [2018] Hidaka, S. and M. Oizumi, 2018: Fast and exact search for the partition with minimal information loss. PLOS ONE, 13(9), 1–14.
  • Hornung et al. [2019] Hornung, B. V. H., R. D. Zwittink, and E. J. Kuijper, 2019: Issues and current standards of controls in microbiome research. FEMS Microbiology Ecology, 95(5).
  • Ingham et al. [2019] Ingham, A. C., K. Kielsen, M. S. Cilieborg, O. Lund, S. Holmes, F. M. Aarestrup, K. G. Müller, and S. J. Pamp, 2019: Specific gut microbiome members are associated with distinct immune markers in pediatric allogeneic hematopoietic stem cell transplantation. Microbiome, 7(1), 131.
  • Janda and Abbott [2007] Janda, J. M. and S. L. Abbott, 2007: 16S rRNA Gene Sequencing for Bacterial Identification in the Diagnostic Laboratory: Pluses, Perils, and Pitfalls. Journal of Clinical Microbiology, 45(9), 2761 LP – 2764.
  • Jervis-Bardy et al. [2015] Jervis-Bardy, J., L. E. X. Leong, S. Marri, R. J. Smith, J. M. Choo, H. C. Smith-Vaughan, E. Nosworthy, P. S. Morris, S. O’Leary, G. B. Rogers, and R. L. Marsh, 2015: Deriving accurate microbiota profiles from human samples with low bacterial content through post-sequencing processing of Illumina MiSeq data. Microbiome, 3(1), 19.
  • Kinney and Atwal [2014] Kinney, J. B. and G. S. Atwal, 2014: Equitability, mutual information, and the maximal information coefficient. Proceedings of the National Academy of Sciences, 111(9), 3354–3359.
  • Law et al. [2014] Law, C. W., Y. Chen, W. Shi, and G. K. Smyth, 2014: voom: precision weights unlock linear model analysis tools for RNA-seq read counts. Genome Biology, 15(2), R29.
  • Li et al. [2016] Li, L., Z. Wang, P. He, S. Ma, J. Du, and R. Jiang, 2016: Construction and Analysis of Functional Networks in the Gut Microbiome of Type 2 Diabetes Patients. Genomics, Proteomics & Bioinformatics, 14(5), 314–324.
  • Logares et al. [2014] Logares, R., S. Audic, D. Bass, L. Bittner, C. Boutte, R. Christen, J.-M. Claverie, J. Decelle, J. Dolan, M. Dunthorn, B. Edvardsen, A. Gobet, W. Kooistra, F. Mahé, F. Not, H. Ogata, J. Pawlowski, M. Pernice, S. Romac, K. Shalchian-Tabrizi, N. Simon, T. Stoeck, S. Santini, R. Siano, P. Wincker, A. Zingone, T. Richards, C. de Vargas, and R. Massana, 2014: Patterns of Rare and Abundant Marine Microbial Eukaryotes. Current Biology, 24(8), 813–821.
  • McKnight et al. [2019] McKnight, D. T., R. Huerlimann, D. S. Bower, L. Schwarzkopf, R. A. Alford, and K. R. Zenger, 2019: microDecon: A highly accurate read-subtraction tool for the post-sequencing removal of contamination in metabarcoding studies. Environmental DNA, 1(1), 14–25.
  • Minich et al. [2018] Minich, J. J., Q. Zhu, S. Janssen, R. Hendrickson, A. Amir, R. Vetter, J. Hyde, M. M. Doty, K. Stillwell, J. Benardini, J. H. Kim, E. E. Allen, K. Venkateswaran, and R. Knight, 2018: KatharoSeq Enables High-Throughput Microbiome Analysis from Low-Biomass Samples. mSystems, 3(3), e00218–17.
  • Naqvi et al. [2010] Naqvi, A., H. Rangwala, A. Keshavarzian, and P. Gillevet, 2010: Network-based modeling of the human gut microbiome. Chemistry & biodiversity, 7(5), 1040–1050.
  • Partula et al. [2019] Partula, V., S. Mondot, M. J. Torres, E. Kesse-Guyot, M. Deschasaux, K. Assmann, P. Latino-Martel, C. Buscail, C. Julia, P. Galan, S. Hercberg, V. Rouilly, S. Thomas, L. Quintana-Murci, M. L. Albert, D. Duffy, O. Lantz, M. Touvier, and t. M. I. Consortium, 2019: Associations between usual diet and gut microbiota composition: results from the Milieu Intérieur cross-sectional study. The American Journal of Clinical Nutrition, 109(5), 1472–1483.
  • Patel [2001] Patel, J. B., 2001: 16S rRNA gene sequencing for bacterial pathogen identification in the clinical laboratory. Molecular Diagnosis, 6(4), 313–321.
  • Pollock et al. [2018] Pollock, J., L. Glendinning, T. Wisedchanwet, and M. Watson, 2018: The Madness of Microbiome: Attempting To Find Consensus “Best Practice” for 16S Microbiome Studies. Applied and Environmental Microbiology, 84(7), e02627–17.
  • Robinson and Oshlack [2010] Robinson, M. D. and A. Oshlack, 2010: A scaling normalization method for differential expression analysis of RNA-seq data. Genome Biology, 11(3), R25.
  • Salter et al. [2014] Salter, S. J., M. J. Cox, E. M. Turek, S. T. Calus, W. O. Cookson, M. F. Moffatt, P. Turner, J. Parkhill, N. J. Loman, and A. W. Walker, 2014: Reagent and laboratory contamination can critically impact sequence-based microbiome analyses. BMC Biology, 12(1), 87.
  • Shannon [1948] Shannon, C. E., 1948: A mathematical theory of communication. Bell System Technical Journal, 27(3), 379–423.
  • Smirnova et al. [2018] Smirnova, E., S. Huzurbazar, and F. Jafari, 2018: PERFect: PERmutation Filtering test for microbiome data. Biostatistics, 20(4), 615–631.
  • Song et al. [2012] Song, L., P. Langfelder, and S. Horvath, 2012: Comparison of co-expression measures: mutual information, correlation, and model based indices. BMC Bioinformatics, 13(1), 328.
  • Sulaimanov and Koeppl [2016] Sulaimanov, N. and H. Koeppl, 2016: Graph reconstruction using covariance-based methods. EURASIP Journal on Bioinformatics and Systems Biology, 2016(1), 19.
  • Sultan et al. [2008] Sultan, M., M. H. Schulz, H. Richard, A. Magen, A. Klingenhoff, M. Scherf, M. Seifert, T. Borodina, A. Soldatov, D. Parkhomchuk, D. Schmidt, S. O'Keeffe, S. Haas, M. Vingron, H. Lehrach, and M.-L. Yaspo, 2008: A Global View of Gene Activity and Alternative Splicing by Deep Sequencing of the Human Transcriptome. Science, 321(5891), 956 LP – 960.
  • Sun et al. [2018] Sun, C., F. Yang, C. Wang, Z. Wang, Y. Zhang, D. Ming, and J. Du, 2018: Mutual information-based brain network analysis in post-stroke patients with different levels of depression. Frontiers in Human Neuroscience, 12, 285.
  • Tavakoli and Yooseph [2019] Tavakoli, S. and S. Yooseph, 2019: Learning a mixture of microbial networks using minorization–maximization. Bioinformatics, 35(14), i23–i30.
  • Weiss et al. [2014] Weiss, S., A. Amir, E. R. Hyde, J. L. Metcalf, S. J. Song, and R. Knight, 2014: Tracking down the sources of experimental contamination in microbiome studies. Genome Biology, 15(12), 564.
  • Xia et al. [2018] Xia, Y., J. Sun, and D. Chen, 2018: Statistical Analysis of Microbiome Data with R. ICSA Book Series in Statistics, Springer Singapore, ISBN 9789811315343.
  • Zhang and Horvath [2005a] Zhang, B. and S. Horvath, 2005a: A general framework for weighted gene co-expression network analysis. Statistical Applications in Genetics and Molecular Biology, 4(1).
  • Zhang and Horvath [2005b] —, 2005b: A general framework for weighted gene co-expression network analysis. Statistical Applications in Genetics and Molecular Biology, 4(1).