DeepAI
Log In Sign Up

Convolutional neural networks for structured omics: OmicsCNN and the OmicsConv layer

Convolutional Neural Networks (CNNs) are a popular deep learning architecture widely applied in different domains, in particular in classifying over images, for which the concept of convolution with a filter comes naturally. Unfortunately, the requirement of a distance (or, at least, of a neighbourhood function) in the input feature space has so far prevented its direct use on data types such as omics data. However, a number of omics data are metrizable, i.e., they can be endowed with a metric structure, enabling to adopt a convolutional based deep learning framework, e.g., for prediction. We propose a generalized solution for CNNs on omics data, implemented through a dedicated Keras layer. In particular, for metagenomics data, a metric can be derived from the patristic distance on the phylogenetic tree. For transcriptomics data, we combine Gene Ontology semantic similarity and gene co-expression to define a distance; the function is defined through a multilayer network where 3 layers are defined by the GO mutual semantic similarity while the fourth one by gene co-expression. As a general tool, feature distance on omics data is enabled by OmicsConv, a novel Keras layer, obtaining OmicsCNN, a dedicated deep learning framework. Here we demonstrate OmicsCNN on gut microbiota sequencing data, for Inflammatory Bowel Disease (IBD) 16S data, first on synthetic data and then a metagenomics collection of gut microbiota of 222 IBD patients.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

09/06/2017

Phylogenetic Convolutional Neural Networks in Metagenomics

Background: Convolutional Neural Networks can be effectively used only w...
07/14/2017

Generalizing the Convolution Operator in Convolutional Neural Networks

Convolutional neural networks have become a main tool for solving many m...
08/05/2022

Isoform Function Prediction Using Deep Neural Network

Isoforms are mRNAs produced from the same gene site in the phenomenon ca...
12/31/2019

AdderNet: Do We Really Need Multiplications in Deep Learning?

Compared with cheap addition operation, multiplication operation is of m...
10/30/2014

DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks

This paper introduces a visual sentiment concept classification method b...
08/03/2022

DeepProphet2 – A Deep Learning Gene Recommendation Engine

New powerful tools for tackling life science problems have been created ...

1 Introduction

The diversity and the complex structure of biological data represent both a challenge and an opportunity Deep Learning (DL) techniques Ching et al. (2017). In fact, the superiority of deep neural network approaches is demonstrated on a growing number of biological and clinical tasks Mamoshina et al. (2016); Chaudhary et al. (2017); Zacharaki (2017). Nonetheless, the DL potential is clearly being limited on omics data: open issues are transversal to other domains, e.g.

, limited availability of data, interpretability, hyperparameters tuning 

Min et al. (2016), but overall there is a need for architectures designed to use biological knowledge and underlying structure. As a consequence, the direct application of deep networks to microarray, genetic or other omics can be disappointing, as found by Ditzler and coauthors Ditzler et al. (2015) that claimed that deep learning ”may not be suitable for metagenomic application”.

In order to bridge this gap, we propose a short path to the application of Convolutional Neural Network (CNNs) architectures to omics data. CNNs is the elective solution for image classification LeCun et al. (1998); Krizhevsky et al. (2012), based on convolving subsets of the input image with different filters, where the key concept is the definition of a set of nearest neighbours for a given pixel. Using the same architecture for non-image data requires the availability of an analogous proximity measure between features.

As a first step, we show how to embed omics data in a metric space: in particular, we consider the case of metagenomics and transcriptomics. In the former case, the metric is a tree distance on the phylogenetic tree, while in the latter case we propose a more complex structure based on the diffusion distance on a 4 layers multilayer network De Domenico (2017). Given the distance, the neighbour detection comes naturally. The convolutional layer combined with the neighbours detection algorithm is operatively implemented as a novel Keras layer Chollet (2015) called OmicsConv. The overall architecture OmicsCNN then consists of a stack of OmicsConv layers first flattened then terminating with a Fully Connected (Dense) and a final classification layer. The environment is completed by an experimental setup for unbiased prediction, i.e., a 105-fold Data Analysis Protocol (DAP) developed for unbiased biomarker identification within the US-FDA led initiatives MAQC/SEQC The MicroArray Quality Control (MAQC) Consortium (2010); The SEQC/MAQC-III Consortium (2014).

As a case study, we conclude with a comparison of the OmicsCNN DAP on the metagenomics IBD dataset Sokol et al. (2017) with other classical and DL algorithm, where the OmicsCNN is applied in a domain adaptation strategy, freezing the first layers after they have been trained on a large synthetic dataset. Further details on the metagenomics case are described in Fioravanti et al. (2017).

Figure 1: The structure of OmicsCNN. In this configuration, OmicsCNN is composed by two OmicsConv layers followed by a Fully Connected layer before decision.

2 Metrizing omics data

Differently from images, a natural distance between features is not immediately defined for most of omics data types. Although an indirect transformation into images of outcomes from sequencing has been proposed, we seek here to start a principled development of methods that can apply to some of the most interesting omics data types for clinical application and scientific discovery.

Metagenomics data are usually described in terms of operational taxonomic units (OTUs), a general definition to identify groups of closely related individuals (e.g. species, genus, …). OTUs have a well-defined structure and this provides our first example because a proximity measure can indeed be inherited by the tree structure connecting the OTUs. In general terms any tree distance can be chosen as base metric for the convolution functions; here we demonstrate our approach on the patristic distance (see Fig. 2(a)), i.e., the sum of the lengths of all branches connecting two OTUs on the phylogenetic tree Stuessy and König (2008).

(a) (b)
Figure 2: (a) Patristic distance on a tree; (b) Diffusion distance for every layer of the multilayer networks for transcriptomics data: the four layers are Absolute Pearson Correlation Coefficient (APCC) and the three Gene Ontology layers Cellular Component (CC), Molecular Function (MF) and Biological Process (BP).

Our second domain is transcriptomic data, i.e. gene expression, for which several definitions of distances between two genes have been proposed. Most of such definitions are taking into account the genetic distance on the genome, or statistics of the gene expression. We need to add that the very fast development of biology of regulation The FANTOM Consortium and the RIKEN PMI and CLST (DGT) (2014) and about the implication of chromatin structure on gene expression Singh et al. (2016), and most recent findings in measuring differences across cell types and functions GTEx Consortium, Lead analysts et al. (2017) make gene expression metrization a moving target. In particular, experiments are currently running on both microarray and RNA-Seq data, with the MAQCII/SEQC Neuroblastoma dataset Zhang et al. (2015); Oberthuer et al. (2015) (498 samples on both platforms) as the first testbed. Here we propose a principled metric for gene expression, relying on the mathematics of multilayer networks Kivelä et al. (2014). The metrics can be applied to other omics data types, and extended to deal with integration. In details, a multilayer undirected weighted network on shared vertices (i.e., an edge-colored graph) is built with genes as the nodes. This graph includes 4 layers: in the first layer, two nodes are connected by an edge if the absolute Pearson correlation between the expression across all samples of the corresponding genes is larger than a significance threshold; in this case, the weight of the edge is given by the absolute value of the correlation coefficient. The other three layers are built according to the three categories of the Gene Ontology, i.e., cellular component, molecular function and biological process Ashburner et al. (2000); The Gene Ontology Consortium (2017). In each layer, a link connects two genes if their semantic similarity (e.g. as defined by GOSemSim Yu et al. (2010)) is not null, and the edge weight is exaclty this non-zero value. Given this multilayer structure, the metric between two genes on each layer is given by the diffusion distance De Domenico (2017) (see Fig. 2(b)), which quantifies the likelihood that two random walkers originating in the two genes meet in some other node of the net. The overall distance in the network is given by the Euclidean product of the diffusion distances of each layer.

Once a omics distance is defined, the concept of nearest neighbours among features comes straightforwardly and the CNN algorithm can be applied. However, by definition, the output of a CNN consists of linear combinations of the original input features: this implies that, after the first CNN layer, the problem of finding the neighbours of a feature becomes the hardest task of finding the neighbours of a linear combination of features. We thus propose to map features into points of a -dimensional metric space preserving distances as well as possible via a MultiDimensional Scaling (MDS) projection Cox and Cox (2001), which can be applied if the starting distance is Euclidean. A further refinement is provided by sparsifying MDS via regularized low rank matrix approximation Shen and Huang (2007) through the addition of the smoothly clipped absolute deviation penalty Fan and Li (2001), tuned by cross-validation.

Figure 3: Data Analysis Protocol for the experimental framework.

3 OmicsCNN and experimental setup

The convolutional layer combined with the neighbours detection algorithm is operatively implemented as a novel Keras layer Chollet (2015) called OmicsConv, which convolves the selected neighbours with the filters. In our metagenomics case study, the number of neighbours and the number of filters are both set to 16. In a generic OmicsConv layer, the structure input is represented by a collection of metafeatures, i.e.

linear combinations of the original features; the first OmicsConv layer has the original set of features as input. The OmicsConv is then repeated; finally, the terminating layers of the OmicsCNN are a MaxPooling, then a Flatten layer and, finally, a Fully Connected with 128 neurons and a 0.25 Dropout. Each convolutional layer has a Scaled Exponential Linear Units (SELU)

Klambauer et al. (2017)

as the activation fuction,and the dense layer in transfer learning experiments uses a sigmoid activation function. Adam 

Kingma and Ba (2014) is used as optimizer with learning rate 0.0005. The global learning architecure is called OmicsCNN, shown in Fig. 1.

To ensure predictive power and limit overfitting effect, the experimental framework is structured following the guidelines recommended by the US-FDA led studies MAQC/SEQC The MicroArray Quality Control (MAQC) Consortium (2010); The SEQC/MAQC-III Consortium (2014) that investigated the development of predictive models for the analysis of high-throughput data. The experimental setup is realized as a 10

5-fold cross-validation schema with a feature selection and ranking procedure, implementing a Data Analysis Protocol (DAP) to control for selection bias and other overfitting effects and warranting honest performance estimates on external validation data subsets. Top ranking features are recursively selected as the

-best at each round, and finally aggregated via Borda algorithm Jurman et al. (2012a)

. Model performance is computed for increasing number of best ranking features by Matthews Correlation Coefficient (MCC), the measure that better convey in an unique value the confusion matrix of a classification task, even in the multiclass case 

Matthews (1975); Baldi et al. (2000); Gorodkin (2004); Jurman et al. (2012b). Experiments with randomized features and labels are also performed as model sanity check.

The OmicsCNN is implemented as a Keras v2.0.8 layer, while the whole DAP is written in Python/Scikit-Learn. All computations were run on a Microsoft Azure platform with 2x NVIDIA Tesla K80 GPUs. A first version of the architecture, dedicated to metagenomics application, together with the IBD and synthetic metagenomics datasets are available at https://gitlab.fbk.eu/MPBA/phylogenetic-cnn.

4 Case study

Sokol’s lab data Sokol et al. (2017) collects microbiome information with 16S sequencing for 38 healthy subjects (HS) and 222 inflammatory bowel disease (IBD) patients. The bacterial composition was analysed using 16S sequencing and a total number of 306 different OTUs was found. IBD is a complex disease arising as a result of the interaction of environmental and genetic factors inducing immunological responses and inflammation in the intestine and primarily including ulcerative colitis (UC) and Crohn’s disease (CD). Both disease classes are characterized by two conditions: flare (f), when symptoms reappear or worsen, and remission (r), when symptoms are reduced or disappear. Finally, since CD can affect different parts of the intestine, we distinguish ileal Crohn’s disease (iCD) and colon Crohn’s disease (CD). Note however,that the number of non zero features varies for the different rom tasks to task, (defined by disease, condition site) since some features may vanish on all samples of a class. Due to the paucity of samples preventing an effective deep learning training phase, a synthetic dataset D was generated to be used as the training set for the domain adaptation task. Operatively, D

is generated as compositional data whose structure resembles the IBD data, sampling 10000 points from multivariate normal distributions with given covariances and mean.

OmicsCNN is first trained on the synthetic dataset D

, then all layers but the last one are freezed, the last layer is substituted by a 2-neurons Dense layer and then retrained on the IBD dataset. The full DAP is then applied to the IBD datasets, comparing the performance with standard (and shallow) learning algorithms such as linear Support Vector Machines (SVM) and Random Forest (RF), and with a standard Multi Layer Perceptron (MLPNN) 

Bishop (1995), obtaining the results listed in Tab. 1.

Task OmicsCNN LSVM MLPNN RF
UCf 0.741 0.740 0.666 0.699
UCr 0.583 0.497 0.608 0.678
CDf 0.858 0.642 0.705 0.707
CDr 0.853 0.654 0.654 0.597
iCDf 0.842 0.418 0.401 0.920
iCDr 0.628 0.414 0.414 0.418
Table 1: Dataset D on IBD: classification performances of OmicsCNN compared to other classifiers on the external validation dataset.

OmicsCNN achieves better results in 4 tasks out of 6, with interesting improvements in the hardest CD/iCD tasks, suggesting even better outcomes are to be expected with more intensive optimization of the components of the proposed OmicsCNN architecture.

5 Conclusions

We introduced here the first steps towards a global plan of making CNNs available to a number of (omics) data types by defining theoretically grounded embedding of features into metric spaces to activate a nearest neighbour function. Encouraging initial results on metagenomics and transcriptomics support further investigations and extension to other metrizable omics structures.

References

  • Ching et al. [2017] T. Ching, D. S. Himmelstein, B. K. Beaulieu-Jones, A. A. Kalinin, B. T. Do, G. P. Way, E. Ferrero, P.-M. Agapow, W. Xie, G. L. Rosen, B. J. Lengerich, J. Israeli, J. Lanchantin, S. Woloszynek, A. E. Carpenter, A. Shrikumar, J. Xu, E. M. Cofer, D. J. Harris, D. DeCaprio, Y. Qi, A. Kundaje, Y. Peng, L. K. Wiley, M. H. S. Segler, A. Gitter, and C. S. Greene. Opportunities And Obstacles For Deep Learning In Biology And Medicine. bioRxiv 142760, 2017.
  • Mamoshina et al. [2016] P. Mamoshina, A. Vieira, E. Putin, and A. Zhavoronkov. Applications of Deep Learning in Biomedicine. Molecular Pharmaceutics, 13(5):1445–1454, 2016.
  • Chaudhary et al. [2017] K. Chaudhary, O. B. Poirion, L. Lu, and L. Garmire. Deep Learning based multi-omics integration robustly predicts survival in liver cancer. bioRxiv 114892, 2017.
  • Zacharaki [2017] E. I. Zacharaki. Prediction of protein function using a deep convolutional neural network ensemble. PeerJ Computer Science, 3:e124, 2017.
  • Min et al. [2016] S. Min, B. Lee, and S. Yoon. Deep learning in bioinformatics. Briefings in Bioinformatics, Advance Access 29 June 2016:bbw068, 2016.
  • Ditzler et al. [2015] G. Ditzler, R. Polikar, and G. Rosen. Multi-Layer and Recursive Neural Networks for Metagenomic Classification. IEEE Transactions on NanoBioscience, 14(6):608–616, 2015.
  • LeCun et al. [1998] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  • Krizhevsky et al. [2012] A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25, pages 1097–1105. Curran Associates, Inc., 2012.
  • De Domenico [2017] M. De Domenico. Diffusion geometry unravels the emergence of functional clusters in collective phenomena. Physical Review Letters, 118(16):168301, 2017.
  • Chollet [2015] F. Chollet. Keras. https://github.com/fchollet/keras, 2015.
  • The MicroArray Quality Control (MAQC) Consortium [2010] The MicroArray Quality Control (MAQC) Consortium. The MAQC-II Project: A comprehensive study of common practices for the development and validation of microarray-based predictive models. Nature Biotechnology, 28(8):827–838, 2010.
  • The SEQC/MAQC-III Consortium [2014] The SEQC/MAQC-III Consortium. A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium. Nature Biotechnology, 32(9):903–914, 2014.
  • Sokol et al. [2017] H. Sokol, V. Leducq, H. Aschard, H.-P. Pham, S. Jegou, C. Landman, D. Cohen, G. Liguori, A. Bourrier, I. Nion-Larmurier, J. Cosnes, P. Seksik, P. Langella, D. Skurnik, M. L. Richard, and L. Beaugerie. Fungal microbiota dysbiosis in IBD. Gut, 66(6):1039–1048, 2017.
  • Fioravanti et al. [2017] D. Fioravanti, Y. Giarratano, V. Maggio, C. Agostinelli, M. Chierici, G. Jurman, and C. Furlanello. Phylogenetic Convolutional Neural Networks in Metagenomics. arXiv:1709.02268, submitted, 2017.
  • Stuessy and König [2008] T. F. Stuessy and C. König. Patrocladistic classification. Taxonomy, 57(2):594–601, 2008.
  • The FANTOM Consortium and the RIKEN PMI and CLST (DGT) [2014] The FANTOM Consortium and the RIKEN PMI and CLST (DGT). A promoter-level mammalian expression atlas. Nature, 507(7493):462–470, 2014.
  • Singh et al. [2016] R. Singh, J. Lanchantin, G. Robins, and Y. Qi. DeepChrome: deep-learning for predicting gene expression from histone modifications. Bioinformatics, 32(17):i639–i648, 2016.
  • GTEx Consortium, Lead analysts et al. [2017] GTEx Consortium, Lead analysts, Laboratory, Data Analysis & Coordinating Center (LDACC), NIH program management, Biospecimen collection, Pathology, eQTL manuscript working group, A. Battle, C. D. Brown, B. E. Engelhardt, and S. B. Montgomery. Genetic effects on gene expression across human tissues. Nature, 550(7675):204–213, 2017.
  • Zhang et al. [2015] W. Zhang, Y. Yu, F. Hertwig, J. Thierry-Mieg, W. Zhang, D. Thierry-Mieg, J. Wang, C. Furlanello, V. Devanarayan, J. Cheng, Y. Deng, B. Hero, H. Hong, M. Jia, L. Li, S.M. Lin, Y. Nikolsky, A. Oberthuer, T. Qing, Z. Su, R. Volland, C. Wang, M.D. Wang, J. Ai, D. Albanese, S. Asgharzadeh, S. Avigad, W. Bao, M. Bessarabova, M.H. Brilliant, B. Brors, M. Chierici, T.-M. Chu, J. Zhang, R.G. Grundy, M.M. He, S. Hebbring, H.L. Kaufman, S. Lababidi, L.J. Lancashire, Y. Li, X.X. Lu, H. Luo, X. Ma, B. Ning, R. Noguera, M. Peifer, J.H. Phan, F. Roels, C. Rosswog, S. Shao, J. Shen, J. Theissen, G.P. Tonini, J. Vandesompele, P.-Y. Wu, W. Xiao, J. Xu, W. Xu, J. Xuan, Y. Yang, Z. Ye, Z. Dong, K.K. Zhang, Y. Yin, C. Zhao, Y. Zheng, R.D. Wolfinger, T. Shi, L.H. Malkas, F. Berthold, J. Wang, W Tong, L. Shi, Z. Peng, and M. Fischer. Comparison of RNA-seq and microarray-based models for clinical endpoint prediction. Genome Biology, 16(1):133, 2015.
  • Oberthuer et al. [2015] A. Oberthuer, D. Juraeva, B. Hero, R. Volland, C. Sterz, R. Schmidt, A. Faldum, Y. Kahlert, A. Engesser, S. Asgharzadeh, R. Seeger, M. Ohira, A. Nakagawara, P. Scaruffi, G.P. Tonini, I. Janoueix-Lerosey, O. Delattre, G. Schleiermacher, J. Vandesompele, F. Speleman, R. Noguera, M. Piqueras, J. Bénard, A. Valent, S. Avigad, I. Yaniv, R.G. Grundy, M. Ortmann, C. Shao, M. Schwab, R. Eils, T. Simon, J. Theissen, F. Berthold, F. Westermann, B. Brors, and M. Fischer. Revised Risk Estimation and Treatment Stratification of Low- and Intermediate-Risk Neuroblastoma Patients by Integrating Clinical and Molecular Prognostic Markers. Clinical Cancer Research, 21(8):1904–1915, 2015.
  • Kivelä et al. [2014] M. Kivelä, A. Arenas, M. Barthelemy, J. P. Gleeson, Y. Moreno, and M. A. Porter. Multilayer networks. Journal of Complex Networks, 2(3):203–271, 2014.
  • Ashburner et al. [2000] M. Ashburner, C. A. Ball, J. A. Blake, D. Botstein, H. Butler, J. M. Cherry, A .P. Davis, K. Dolinski, S. S.. Dwight, J. T.. Eppig, M. A. Harris, D. P.. Hill, L. Issel-Tarver, A. Kasarskis, S. Lewis, J. C. Matese, J. E. Richardson, M. Ringwald, G. M. Rubin, and G. Sherlock. Gene Ontology: tool for the unification of biology. Nature Genetics, 25(1):25–29, 2000.
  • The Gene Ontology Consortium [2017] The Gene Ontology Consortium. Expansion of the Gene Ontology knowledgebase and resources. Nucleic Acid Research, 45(D1):D331–D338, 2017.
  • Yu et al. [2010] G. Yu, F. Li, Y. Qin, X. Bo, Y. Wu, and S. Wang. GOSemSim: an R package for measuring semantic similarity among GO terms and gene products. Bioinformatics, 26(7):976–978, 2010.
  • Cox and Cox [2001] T. F. Cox and M. A. A. Cox. Multidimensional Scaling. Chapman and Hall, 2001.
  • Shen and Huang [2007] H. Shen and J. Z. Huang. Sparse principal componenent analysis via regularized low rank matrix approximation.

    Journal of Multivariate Analysis

    , 99:1015–1034, 2007.
  • Fan and Li [2001] J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96:1348–1360, 2001.
  • Klambauer et al. [2017] G. Klambauer, T. Unterthiner, A. Mayr, and S. Hochreiter. Self-Normalizing Neural Networks. arXiv:1706.02515, 2017.
  • Kingma and Ba [2014] D. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. arXiv:1412.6980; 3rd International Conference for Learning Representations, ICLR 2015, 2014.
  • Jurman et al. [2012a] G. Jurman, S. Riccadonna, R. Visintainer, and C. Furlanello. Algebraic Comparison of Partial Lists in Bioinformatics. PLOS ONE, 7(5):e36540, 2012a.
  • Matthews [1975] B.W. Matthews. Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta - Protein Structure, 405(2):442–451, 1975.
  • Baldi et al. [2000] P. Baldi, S. Brunak, Y. Chauvin, C.A.F. Andersen, and H. Nielsen. Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics, 16(5):412–424, 2000.
  • Gorodkin [2004] J. Gorodkin. Comparing two K-category assignments by a K-category correlation coefficient. Computational Biology and Chemistry, 28:367–374, 2004.
  • Jurman et al. [2012b] G. Jurman, S. Riccadonna, and C. Furlanello. A comparison of MCC and CEN error measures in multi-class prediction. PLOS ONE, 7(8):e41882, 2012b.
  • Bishop [1995] C. M. Bishop.

    Neural Networks for Pattern Recognition

    .
    Oxford University Press, 1995.