Ensembles of Models and Metrics for Robust Ranking of Homologous Proteins
share
An ensemble of models (EM), where each model is constructed on a diverse subset of feature variables, is proposed to rank rare class items ahead of majority class items in a highly unbalanced two class problem. The proposed ensemble relies on an algorithm to group the feature variables into subsets where the variables in a subset work better together in a model and the variables in different subsets work better in separate models. The strength of the EM depends on the algorithm's ability to identify strong and diverse subsets of feature variables. A second phase of ensembling is achieved by aggregating several EMs each optimized on a diverse evaluation metric. The resulting ensemble is called ensemble of models and metrics (EMM). Here, the diverse/complementary evaluation metrics ensure increased diversity among EMs to aggregate. The ensembles are applied to the protein homology data, downloaded from the 2004 KDD cup competition website, to rank proteins in such a way that the rare homologous proteins are found ahead of the majority non-homologous proteins. The ensembles are constructed using feature variables which are various scores from sequence alignments of amino acids in a candidate protein and three dimensional descriptions of a native protein representing functional and structural similarity of proteins. While prediction performances of the EMs are better than the contemporary state-of-the-art ensembles and competitive to the winning procedures of the 2004 KDD cup competition, the performances of the EMM are found on the top of all. In this application, we have two diverse EMs constructed on two complementary evaluation metrics average precision and rank last, where the former is robust against ranking close homologs and the latter is robust against ranking distant homologs. The advantage of using EMM is that it is robust against both close and distant homologs.
READ FULL TEXT