Biclustering is a popular family of data analysis techniques within cluster-analysis. Previously biclustering was known under the names direct clustering or subspace clustering. The term biclustering was proposed by Boris Mirkin in , p. 296:
The term biclustering refers to simultaneous clustering of both row and column sets in a data matrix. Biclustering addresses the problems of aggregate representation of the basic features of interrelation between rows and columns as expressed in the data.
The main advantage of biclustering technique lies in its ability to keep similarity of grouped objects in terms of their common attributes. So, biclustering is able to capture object similarity (homogeneity) expressed only by a subset of attributes, which allows an analyst to clearly see why certain objects were grouped together.
In the previous decade biclustering methods became extremely popular for gene expression analysis analysis in bioinformatics. Here, genes which demonstrate similar properties only in a subset of observable situations are considered to be within a bicluster along with those situations. The first rather comprehensive survey in the field was done in . Even though the survey was limited only to biclustering in bioinformatics, the field came to its maturity to have its own classification of the methods. The authors classified biclustering techniques according to several properties: biclustering type, biclustering structure, the way of bicluster generation, and the algorithmic strategy.
As it often happens, researchers from the bioinformatics domain overlooked or even rediscovered biclustering methods which have been known for decades. Thus, the notion of formal concept was known since the early 80-s , it corresponds to maximal inclusion unit submatrices in Boolean matrices [5, 6, 7]. The idea of closed sets from Formal Concept Analysis and from frequent itemset mining were not considered in the bioinformatics domain. However, there are numerous efficient algorithms and applications, which can be treated as special cases of biclustering-based ones. To the best of our knowledge there is no any biclustering technique mentioned in bioinformatics which exploits ordered bicluster hierarchies. Thus, in Formal Concept Analsis, biclusters (formal concepts) are hierarchically ordered by the relation “be more general concept than”, which proved its helpfulness for data exploration and taxonomy building in different domains.
The aim of this work is two-fold: on the one hand, we are going to shed light on neighbouring domains where biclustering is actively used, and on the other hand build lattice-based taxonomies using the existing classifications of biclustering algorithms in the literature. The main open question in this work is as follows: How to build a unified taxonomy of the biclustering techniques.
The rest of the paper is organised as follows. In Section 2, we shortly review previous work on biclustering, taxonomies of algorithms, and related fields. In Section 3 we give basic definitions of FCA and biclustering (in the most general form). In Section 5 we outline several existing biclustering extensions under the the name of multimodal clustering. Section 4 is the main part of the paper which provides examples of different taxonomies of biclustering algorithms obtained from the literature.
2 Previous work
Construction of taxonomies of algorithms in Computer Science is not new. Thus, in [8, 9] a taxonomy of string matching algorithms was built guided by domain experts according to TABASCO methodology. In those papers, it was shown that concept lattices can be a good visualisation tool paired with interactive abilities of modern computer software. Moreover, concept lattices were successfully used for epistemic taxonomy building  combining multiple inheritance feature with compact graphical representation.
In addition to the existing term biclustering, there are several others like co-clustering or simultaneous clustering. Triclustering, Triadic FCA, multimodal clustering, clustering of Boolean tensors, closed n-sets, relational clustering and several other techniqes are all examples of possible biclustering extensions.
3 Basic definitions and models
3.1 Formal Concept Analysis
A formal context consists of two sets and and a relation between and . The elements of are called the objects and the elements of are called the attributes of the context. The notation or means that the object has attribute .
For , let
and, for , let
These operators are called derivation operators or concept-forming operators for .
Let be a formal context, for subsets and we have
Similar properties hold for subsets of attributes.
A closure operator on set is a mapping with the following properties:
For a closure operator the set is called closure of .
A subset is called closed if .
Let be a context, one can prove that operators
are closure operators.
A formal concept of a formal context is a pair with , , and . The sets and are called the extent and the intent of the formal concept , respectively. The subconcept-superconcept relation is given by iff ().
This definition says that every formal concept has two parts, namely, its extent and intent. This follows an old tradition of the Logic of Port Royal (1662), and is in line with the International Standard ISO 704 that formulates the following definition: “A concept is considered to be a unit of thought constituted of two parts: its extent and its intent.”
The set of all formal concepts of a context together with the order relation forms a complete lattice, called the concept lattice of and denoted by .
Implication , where holds in context if , i.e., each object having all attributes from also has all attributes from .
In the first survey on biclustering techniques , bicluster is defined as a submatrix of an input object-attribute matrix. That is for a given matrix , a bicluster is a pair , where and . The bicluster should fulfil a certain homogeneity property, which varies from method to method, e.g., it may be allowed to contain only 1s inside the corresponding submatrix (bicluster) in Boolean case.
If , then is called an object-attribute or OA-bicluster with density .
Here are some basic properties of oa-biclusters.
2. oa-bicluster is a formal concept iff .
3. if is a oa-bicluster, then .
In figure 1 you can see the example of the oa-bicluster for a particular pair of a certain context . In general, only the regions and are full of non-empty pairs, i.e. have maximal density , since they are object and attribute formal concepts respectively. Some black cells indicate non-empty pairs which one may found in such a bicluster. Therefore, the density parameter is a bicluster quality measure which shows how many non-empty pairs the bicluster contains.
Let be a oa-bicluster and be a nonnegative real number, such that , then is called dense if it satisfies the constraint .
4 Existing taxonomies and their analysis
Since formal concept is a natural notion of bicluster for Boolean data and was rediscovered or reused in bioinformatics, one may suppose that the taxonomy of FCA algorithms is a part of the taxonomy of biclustering algorithms. In fact, paper  proposed such a taxonomy (see fig.2).
The classification properties of the concept lattice building algorithms encoded as follows:
means incremental approach;
means that an algorithm uses canonicity based on the lexical order;
means that an algorithm divides the set of concepts into several parts;
designates that an algorithm uses hashing;
means that an algorithm maintains an auxiliary tree structure;
means usage of attribute cache;
encodes that an algorithm computes intents by subsequently computing intersections of object intents (i.e., );
means that an algorithm computes intersections of already generated intents;
encodes that an algorithm computes intersections of non-object intents and object intents;
means that an algorithm uses supports of attribute sets.
Originally, the authors used several criteria to classify the existing (reviewed) biclustering algorithm: type of bicluster, structure of biclusters, type of bicluster discovery, and algorithmic strategy.
Thus, with respect to the definition of bicluster (its type) the authors differentiate between 1) biclusters with constant values, 2) biclusters with constant values on rows or columns, 3) biclusters with coherent values, and 4) biclusters with coherent evolutions.
The biclusters were classified into one of 9 classes according to their structure.
Exclusive row and column biclusters (rectangular diagonal blocks after row and column reorder).
Non-Overlapping biclusters with checkerboard structure.
Non-Overlapping biclusters with tree structure.
Non-Overlapping non-exclusive biclusters.
Overlapping biclusters with hierarchical structure.
Arbitrarily positioned overlapping biclusters.
Different biclustering methods pursue different goals in terms of the number of discovered biclusters. Thus, they may identify one bicluster at a time or be targeted to discovering one set of biclusters at a time, or they can follow simultaneous bicluster identification, which means that the biclusters are discovered all at the same time. All the three types are possible values of Discovery type attribute in the proposed taxonomy.
Since in many cases the biclustering enumeration is a hard task (the corresponding counting problem may belong to complexity class), different algorithmic enumeration strategies were proposed. Thus, Madeira and Oliviera sort out several categories: 1) Iterative Row and Column Clustering Combination, 2) Divide and Conquer, 3) Greedy Iterative Search, 4) Exhaustive Bicluster Enumeration, and 5) Distribution Parameter Identification.
One of the reasonable questions here is: Why should we build diagrams instead of looking at tables? The answer is we need two complementary views, object-attribute descriptions in tables and ordered clusters of objects such that the objects inside a particular cluster (formal concept) share the same attributes. It is not easy to find such clusters with respect to permutations of rows and columns manually even for small contexts. Moreover, by examining the concept lattice of a certain taxonomy we can find useful attribute dependencies, which can help to discover the underlying taxonomy’s domain.
In fact, the number of classified methods were extended to 47 from 16.
The authors slightly redesigned the proposed classification criteria. Thus, they split the analysed methods into two categories: metric-based and non-metric based. We counted this split as two corresponding attributes in the related formal context. However, we also decided to include all the mentioned evaluation metrics into our analysis like “Measure:MSR” meaning Mean Squared Residue.
The remaining criteria have been changed or extended by the authors. For instance, instead of bicluster types, now eight patterns has been proposed:
Additive coherent values;
Multiplicative coherent values;
Simultaneous coherent values;
The sub-taxonomy based on bicluster structure now contains only six criteria: row exhaustive, column exhaustive, non-exhaustive, row exclusive, column exclusive, and non-exclusive.
By means of terms “exhaustive” and “exclusive” it is possible to describe the desired structure. Thus, exhaustive means where all genes (conditions) should belong to some bicluster, i.e. to be covered by it. Exclusive means whether a gene (condition) has to belong no more than one bicluster; e.g., in non-exclusive case overlapping is allowed.
The attribute algorithmic strategy has been altered in its original form from . The attribute Discovery from  has been renamed to Strategy, but the values remain the same: one bicluster at a time, set of biclusters at a time, and simultaneous bicluster identification.
In the beginning of 2000s it was unusual that data analysts and biologists can miss existing biclustering methods (like FCA), which were not applied in the bioinformatics domain yet. However, later FCA was successfully applied in the domain of gene expression analysis [15, 16, 17, 7, 18], formal concepts were rediscovered by [6, 19] in bioinformatics, approximate greedy  and fast [21, 12] methods for dense bicluster discovery in Boolean setting appeared.
However, even the recent taxonomy from  does not include any of them.
To overcome incompleteness caused by the bioinformatics domain view restriction, an attempt to extend the taxonomy of Madeira and Oliveira was done in .
In addition to the existing criteria, the attribute bicluster values type was added taking two values: binary and numeric. The former shows whether the method is able to find patterns in Boolean object-attribute tables and the latter indicates whether the input entries from can be processed by an algorithm. Another important criteria is whether an algorithm based on the notion of closure (operator) from FCA and Closed Frequent Itemset Mining, implicitly or explicitly. The corresponding formal context is given below.
FCA-related biclusteringType:const Type:const with exceptions Struct:Arbitr. overl. Value type:binary Closure:explicit Closure:implicit Val.type:numeric X.XX.X.BiMax X.XX.XXBox biclustering X.XXX..FCA X.XXX..Freq. Closed Itemsets X.XXX..Association rules XXXX.X.Fault-tolerant concepts XXXXX..OA-biclusters
In Fig.5 we show only those objects (biclustering algorithms) which were not included in the taxonomy of Madeira and Oliveira. By so doing, we considered three general approaches (Formal Concept Analysis, Frequent Closed Itemsets, and Association Rules) and particular modifications of formal concepts (box biclusters , fault-tolerant concepts , OA-biclusters [11, 12]).
In  it was noted that every association rule of the form corresponds to the bicluster or , hence it is possible to consider association rules as a biclustering technique.
Later, the taxonomical issues of biclustering were discussed in ; the authors sorted out binary, integer, and real-valued biclustering approaches. Frequent itemset mining algorithms, BiMax, Association Rules and several others were included into a recent survey on pattern-based biclustering algorithms . Even though the idea of closure was mentioned by the survey authors, they did not use closedness property of the studied patterns as a taxonomical attribute. Moreover, even though the authors cited several FCA-based papers ob biclustering, they do not treat relationship of this discipline with biclustering. However, as we have mentioned, such relationships were studied on a solid mathematical and experimental level; for example, in  it was shown that biclustering of numerical data is possible by means of triclustering of the corresponding binary relation.
Implications and Attribute Exploration can help to find hidden attribute dependencies and extend the built taxonomy by new (counter)examples.
Since we deal with implications, their confidence measure is equal to 1. The size of the whole set of implications in Duquenne-Gigues base is 105.
If we start attribute exploration for the same context, then the first question in a row is the following:
Is it true, that when biclustering technique has attribute “Strategy:One set”, that it also has attribute “Struct:Non-exhaustive”?
An expert can either agree with the implication or disagree. In the latter case, (s)he needs to provide a counterexample: a biclustering technique which follows discovery strategy “one set of biclusters at a time” but does not result in biclusters of the structure type “exhaustive”. There is also an option to stop Attribute Exploration process at every step.
5 Multimodal clustering and closed n-sets
Since the field of biclustering is a subdomain of multimodal or relational clustering, the taxonomy can be extended by applying similar criteria to n-ary relation and tensor clustering algorithms.
Thus, the notion of formal concept was generalised for triadic  and polyadic case . There are efficient algorithms to find triconcepts  and poliadic concepts (closed n-sets) . There exist relaxations of triconcept and poliadic concept notions, triclusters and n-clusters, which allow for certain entries inside such n-dic concept to be zeros [20, 31, 32]; the theoretical and experimental comparison is done in . There are also methods for mining closed patterns in n-ary relations . Two biclustering approaches can be used for mining two formal contexts simultaneously, which shares either set of attributes or objects; this results in pseudotriclusters . As for purely biological applications of triclustering we may suggest reading, for example,  and .
6 Conclusion and future work
Even though the taxonomy building of a particular subfield of Data Analysis or Computer Science is not as laborious as devising Carl Linnaeus’ pre-phylogenetic taxonomy, this is not an easy task to merge several such existing taxonomies and build a unified one. Similarly to new species discovery, new algorithms can be proposed and since they can contain new specific features, new classification attributes may be needed.
One of the possible schemes of taxonomy maintaining here could be done in terms of Attribute Exploration .
At a certain moment the group of expert fixes the set of existing biclustering methods and proposes suitable criteria for their classification. A person or a team which has proposed a new biclustering method should classify the method according to the chosen scheme, then it should be validated by experts. Such a team can propose an extra criterion for method classification. If a person or a team is going to propose a new method for an unexplored combination of classification attributes, it is possible to run attribute exploration to see which prospective types of methods are missing to date. By means of Object Exploration, it may become clear that some attributes are missing, e.g. it is evident that formal concepts or Boolean biclusters is only a particular case of bicluster type with constant values and we need at least one new attribute, Boolean entry values.
Since a taxonomy may be used not only for classification itself, but as a search index for potential users, we may suggest using several ways of interactive visualisation: tree-based (TABASCO-like), concept lattice based (line diagrams), object-attribute tables, and nested line diagrams. The latter can help when someone is interested in a special main set of attributes, which should be shown in the outer taxonomy on the line diagram; the inner taxonomy can be shown if the method-seeker needs a finer level granularity or more detailed description inside of the selected node from the outer taxonomy.
There are two main tasks for our future studies: 1) unifying the existing bicluster taxonomies, and 2) creation a taxonomy of multimodal clustering techniques. Even though there are several good tools for building and managing concept lattices like Concept Explorer, we need to rely on more flexible tools with extensible components. In particular we hope that FCART can become our tool of choice in the near future .
We would like to thank Sergei Obiedkov and Derrick Kourie for a piece of advice and their earlier work on the topic. This work was supported by the Basic Research Program at the National Research University Higher School of Economics in 2015-2016 and performed in the Laboratory of Intelligent Systems and Structural Analysis. The first author was also supported by Russian Foundation for Basic Research (grant #13-07-00504).
-  Hartigan, J.A.: Direct Clustering of a Data Matrix. Journal of the American Statistical Association 67(337) (1972) 123–129
-  Mirkin, B.: Mathematical Classification and Clustering. Kluwer, Dordrecht (1996)
-  Madeira, S.C., Oliveira, A.L.: Biclustering algorithms for biological data analysis: A survey. IEEE/ACM Trans. Comput. Biology Bioinform. 1(1) (2004) 24–45
-  Wille, R.: Restructuring lattice theory: An approach based on hierarchies of concepts. In Rival, I., ed.: Ordered Sets. Volume 83 of NATO Advanced Study Institutes Series. Springer Netherlands (1982) 445–470
-  Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations. Springer, Berlin/Heidelberg (1999)
-  Barkow, S., Bleuler, S., Prelic, A., Zimmermann, P., Zitzler, E.: Bicat: a biclustering analysis toolbox. Bioinformatics 22(10) (2006) 1282–1283
-  Kaytoue, M., Duplessis, S., Kuznetsov, S., Napoli, A.: Two fca-based methods for mining gene expression data. In FerrГ©, S., Rudolph, S., eds.: Formal Concept Analysis. Volume 5548 of Lecture Notes in Computer Science. Springer Berlin Heidelberg (2009) 251–266
-  Cleophas, L.G., Watson, B.W.: Taxonomy-based software construction of SPARE time: a case study. IEE Proceedings - Software 152(1) (2005) 29–37
-  Cleophas, L.G., Watson, B.W., Kourie, D.G., Boake, A., Obiedkov, S.A.: TABASCO: using concept-based taxonomies in domain engineering. South African Computer Journal 37 (2006) 30–40
-  ROTH, C., OBIEDKOV, S., KOURIE, D.G.: On succinct representation of knowledge community taxonomies with formal concept analysis. International Journal of Foundations of Computer Science 19(02) (2008) 383–404
-  Ignatov, D.: Models, Algorithms, and Software Tools of Biclustering Based on Closed Sets. PhD thesis, State University Higher School of Economics, Moscow (2010)
-  Ignatov, D.I., Kuznetsov, S.O., Poelmans, J.: Concept-Based Biclustering for Internet Advertisement. In: Data Mining Workshops (ICDMW), 2012 IEEE 12th International Conference on. (Dec 2012) 123–130
-  Kuznetsov, S.O., Obiedkov, S.A.: Comparing performance of algorithms for generating concept lattices. J. Exp. Theor. Artif. Intell. 14(2-3) (2002) 189–216
-  Pontes, B., Giráldez, R., Aguilar-Ruiz, J.S.: Biclustering on expression data: A review. Journal of Biomedical Informatics 57 (2015/12/28) 163–180
-  Besson, J., Robardet, C., Boulicaut, J., Rome, S.: Constraint-based concept mining and its application to microarray data analysis. Intell. Data Anal. 9(1) (2005) 59–82
-  Kaytoue-Uberall, M., Duplessis, S., Napoli, A.: Using formal concept analysis for the extraction of groups of co-expressed genes. In Le Thi, H., Bouvry, P., Pham Dinh, T., eds.: Modelling, Computation and Optimization in Information Systems and Management Sciences. Volume 14 of Communications in Computer and Information Science. Springer Berlin Heidelberg (2008) 439–449
-  Motameny, S., Versmold, B., Schmutzler, R.: Formal concept analysis for the identification of combinatorial biomarkers in breast cancer. In Medina, R., Obiedkov, S., eds.: Formal Concept Analysis. Volume 4933 of Lecture Notes in Computer Science. Springer Berlin Heidelberg (2008) 229–240
-  Kaytoue, M., Kuznetsov, S.O., Napoli, A., Duplessis, S.: Mining gene expression data with pattern structures in formal concept analysis. Information Sciences 181(10) (2011) 1989 – 2001 Special Issue on Information Engineering Applications Based on Lattices.
-  Prelić, A., Bleuler, S., Zimmermann, P., Wille, A., Bühlmann, P., Gruissem, W., Hennig, L., Thiele, L., Zitzler, E.: A systematic comparison and evaluation of biclustering methods for gene expression data. Bioinformatics 22(9) (2006) 1122–1129
-  Mirkin, B., Kramarenko, A.: Approximate bicluster and tricluster boxes in the analysis of binary data. In Kuznetsov, S., Ślęzak, D., Hepting, D., Mirkin, B., eds.: Rough Sets, Fuzzy Sets, Data Mining and Granular Computing. Volume 6743 of Lecture Notes in Computer Science. Springer Berlin Heidelberg (2011) 248–256
-  Ignatov, D.I., Kuznetsov, S.O.: Concept-based recommendations for internet advertisement. In Belohlavek, R., Kuznetsov, S.O., eds.: Proc. of The Sixth International Conference Concept Lattices and Their Applications (CLA’08), Palacky University, Olomouc (2008) 157––166
-  Pensa, R.G., Boulicaut, J.: Towards fault-tolerant formal concept analysis. In: AI*IA. (2005) 212–223
-  Flores, J.L., Inza, I., Larrañaga, P., Calvo, B.: A new measure for gene expression biclustering based on non-parametric correlation. Computer Methods and Programs in Biomedicine 112(3) (2013) 367 – 397
-  Henriques, R., Antunes, C., Madeira, S.C.: A structured view on pattern mining-based biclustering. Pattern Recognition 48(12) (12 2015) 3941–3958
-  Kaytoue, M., Kuznetsov, S.O., Macko, J., Napoli, A.: Biclustering meets triadic concept analysis. Ann. Math. Artif. Intell. 70(1-2) (2014) 55–79
-  Guigues, J.L., Duquenne, V.: Familles minimales d’implications informatives résultant d’un tableau de données binaires. Mathématiques et Sciences Humaines 95(1) (1986) 5–18 In French.
-  Lehmann, F., Wille, R.: A triadic approach to formal concept analysis. In: Conceptual Structures: Applications, Implementation and Theory, Third International Conference on Conceptual Structures, ICCS ’95, Santa Cruz, California, USA, August 14-18, 1995, Proceedings. (1995) 32–43
-  Voutsadakis, G.: Polyadic concept analysis. Order 19(3) (2002) 295–304
-  Jäschke, R., Hotho, A., Schmitz, C., Ganter, B., Stumme, G.: TRIAS - an algorithm for mining iceberg tri-lattices. In: Proceedings of the 6th IEEE International Conference on Data Mining (ICDM 2006), 18-22 December 2006, Hong Kong, China. (2006) 907–911
-  Cerf, L., Besson, J., Robardet, C., Boulicaut, J.: Closed patterns meet n-ary relations. TKDD 3(1) (2009)
-  Ignatov, D.I., Kuznetsov, S.O., Poelmans, J., Zhukov, L.E.: Can triconcepts become triclusters? Int. J. General Systems 42(6) (2013) 572–593
-  Cerf, L., Besson, J., Nguyen, K., Boulicaut, J.: Closed and noise-tolerant patterns in n-ary relations. Data Min. Knowl. Discov. 26(3) (2013) 574–619
-  Ignatov, D., Gnatyshak, D., Kuznetsov, S., Mirkin, B.: Triadic formal concept analysis and triclustering: searching for optimal patterns. Machine Learning (2015) 1–32
-  Spyropoulou, E., Bie, T.D., Boley, M.: Interesting pattern mining in multi-relational data. Data Min. Knowl. Discov. 28(3) (2014) 808–849
-  Gnatyshak, D., Ignatov, D.I., Semenov, A., Poelmans, J.: Gaining insight in social networks with biclustering and triclustering. In: Perspectives in Business Informatics Research - 11th International Conference, BIR 2012, Nizhny Novgorod, Russia, September 24-26, 2012. Proceedings. (2012) 162–171
-  Zhao, L., Zaki, M.J.: Tricluster: An effective algorithm for mining coherent clusters in 3d microarray data. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, Baltimore, Maryland, USA, June 14-16, 2005. (2005) 694–705
-  Li, A., Tuck, D.: An effective tri-clustering algorithm combining expression data with gene regulation information. Gene regulation and systems biology 3 (2009) 49–64
-  Ganter, B., Stumme, G.: Creation and merging of ontology top-levels. In: Conceptual Structures for Knowledge Creation and Communication, 11th International Conference on Conceptual Structures, ICCS 2003 Dresden, Germany, July 21-25, 2003 Proceedings. (2003) 131–145
-  Sertkaya, B.: A survey on how description logic ontologies benefit from FCA. In: Proceedings of the 7th International Conference on Concept Lattices and Their Applications, Sevilla, Spain, October 19-21, 2010. (2010) 2–21
Neznanov, A., Parinov, A.:
FCA analyst session and data access tools in FCART.
In Agre, G., Hitzler, P., Krisnadhi, A.A., Kuznetsov, S.O., eds.: Artificial Intelligence: Methodology, Systems, and Applications - 16th International Conference, AIMSA 2014, Varna, Bulgaria, September 11-13, 2014. Proceedings. Volume 8722 of Lecture Notes in Computer Science., Springer (2014) 214–221