Multi-class Classification Model Inspired by Quantum Detection Theory

10/10/2018
by   Prayag Tiwari, et al.
0

Machine Learning has become very famous currently which assist in identifying the patterns from the raw data. Technological advancement has led to substantial improvement in Machine Learning which, thus helping to improve prediction. Current Machine Learning models are based on Classical Theory, which can be replaced by Quantum Theory to improve the effectiveness of the model. In the previous work, we developed binary classifier inspired by Quantum Detection Theory. In this extended abstract, our main goal is to develop multi-class classifier. We generally use the terminology multinomial classification or multi-class classification when we have a classification problem for classifying observations or instances into one of three or more classes.

READ FULL TEXT VIEW PDF
03/04/2019

Binary Classifier Inspired by Quantum Theory

Machine Learning (ML) helps us to recognize patterns from raw data. ML i...
08/13/2020

Metrics for Multi-Class Classification: an Overview

Classification tasks in machine learning involving more than two classes...
11/03/2020

Image Classification via Quantum Machine Learning

Quantum Computing and especially Quantum Machine Learning, in a short pe...
12/05/2021

Quantum Machine Learning for Radio Astronomy

In this work we introduce a novel approach to the pulsar classification ...
08/29/2011

Datum-Wise Classification: A Sequential Approach to Sparsity

We propose a novel classification technique whose aim is to select an ap...
02/06/2020

Quantification of Differential Information using Matrix Pencil

Any traditional classification problem in general involves modelling ind...
02/27/2020

Comparison of Multi-Class and Binary Classification Machine Learning Models in Identifying Strong Gravitational Lenses

Typically, binary classification lens-finding schemes are used to discri...

1 Introduction

Quantum Mechanics has already shown its effectiveness in many fields so there is a good possibility that it will prove to be useful in Machine Learning (ML) as well. Quantum theory can open a new way towards quantum inspired ML which might outperform traditional machine learning if used properly. The theory of Quantum Mechanics (QM) has been implemented in several domain of IR recently by Li et al. (2018); Zhang, Song, Zhang, Wang, Li, Li and Wang (2018); Zhang et al. (2016); Zhang, Niu, Su, Wang, Ma and Song (2018); Wang et al. (2016); Zhang, Su, Zhang, Wang and Song (2018); Li et al. (2018, 2015)

. Quantum Probability theory is the quantum generalization of classical probability theory, which was developed by

von Neumann (1955). Classical probability theory provides that a system can have either state 0 or 1 and quantum probability comes into existence to go beyond classical theory and describe states in between 0 and 1 with classical states.

The effectiveness of the state-of-the-art classification algorithms relies on logical theory of sets, theory of probability and the algebra of vector spaces. For example, the most straightforward technique is called Naive Bayes, which considers objects (e.g. documents) as elements of sets and applies basic probability measures to these sets for selecting classes. Another effective classification technique called Support Vector Machines considers objects as points of a multi-dimensional space and aims to select subspaces as classes. However, an effective combination of techniques stemming from different theories is still missing, although it has been investigated in IR since the book on the Geometry of IR by

van Rijsbergen (1979).

Despite its effectiveness in some domains, classification effectiveness is still unsatisfactory in a number of application domains due to a variety of reasons, such as the number of categories and the nature of data. The number of categories may be so large that a classification technique that is effective for a few categories may be ineffective when thousands of categories are required; moreover, the nature of data may be so complex that the techniques that are effective for simple objects may prove to be ineffective for complex objects. A sensible approach to addressing the problems caused by unconventional categorical systems or complex data is to adapt well-known and effective techniques to these new contexts. Another approach, which is indeed the focus of this paper, is to radically change paradigm and to investigate whether a new theoretical framework may be beneficial and be a new research direction.

Quantum Theory may provide a theoretical model for classification. To our knowledge no work has been done on quantum based classification so this paper is a first step to enter into quantum inspired machine learning and prove the effectiveness of quantum theory in classification.

2 Proposed Methodology and Discussion

In the previous work, we developed a binary classifier inspired by Quantum Detection Theory by Di Buccio et al. (2018). The main task was to identify whether a document belongs to a given topic or not. We used Reuters21578111http://www.daviddlewis.com/resources/testcollections/reuters21578/ in order to check the performance of our model. Our proposed binary classifier model inspired from quantum detection theory performed very well in terms of recall and F-measures for most of the topics. An experiment was done on the small dataset so work is still in progress. Our algorithm works as follows: it starts by computing the density operators and from positive and negative samples, respectively. In order to achieve this, for a particular feature, we first compute the number of documents with non-zero values in the feature. In this way, one vector is generated from each class, thus obtaining two vectors which are respectively denoted as and . These vectors can be regarded as a representation of the feature statistics among a class. We normalize the vectors and compute the outer spaces in order to obtain the density operators and :

(1)

We computed the projection operator according to the eigen decomposition described in (Melucci, 2015), that is,

(2)

where

is the prior probability of the negative class and

. Moreover,

is the positive eigenvalue corresponding to

which represents the subspaces of the vectors representing the documents to be accepted in the target class.

In this extended abstract, our main goal is to develop multi class classifier. We generally use the terminology ”multinomial classification or multi-class classification” when we have a classification problem for classifying observations or instances into one of three or more classes.

The main theory behind quantum inspired multi-class classification is as follows: The choice among the hypotheses, which the asserts ”The system has the density operator ,” in which can be based on the result of the measurement of commuting operators , making a resolution of identity operator :

(3)

Our problem is getting the set of projectors so that the choice among the

hypothesis can be made with the minimum average cost. It will assist in the event of Quantum Detection Theory, in constructing and estimating the best receiver for the communications system. In this, messages are coded into an alphabets of 3 or more symbols, and a distinct signal is transmitted for each.

Assume is the prior probability of the hypothesis , and is the cost for choosing when is correct. So the average cost per decision can be described as,

(4)

which has to be minimized by the set of given commuting projection operators . In particular, , , , can be approximate to the average probability of error.

In each hypotheses, the state of the system is in pure state , so the projection operator will have such form, . Here is the linear combination of the the given state . The main problem is to find the set of projectors minimizing the average cost when more than two categories or hypotheses are available; the solution can be a generalization of the solution of the problem of finding the set of projectors in the event of two categories.

3 Conclusion and Future Works

Although research work is still in progress we are testing a multi class classifier based on Quantum Detection Theory and we expect that it is possible to develop such a model. In order to learn about Quantum Detection Theory (QDT) and classification tasks in more detail, these works may be beneficial and a basis for developing this model. (Helstrom, 1971; Yuen et al., 1975; Helstrom, 1969, 1972; Eldar and Forney, 2001; Helstrom and Kennedy, 1974; Helstrom, 1968; Holevo, 1998; Vilnrotter and Lau, 2001; Di Buccio and Melucci, n.d.; Melucci, n.d.; Melucci et al., 2012; Melucci and van Rijsbergen, 2011; Melucci and Baeza-Yates, 2011; Melucci, 2018, 2017; Nanni and Melucci, 2016)

Acknowledgement

This work is supported by the Quantum Access and Retrieval Theory (QUARTZ) project, which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 721321.

References

  • (1)
  • Di Buccio et al. (2018) Di Buccio, E., Li, Q., Melucci, M. and Tiwari, P. (2018), Binary classification model inspired from quantum detection theory, in ‘Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval’, ACM, pp. 187–190.
  • Di Buccio and Melucci (n.d.) Di Buccio, E. and Melucci, M. (n.d.), Evaluation of a feedback algorithm inspired by quantum detection for dynamic search tasks.
  • Eldar and Forney (2001) Eldar, Y. C. and Forney, G. D. (2001), ‘On quantum detection and the square-root measurement’, IEEE Transactions on Information Theory 47(3), 858–872.
  • Helstrom and Kennedy (1974) Helstrom, C. and Kennedy, R. (1974), ‘Noncommuting observables in quantum detection and estimation theory’, IEEE Transactions on Information Theory 20(1), 16–24.
  • Helstrom (1968) Helstrom, C. W. (1968), ‘Detection theory and quantum mechanics (ii)’, Information and Control 13(2), 156–171.
  • Helstrom (1969) Helstrom, C. W. (1969), ‘Quantum detection and estimation theory’, Journal of Statistical Physics 1(2), 231–252.
  • Helstrom (1971) Helstrom, C. W. (1971), ‘Quantum detection theory’.
  • Helstrom (1972) Helstrom, C. W. (1972), Vii quantum detection theory, in ‘Progress in optics’, Vol. 10, Elsevier, pp. 289–369.
  • Holevo (1998) Holevo, A. S. (1998), ‘The capacity of the quantum channel with general signal states’, IEEE Transactions on Information Theory 44(1), 269–273.
  • Li et al. (2015) Li, Q., Li, J., Zhang, P. and Song, D. (2015), Modeling multi-query retrieval tasks using density matrix transformation, in ‘Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval’, ACM, pp. 871–874.
  • Li et al. (2018) Li, Q., Uprety, S., Wang, B. and Song, D. (2018), ‘Quantum-inspired complex word embedding’, arXiv preprint arXiv:1805.11351 .
  • Melucci (2015) Melucci, M. (2015), ‘Relevance feedback algorithms inspired by quantum detection’, IEEE Transactions on Knowledge and Data Engineering 28(4), 1022–1034.
  • Melucci (2017) Melucci, M. (2017), ‘An algorithm to calculate a quantum probability space’, arXiv preprint arXiv:1710.10158 .
  • Melucci (2018) Melucci, M. (2018), ‘An efficient algorithm to compute a quantum probability space’, IEEE Transactions on Knowledge and Data Engineering .
  • Melucci (n.d.) Melucci, M. (n.d.), Introduction to information retrieval and quantum mechanics, Springer.
  • Melucci and Baeza-Yates (2011) Melucci, M. and Baeza-Yates, R. (2011), Advanced topics in information retrieval, Vol. 33, Springer Science & Business Media.
  • Melucci and van Rijsbergen (2011) Melucci, M. and van Rijsbergen, K. (2011), Quantum mechanics and information retrieval, in ‘Advanced topics in information retrieval’, Springer, pp. 125–155.
  • Melucci et al. (2012) Melucci, M. et al. (2012), ‘Contextual search: A computational framework’, Foundations and Trends® in Information Retrieval 6(4–5), 257–405.
  • Nanni and Melucci (2016) Nanni, L. and Melucci, M. (2016), ‘Combination of projectors, standard texture descriptors and bag of features for classifying images’, Neurocomputing 173, 1602–1614.
  • van Rijsbergen (1979) van Rijsbergen, C. J. (1979), Information Retrieval, II edn, Butterworths, London, chapter Automatic Classification, pp. 36–65.
  • Vilnrotter and Lau (2001) Vilnrotter, V. and Lau, C. (2001), ‘Quantum detection theory for the free-space channel’, The InterPlanetary Network Progress Report 42-146, April–June 2001 pp. 1–34.
  • von Neumann (1955) von Neumann, J. (1955), Mathematical Foundations of Quantum Mechanics, Princeton University Press, Princeton, New Jersey, USA.
  • Wang et al. (2016) Wang, B., Zhang, P., Li, J., Song, D., Hou, Y. and Shang, Z. (2016), ‘Exploration of quantum interference in document relevance judgement discrepancy’, Entropy 18(4), 144.
  • Yuen et al. (1975) Yuen, H., Kennedy, R. and Lax, M. (1975), ‘Optimum testing of multiple hypotheses in quantum detection theory’, IEEE Transactions on Information Theory 21(2), 125–134.
  • Zhang et al. (2016) Zhang, P., Li, J., Wang, B., Zhao, X., Song, D., Hou, Y. and Melucci, M. (2016), ‘A quantum query expansion approach for session search’, Entropy 18(4), 146.
  • Zhang, Niu, Su, Wang, Ma and Song (2018) Zhang, P., Niu, J., Su, Z., Wang, B., Ma, L. and Song, D. (2018), ‘End-to-end quantum-like language models with application to question answering’.
  • Zhang, Su, Zhang, Wang and Song (2018) Zhang, P., Su, Z., Zhang, L., Wang, B. and Song, D. (2018), ‘A quantum many-body wave function inspired language modeling approach’, arXiv preprint arXiv:1808.09891 .
  • Zhang, Song, Zhang, Wang, Li, Li and Wang (2018)

    Zhang, Y., Song, D., Zhang, P., Wang, P., Li, J., Li, X. and Wang, B. (2018), ‘A quantum-inspired multimodal sentiment analysis framework’,

    Theoretical Computer Science .