I Introduction
Since the early developments in the 1950’s Samuel (1959), Machine Learning (ML) has evolved into a science addressing various big data problems. The techniques developed for ML, such as decision tree learning Quinlan (1986) and artificial neural networks (ANN) Peterson et al. (1994), allow to train computers in order to perform specific tasks usually deemed to be complex for handwoven algorithms. For supervised learning
, the algorithm is first trained on labeled data, and then to classify testing data into the categories defined during training. In contrast, in
semisupervised and unsupervised learning, where partially labeled or unlabeled data is provided, the algorithm is expected to find the relevant patterns unassistedly.The last decade has seen a rapid progress in ML techniques, in particular the development of deep ANN. A deep ANN is a multilayered network of threshold units LeCun et al. (2015)
. Each unit computes only a simple nonlinear function of its inputs, which allows each layer to represent a certain level of relevant features. Unlike traditional ML techniques (e.g. boosted decision trees) which rely heavily on expertdesigned features in order to reduce the dimensionality of the problem, deep ANN automatically extract pertinent features from data, enabling datamining without prior assumptions. Fueled by vast amounts of big data and the fast development in training techniques and parallel computing architectures, modern deep learning systems have achieved major successes in computer vision
Krizhevsky et al. (2012), speech recognition Hinton et al. (2012)Mikolov et al. (2013), and have recently emerged as a promising tool for scientific research Zdeborová (2017); Carleo and Troyer (2017); Carrasquilla and Melko (2017); Zhang et al. (2018), where the plethora of experimental data presents a challenge for insightful analysis.High Energy Physics (HEP) is a big data science and has a long history of using supervised ML for data analysis. Recently, pioneering works have demonstrated the capability of deep ANN in understanding jet substructure
Baldi et al. (2016); Butter et al. (2017); Larkoski et al. (2017); Macaluso and Shih (2018) and the identification of particles Baldi et al. (2014) or even whole signal signatures (see e.g. Cohen et al. (2018), where weakened supervised learning is applied). However, the primary goal of the HEP experiments is to detect predicted or unpredicted physics Beyond the Standard Model (BSM) in order to establish the underlying fundamental laws of nature. Despite its significant role in current data analysis, supervised ML techniques suffer from the model dependence introduced during training. This problem can potentially be addressed by the semisupervised/unsupervised techniques developed for novelty detection (for a review see, e.g. Pimentel et al. (2014)). Novelty detection is the ML task to recognize data belonging to an unknown pattern. If being interpreted as novel signal, BSM physics could be detected without specifying an underlying theory during data analysis. Hence, a combination of novelty detection and supervised ML may lay out a framework for the future HEP data analysis.Some preliminary and at least partially related efforts have been made at jet Metodiev et al. (2017); Andreassen et al. (2018) and event Aaltonen et al. (2008); CMS (2008); collaboration (2017); Kuusela et al. (2012); Collins et al. (2018); D’Agnolo and Wulzer (2018)
level. For novelty detection with given feature representation, its sensitivity depends crucially on the performance of novelty evaluators. Welldesigned evaluators will allow to evaluate the data novelty efficiently and precisely. As a matter of fact the design of novelty evaluators or the relevant test statistics defines the frontier of novelty detection
Pimentel et al. (2014). In this letter, we propose a set of densitybased novelty evaluators. In contrast to traditional densitybased ones, which only quantify isolation of testing data from the known patterns, the new novelty evaluators are sensitive to the clustering of testing data. On this basis, we design algorithms for novelty detection using an autoencoder, which are subsequently applied for detecting several BSM benchmarks at LHC and future colliders.Ii Algorithms
Novelty detection using a deep ANN can be separated into three steps: 1) feature learning, 2) dimensional reduction, 3) novelty evaluation. During the first step the ANN is trained under supervision, using labeled known patterns. The nodes of the trained ANN contain the information gathered for classification and constitute the feature space, which has typically a large dimension. In order to reduce the sparse error and to improve the efficiency of the analysis, one removes the irrelevant features by dimensional reduction, which can be implemented using an autoencoder Vincent et al. (2008)
. An autoencoder is an ANN with identical number of nodes for input and output layers and fewer nodes for hidden layers. Its lossfunction measures the difference between input and output, defined as the reconstruction error
. Here andare the vectors of input and output nodes, respectively. Hence the autoencoder learns unsupervised how to reconstruct its input. This allows it to form a submanifold in the full feature space. Afterwards, the novelty of testing data is evaluated, for the final significance analysis. The algorithm is shown in FIG.
1. For the HEP data analysis, the data with known and unknown patterns can be interpreted as SM background and BSM signal, respectively.We generate Monte Carlo data using MadGraph5_aMC@NLO Alwall et al. (2014) and rely on Keras Chollet et al. (2015) (TensorFlow Abadi et al. (2015)based) for the ANN construction. For the supervised classification of events with visibleparticle fourmomenta (which we internally normalise by ) and labeled patterns we use an ANN with input nodes, output nodes, and three hidden layers with 30, 30 and 10 nodes, respectively. We use Nesterov’s accelerated gradient descent optimizer Nesterov (1983) with a learning rate of 0.3, a learning momentum of 0.99 and a decay rate of . The batch size is fixed to be 30 and the loss function is the categorical cross entropy Rubinstein (1999, 2001). The collection of all nodes constitute the feature space with dimension . This ensures that it contains the nonlinear information learned from classification. We normalize the axes of the feature space to and use
as activation function for the autoencoder. Finally, an autoencoder consisting of five hidden layers with 40, 20, 8, 20 and 40 nodes, respectively, and a learning rate of 2.0 projects this feature space onto an eightdimensional subspace. We have checked that the results of all ANNs are stable against variations in the numbers of hidden layers and nodes.
Iii Novelty Evaluation
Novelty evaluation of testing data is a crucial step for novelty detection. Various approaches have been developed in the past decades Pimentel et al. (2014). For nontime series data, one of the most popular approaches is densitybased Breunig et al. (2000)
, in which a Local Outlier Factor (LOF), i.e., the ratio of the local density of a given testing data and the local densities of its neighbors, is proposed as a novelty measure. Explicitly, this traditional measure is
Kriegel et al. (2009); Socher et al. (2013)(1) 
here is the mean distance of a testing data to its nearest neighbors, is the average of the mean distances defined for its nearest neighbors, and
is the standard deviation of the latter. The subscript of “train” indicates that all quantities are defined w.r.t the training dataset. We calculate
using the method suggested in Kriegel et al. (2009); Socher et al. (2013). The probabilistic novelty evaluator can be defined as the cumulative distribution function
. Here is a normalization factor, defined as the root mean square of the measure values for all testing data. This evaluator measures the isolation of testing data from training data. A testing data located away from or at the tail of the training data distribution thus tends to be scored high by Breunig et al. (2000); Kriegel et al. (2009).However, is blind to the clustering of testing data which generically exists in the BSM datasets and may result in nontrivial structures such as resonance. In order to utilize this feature, we introduce a measure:
(2) 
with being the dimension of the feature space. Here is the mean distance of the testing data to its nearest neighbors in the testing dataset, whereas is the SM prediction of the same, which can be approximately calculated using the training dataset. This measure is reminiscent of the test statistic introduced in Wang et al. (2006); Dasu et al. (2006)
, where similar idea is employed for estimating the divergence of data distribution. As
is approximately , with and being the numbers of signal and background events in a local bin with unit volume, this measure can be interpreted as the significance of discovery (up to a calibration constant) for this local bin.is defined in a similar way as does. To compare the performance of and
in probing the clustering, we introduce a toy model, where the data resides in a twodimensional space. The known pattern is a Gaussian distribution centered around the origin
, while the unknown pattern is an overlapping narrow Gaussian distribution shifted away from the origin . The training dataset consists of events with known pattern (cf. FIG. 1(a)), while the testing dataset contains from each, known and unknown pattern, events (cf. FIG. 1(b)). As shown in FIG. 1(c) and FIG. 1(d), the clustering of the unknownpattern data, although being hidden from , is pickedup by .The detection based on (or ) however may suffer from fluctuations of the knownpattern testing data in the nonsignal regions, via the term in Eq. (2). While is expected to be zero if the data only consists of events with known patterns, the fluctuations result in nonzero values, since the measure picks up local data excess. This in essence is a kind of Look Elsewhere Effect (LEE). The fluctuations in on the other hand can be neglected, as long as the training dataset used for calculating is much larger than the testing one, with being properly scaled.
The influence of fluctuations on detection sensitivity can be compensated for as the luminosity increases, if scales with . In this case more and more data are used to calculate
in the local bin which is barely changed. This compensation is approximately predicted by the Central Limit Theorem (CLT), which states in this context that the standard deviation of the
response scales with or , for the testing data with known patterns only. We show this in Fig 3, using the knownpattern Gaussian datasets defined before. Indeed, as the number of testing data increases, becomes less and less sensitive to the fluctuations (see Fig. 2(a)).If the fluctuations are not fully compensated for by luminosity, the knownpattern testing data could still be scored high by , and hence diminish the detection sensitivity. This is often true if is small, as typically occurs in the analyses at LHC. To address this potential problem, we propose one more evaluator
(3) 
This evaluator utilizes the fact that the knownpattern testing data with high scores pretty often come from the highdensity regions in the feature space, whereas such data are typically scored low by . As indicated in Fig. 4, performs very well in a typical case where the known and unknownpattern data distributions are partially overlapped, and many of the knownpattern data, especially the ones in the central region, are scored high by due to the fluctuations. The knownpattern datasets used here are the same as before, containing events. The unknown pattern is defined as , with . Indeed, many highscoring data of known pattern in Fig. 3(a) are pushed to the lowscoring end in Fig. 3(c), due to the compensation of . This effect results in improvement in sensitivity, compared to the ones based on or only. Here (and similarly below) the significance is calculated against the known unknownpattern hypothesis for testing data, using the Poissonprobabilitybased test statistic Cowan et al. (2011).
Iv Study on Benchmark Scenarios
Parameter values  

1.2 TeV,  0.152  
, ,  1.55  
,  0.108  
,  0.053 
In order to illustrate their performance, we apply the algorithms designed above to two partonlevel analyses, with two BSM benchmarks defined for each. Though being unrealistic, it is sufficient for proof of concept.
In the first analysis, we simulate the final state at the 14 TeV LHC, with a luminosity of . We require exactly two bottom quarks with and two charged leptons ( and ) with . The SM background stems mainly from

, ,

, ,

, .
Here the physical cross sections have been universally suppressed by a factor for simplification. The signal could arise from multiple BSM scenarios in this analysis. Here we consider:

where and are fermionic top partners,

where is a new gauge boson.
In the second analysis, we simulate unpolarized production with the final state at , with a luminosity of . We require exactly two bottom quarks with and two charged leptons ( and ) with . The SM background arises mainly from

, ,

, .
For BSM scenarios, we consider two specific modes of exotic Higgs decay Curtin et al. (2014):

in the 2HDM and the NMSSM Curtin et al. (2014).
The parameter values and cross sections for the four benchmark scenarios are summarized in TAB. 1.
The sensitivity performance of the algorithms is presented in FIG. 5. In each panel, we show one curve in the “Ideal” case (assuming 100% signal efficiency and background rejection) and one curve with supervised learning as the references for performance evaluation. In the first analysis, the toy model discussed above precisely mimics what happens in benchmark . In this case, the BSM signal and the SM data are partially overlapped in the feature space. Many of the SM data in the nonsignal regions have a strong response, due to fluctuations, and hence diminish the detection sensitivity. However, with the compensation, sizable improvement in sensitivity is achieved. As shown in Fig. 4(a), the sensitivity is approximately doubled using , compared to the ones using or only. For benchmark , is about one order larger than that in benchmark , as indicated in TAB. 1. This tends to enhance the response of the signal, compared to the SM data, and hence results in comparable sensitivities for the analyses based on , and , respectively. For the benchmarks and in the second analysis, the fluctuation effect on is negligibly small, due to (typical for the analyses at collider), while the known and unknownpattern data distributions are not fully separated, hence limiting the efficiency of . This results in a sensitivity performance for which is universally better than the others.
V Summary and Discussion
In this letter, we proposed a set of densitybased novelty evaluators, and , which are sensitive to the clustering of the unknownpattern testing data, for novelty detection in the HEP data analysis. These evaluators allow to design the algorithms with broad applications in detecting BSM physics. They can be also applied to measuring the SM processes yet to be discovered, if we interpret them as “novel” events. As these algorithms are designed using only general assumptions their application could be extended to other bigdata domains as well.
This study could be generalized in multiple directions. We have focused on developing the algorithms for novelty detection in HEP, using partonlevel analysis to demonstrate their sensitivity performance. To fill up the gap between the concept and its application to real data analysis, hadronlevel analysis is definitely needed. In addition, the algorithms could be improved in several aspects. First, the feature selection in the ANN training process might be not yet fully optimized. The features learned from classification of data with labeled known patterns are likely to be suboptimal for enhancing the isolation or clustering of the unknownpattern data. Nevertheless, we may introduce dynamical ML or some feedback mechanisms using the testing dataset, to reinforce the learning of the unknownpattern features. Second, the distance definition of data depends on the geometry of the feature space. We adopted the Euclidean geometry for simplicity, but it is worthwhile to explore the other possibilities. Third, the amount of memory and time needed to implement
increases rapidly with the data size and dimension, which renders not very efficient for large dataset. Ways of accelerating the calculation might be needed. More than that, we would extend the performance analysis of the algorithms to other BSM scenarios, e.g., the ones with interference between the known and unknown patterns, or nontrivial data clusters such as a dip Dicus et al. (1994). Although it is beyond the scope of this study, at last we mention that, a full analysis of the systematic and theoretical uncertainties is absent (for recent effort partially addressing this see Englert et al. (2018)). We leave these topics to a future study.Note added
While this letter was being finalized, De Simone and Jacques (2018) appeared. Both the novelty evaluators proposed here and the test statistic defined in De Simone and Jacques (2018) (as well as the one developed in D’Agnolo and Wulzer (2018) recently) are able to measure the clustering of testing data with unknown pattern. We would like to stress that we developed this project and the relevant ideas independently. Particularly, two significant differences exist between them. First, unlike the test statistic in D’Agnolo and Wulzer (2018); De Simone and Jacques (2018) which measures the divergence of the testing dataset from the training dataset, the evaluators proposed quantify the novelty of individual testing data. Such a design difference enables the evaluators to probe the fine/differential structure of the clustering such as peakdip (a famous BSM example can be found in Dicus et al. (1994)) more efficiently. Second, as the LEE could be a severe problem for novelty detection at Hadron colliders, we explored how to diminish its influences on detection sensitivity (in relation to this, was designed). This was not developed in D’Agnolo and Wulzer (2018); De Simone and Jacques (2018).
v.1 Acknowledgments
Acknowledgements.
We would greatly thank Prof. Michael Wong, our colleague at the HKUST, for highly valuable discussions on the novelty evaluators and the ANN algorithms which were proposed in this letter. T. Liu would thank HuaiKe Guo for discussions on CLT in this context during the MITP (the Mainz Institute for Theoretical Physics) workshop “Probing Baryogenesis via LHC and Gravitational Wave Signatures”, June, 2018. We would thank the experimental colleagues Aurelio Juste, Kirill Prokofiev and Junjie Zhu for reading the manuscript and raising valuable comments. We would also thank LianTao Wang and Zhen Liu for general discussions on this idea at an early stage. J. Hajer is partly supported by the General Research Fund (GRF) under Grant № 16304315. Y. Y. Li would thank the Kavli Institute for Theoretical Physics, where most of her work was done, for the award of the graduate fellowship which was provided by Simons Foundation under Grant № 216179 and Gordon and Betty Moore Foundation under Grant № 4310. This research was also supported in part by the National Science Foundation under Grant № PHY1748958. T. Liu is jointly supported by the GRF under Grant № 16312716 and 16302117. The GRF is issued by the Research Grants Council of Hong Kong S.A.R. He would also thank the MITP for its hospitality, where part of his work was done.References
 Samuel (1959) A. L. Samuel, IBM Journal of Research and Development 3, 210 (1959).
 Quinlan (1986) J. R. Quinlan, Mach. Learn. 1, 81 (1986).
 Peterson et al. (1994) C. Peterson, T. Rögnvaldsson, and L. Lönnblad, Comput. Phys. Commun. 81, 185 (1994).
 LeCun et al. (2015) Y. LeCun, Y. Bengio, and G. Hinton, Nature 521, 436 (2015).
 Krizhevsky et al. (2012) A. Krizhevsky, I. Sutskever, and G. E. Hinton, in Advances in Neural Information Processing Systems 25, edited by F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Curran Associates, Inc., 2012) pp. 1097–1105.
 Hinton et al. (2012) G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, IEEE Signal Proc. Mag. 29, 82 (2012).
 Mikolov et al. (2013) T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, in Advances in Neural Information Processing Systems 26, edited by C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Curran Associates, Inc., 2013) pp. 3111–3119.
 Zdeborová (2017) L. Zdeborová, Nat. Phys. 13, 420 (2017).
 Carleo and Troyer (2017) G. Carleo and M. Troyer, Science 355, 602 (2017).
 Carrasquilla and Melko (2017) J. Carrasquilla and R. G. Melko, Nat. Phys. 13, 431 (2017).
 Zhang et al. (2018) P. Zhang, H. Shen, and H. Zhai, Phys. Rev. Lett. 120, 066401 (2018).
 Baldi et al. (2016) P. Baldi, K. Bauer, C. Eng, P. Sadowski, and D. Whiteson, Phys. Rev. D 93, 094034 (2016).
 Butter et al. (2017) A. Butter, G. Kasieczka, T. Plehn, and M. Russell, (2017), arXiv:1707.08966 [hepph] .
 Larkoski et al. (2017) A. J. Larkoski, I. Moult, and B. Nachman, (2017), arXiv:1709.04464 [hepph] .
 Macaluso and Shih (2018) S. Macaluso and D. Shih, (2018), arXiv:1803.00107 [hepph] .
 Baldi et al. (2014) P. Baldi, P. Sadowski, and D. Whiteson, Nat. Commun. 5, 436 (2014).
 Cohen et al. (2018) T. Cohen, M. Freytsis, and B. Ostdiek, JHEP 02, 034 (2018), arXiv:1706.09451 [hepph] .
 Pimentel et al. (2014) M. A. Pimentel, D. A. Clifton, L. Clifton, and L. Tarassenko, Signal Processing 99, 215 (2014).
 Metodiev et al. (2017) E. M. Metodiev, B. Nachman, and J. Thaler, JHEP 10, 174 (2017), arXiv:1708.02949 [hepph] .
 Andreassen et al. (2018) A. Andreassen, I. Feige, C. Frye, and M. D. Schwartz, (2018), arXiv:1804.09720 [hepph] .
 Aaltonen et al. (2008) T. Aaltonen et al. (CDF), Phys. Rev. D78, 012002 (2008), arXiv:0712.1311 [hepex] .
 CMS (2008) “MUSIC – An Automated Scan for Deviations between Data and Monte Carlo Simulation,” (2008).
 collaboration (2017) T. A. collaboration (ATLAS), (2017).
 Kuusela et al. (2012) M. Kuusela, T. Vatanen, E. Malmi, T. Raiko, T. Aaltonen, and Y. Nagai, Proceedings, 14th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2011): Uxbridge, UK, September 59, 2011, J. Phys. Conf. Ser. 368, 012032 (2012), arXiv:1112.3329 [physics.dataan] .
 Collins et al. (2018) J. H. Collins, K. Howe, and B. Nachman, (2018), arXiv:1805.02664 [hepph] .
 D’Agnolo and Wulzer (2018) R. T. D’Agnolo and A. Wulzer, (2018), arXiv:1806.02350 [hepph] .
 Vincent et al. (2008) P. Vincent, H. Larochelle, Y. Bengio, and P.A. Manzagol, in Proceedings of the 25th International Conference on Machine Learning, ICML ’08 (ACM, New York, NY, USA, 2008) pp. 1096–1103.
 Alwall et al. (2014) J. Alwall, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, O. Mattelaer, H. S. Shao, T. Stelzer, P. Torrielli, and M. Zaro, J. High Energy Phys. 2014, 79 (2014).
 Chollet et al. (2015) F. Chollet et al., “Keras: The python deep learning library,” https://keras.io (2015).
 Abadi et al. (2015) M. Abadi et al., “TensorFlow: Largescale machine learning on heterogeneous systems,” https://tensorflow.org (2015).
 Nesterov (1983) Y. Nesterov, in Doklady AN USSR, Vol. 269 (1983) pp. 543–547.
 Rubinstein (1999) R. Rubinstein, Methodol. Comput. Appl. 1, 127 (1999).
 Rubinstein (2001) R. Y. Rubinstein, in Stochastic optimization: algorithms and applications (Springer, 2001) pp. 303–363.
 Breunig et al. (2000) M. Breunig, H. Kriegel, R. Ng, and J. Sander, in Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD ’00, Vol. 29 (ACM, New York, NY, USA, 2000) pp. 93–104.
 Kriegel et al. (2009) H. Kriegel, P. Kroger, E. Schubert, and A. Zimek, in Proceedings of the 18th ACM conference on Information and knowledge management, CIKM ’09 (ACM, New York, NY, USA, 2009) pp. 1649–1652.
 Socher et al. (2013) R. Socher, M. Ganjoo, C. D. Manning, and A. Ng, in Advances in Neural Information Processing Systems 26, edited by C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Curran Associates, Inc., 2013) pp. 935–943.
 Wang et al. (2006) Q. Wang, S. R. Kulkarni, and S. Verdu, 2006 IEEE International Symposium on Information Theory (2006), 10.1109/ISIT.2006.261842.
 Dasu et al. (2006) T. Dasu, S. Krishnan, S. Venkatasubramanian, and K. Yi, In Proc. Symp. on the Interface of Statistics, Computing Science, and Applications (2006).
 Cowan et al. (2011) G. Cowan, K. Cranmer, E. Gross, and O. Vitells, Eur. Phys. J. C71, 1554 (2011), [Erratum: Eur. Phys. J.C73,2501(2013)], arXiv:1007.1727 [physics.dataan] .
 Curtin et al. (2014) D. Curtin et al., Phys. Rev. D90, 075004 (2014), arXiv:1312.4992 [hepph] .
 Draper et al. (2011) P. Draper, T. Liu, C. E. M. Wagner, L.T. Wang, and H. Zhang, Phys. Rev. Lett. 106, 121805 (2011), arXiv:1009.3963 [hepph] .
 Huang et al. (2014) J. Huang, T. Liu, L.T. Wang, and F. Yu, Phys. Rev. Lett. 112, 221803 (2014), arXiv:1309.6633 [hepph] .
 Dicus et al. (1994) D. Dicus, A. Stange, and S. Willenbrock, Phys. Lett. B333, 126 (1994), arXiv:hepph/9404359 [hepph] .
 Englert et al. (2018) C. Englert, P. Galler, P. Harris, and M. Spannowsky, (2018), arXiv:1807.08763 [hepph] .
 De Simone and Jacques (2018) A. De Simone and T. Jacques, (2018), arXiv:1807.06038 [hepph] .