I Introduction
Machine learning (ML) is a branch of Artificial Intelligence that pushes forward the idea that, by giving access to the right data, machines can learn by themselves how to solve a specific problem [1]. By leveraging complex mathematical and statistical tools, ML renders machines capable of performing independently intellectual tasks that have been traditionally solved by human beings. This idea of automating complex tasks has generated high interest in the networking field, on the expectation that several activities involved in the design and operation of communication networks can be offloaded to machines. Some applications of ML in different networking areas have already matched these expectations in areas such as intrusion detection [2], traffic classification [3], cognitive radios [4].
Among various networking areas, in this paper we focus on ML for optical networking. Optical networks constitute the basic physical infrastructure of all largeprovider networks worldwide, thanks to their high capacity, low cost and many other attractive properties [5]. They are now penetrating new important telecom markets as datacom [6] and the access segment [7], and there is no sign that a substitute technology might appear in the foreseeable future. Different approaches to improve the performance of optical networks have been investigated, such as routing, wavelength assignment, traffic grooming and survivability [8, 9].
In this paper we give an overview of the application of ML to optical networking. Specifically, the contribution of the paper is twofold, namely, ) we provide an introductory tutorial on the use of ML methods and on their application in the optical networks field, and ) we survey the existing work dealing with the topic, also performing a classification of the various use cases addressed in literature so far. We cover both the areas of optical communication and optical networking to potentially stimulate new crosslayer research directions. In fact, ML application can be useful especially in crosslayer settings, where data analysis at physical layer, e.g., monitoring Bit Error Rate (BER), can trigger changes at network layer, e.g., in routing, spectrum and modulation format assignments. The application of ML to optical communication and networking is still in its infancy and the literature survey included in this paper aims at providing an introductory reference for researchers and practitioners willing to get acquainted with existing ML applications as well as to investigate new research directions.
A legitimate question that arises in the optical networking field today is: why machine learning, a methodological area that has been applied and investigated for at least three decades, is only gaining momentum now? The answer is certainly very articulated, and it most likely involves not purely technical aspects [10]. From a technical perspective though, recent technical progress at both optical communication system and network level is at the basis of an unprecedented growth in the complexity of optical networks.
On a system side, while optical channel modeling has always been complex, the recent adoption of coherent technologies [11] has made modeling even more difficult by introducing a plethora of adjustable design parameters (as modulation formats, symbol rates, adaptive coding rates and flexible channel spacing) to optimize transmission systems in terms of bitrate transmission distance product. In addition, what makes this optimization even more challenging is that the optical channel is highly nonlinear.
From a networking perspective, the increased complexity of the underlying transmission systems is reflected in a series of advancements in both data plane and control plane. At data plane, the Elastic Optical Network (EON) concept [12, 13, 14, 15] has emerged as a novel optical network architecture able to respond to the increased need of elasticity in allocating optical network resources. In contrast to traditional fixedgrid Wavelength Division Multiplexing (WDM) networks, EON offers flexible (almost continuous) bandwidth allocation. Resource allocation in EON can be performed to adapt to the several abovementioned decision variables made available by new transmission systems, including different transmission techniques, such as Orthogonal Frequency Division Multiplexing (OFDM), Nyquist WDM (NWDM), transponder types (e.g., BVT^{1}^{1}1For a complete list of acronyms, the reader is referred to the Glossary at the end of the paper., SBVT), modulation formats (e.g., QPSK, QAM), and coding rates. This flexibility makes the resource allocation problems much more challenging for network engineers. At control plane, dynamic control, as in Softwaredefined networking (SDN), promises to enable longawaited ondemand reconfiguration and virtualization. Moreover, reconfiguring the optical substrate poses several challenges in terms of, e.g., network reoptimization, spectrum fragmentation, amplifier power settings, unexpected penalties due to nonlinearities, which call for strict integration between the control elements (SDN controllers, network orchestrators) and optical performance monitors working at the equipment level.
All these “degrees of freedom” and limitations do pose severe challenges to system and network engineers when it comes to deciding what the best system and/or network design is. Machine learning is currently perceived as a paradigm shift for the design of future optical networks and systems. These techniques should allow to infer, from data obtained by various types of monitors (e.g., signal quality, traffic samples, etc.), useful characteristics that could not be easily or directly measured. Some envisioned applications in the optical domain include fault prediction, intrusion detection, physicalflow security, impairmentaware routing, lowmargin design, trafficaware capacity reconfigurations, but many others can be envisioned and will be surveyed in the next sections.
The survey is organized as follows. In Section II, we overview some preliminary ML concepts, focusing especially on those targeted in the following sections. In Section III we discuss the main motivations behind the application of ML in the optical domain and we classify the main areas of applications. In Section IV and Section V, we classify and summarize a large number of studies describing applications of ML at the transmission layer and network layer. In Section VI, we quantitatively overview a selection of existing papers, identifying, for some of the applications described in Section III, the ML algorithms which demonstrated higher effectiveness for each specific use case, and the performance metrics considered for the algorithms evaluation. Finally, Section VII discusses some possible open areas of research and future directions, whereas Section VIII concludes the paper.
Ii Overview of machine learning methods used in optical networks
This section provides an overview of some of the most popular algorithms that are commonly classified as machine learning. The literature on ML is so extensive that even a superficial overview of all the main ML approaches goes far beyond the possibilities of this section, and the readers can refer to a number of fundamental books on the subjects [16, 17, 18, 19, 20]. However, in this section we provide a high level view of the main ML techniques that are used in the work we reference in the remainder of this paper. We here provide the reader with some basic insights that might help better understand the remaining parts of this survey paper. We divide the algorithms in three main categories, described in the next sections, which are also represented in Fig. 1
: supervised learning, unsupervised learning and reinforcement learning. Semisupervised learning, a hybrid of supervised and unsupervised learning, is also introduced. ML algorithms have been successfully applied to a wide variety of problems. Before delving into the different ML methods, it is worth pointing out that, in the context of telecommunication networks, there has been over a decade of research on the application of ML techniques to wireless networks, ranging from opportunistic spectrum access
[21]to channel estimation and signal detection in OFDM systems
[22], to MultipleInputMultipleOutput communications [23], and dynamic frequency reuse [24].Iia Supervised learning
Supervised learning is used in a variety of applications, such as speech recognition, spam detection and object recognition. The goal is to predict the value of one or more output variables given the value of a vector of input variables
. The output variable can be a continuous variable (regression problem) or a discrete variable (classification problem). A training data set comprises samples of the input variables and the corresponding output values. Different learning methods construct a function that allows to predict the value of the output variables in correspondence to a new value of the inputs. Supervised learning can be broken down into two main classes, described below: parametric models, where the number of parameters to use in the model is fixed, and nonparametric models, where their number is dependent on the training set.IiA1 Parametric models
In this case, the function is a combination of a fixed number of parametric basis functions. These models use training data to estimate a fixed set of parameters . After the learning stage, the training data can be discarded since the prediction in correspondence to new inputs is computed using only the learned parameters . Linear models for regression and classification, which consist of a linear combination of fixed nonlinear basis functions, are the simplest parametric models in terms of analytical and computational properties. Many different choices are available for the basis functions: from polynomial to Gaussian, to sigmoidal, to Fourier basis, etc. In case of multiple output values, it is possible to use separate basis functions for each component of the output or, more commonly, apply the same set of basis functions for all the components. Note that these models are linear in the parameters , and this linearity results in a number of advantageous properties, e.g., closedform solutions to the leastsquares problem. However, their applicability is limited to problems with lowdimensional input space. In the remainder of this subsection we focus on neural networks (NNs)^{2}^{2}2Note that NNs are often referred to as Artificial Neural Networks (ANNs). In this paper we use these two terms interchangeably., since they are the most successful example of parametric models.
NNs apply a series of functional transformations to the inputs (see chapter V in [16], chapter VI in [17], and chapter XVI in [20]). A NN is a network of units or neurons
. The basis function or activation function used by each unit is a nonlinear function of a linear combination of the unit’s inputs. Each neuron has a bias parameter that allows for any fixed offset in the data. The bias is incorporated in the set of parameters by adding a dummy input of unitary value to each unit (see Figure
2). The coefficients of the linear combination are the parametersestimated during the training. The most commonly used nonlinear functions are the logistic sigmoid and the hyperbolic tangent. The activation function of the output units of the NN is the identity function, the logistic sigmoid function, and the softmax function, for regression, binary classification, and multiclass classification problems respectively.
Different types of connections between the units result in different NNs with distinct characteristics. All units between the inputs and output of the NN are called hidden units. In the case of a NN, the network is a directed acyclic graph. Typically, NNs are organized in layers, with units in each layer receiving inputs only from units in the immediately preceding layer and forwarding their output only to the immediately following layer. NNs with one layer of hidden units and linear output units can approximate arbitrary well any continuous function on a compact domain provided that a sufficient number of hidden units is used [25].
Given a training set, a NN is trained by minimizing an error function with respect to the set of parameters . Depending on the type of problem and the corresponding choice of activation function of the output units, different error functions are used. Typically in case of regression models, the sum of square error is used, whereas for classification the crossentropy error function is adopted. It is important to note that the error function is a non convex function of the network parameters, for which multiple optimal local solutions exist. Iterative numerical methods based on gradient information are the most common methods used to find the vector
that minimizes the error function. For a NN the error backpropagation algorithm, which provides an efficient method for evaluating the derivatives of the error function with respect to
, is the most commonly used.We should at this point mention that, before training the network, the training set is typically preprocessed by applying a linear transformation to rescale each of the input variables independently in case of continuous data or discrete ordinal data. The transformed variables have zero mean and unit standard deviation. The same procedure is applied to the target values in case of regression problems. In case of discrete categorical data, a 1ofK coding scheme is used. This form of preprocessing is known as feature normalization and it is used before training most ML algorithms since most models are designed with the assumption that all features have comparable scales
^{3}^{3}3However, decision tree based models are a wellknown exception.
.IiA2 Nonparametric models
In nonparametric methods the number of parameters depends on the training set. These methods keep a subset or the entirety of the training data and use them during prediction. The most used approaches are knearest neighbor models (see chapter IV in [17]
) and support vector machines (SVMs) (see chapter VII in
[16] and chapter XIV in [20]). Both can be used for regression and classification problems.In the case of knearest neighbor methods, all training data samples are stored (training phase). During prediction, the knearest samples to the new input value are retrieved. For classification problem, a voting mechanism is used; for regression problems, the mean or median of the k nearest samples provides the prediction. To select the best value of k, crossvalidation [26] can be used. Depending on the dimension of the training set, iterating through all samples to compute the closest k neighbors might not be feasible. In this case, kd trees or localitysensitive hash tables can be used to compute the knearest neighbors.
In SVMs, basis functions are centered on training samples; the training procedure selects a subset of the basis functions. The number of selected basis functions, and the number of training samples that have to be stored, is typically much smaller than the cardinality of the training dataset. SVMs build a linear decision boundary with the largest possible distance from the training samples. Only the closest points to the separators, the support vectors, are stored. To determine the parameters of SVMs, a nonlinear optimization problem with a convex objective function has to be solved, for which efficient algorithms exist. An important feature of SVMs is that by applying a kernel function they can embed data into a higher dimensional space, in which data points can be linearly separated. The kernel function measures the similarity between two points in the input space; it is expressed as the inner product of the input points mapped into a higher dimension feature space in which data become linearly separable. The simplest example is the linear kernel, in which the mapping function is the identity function. However, provided that we can express everything in terms of kernel evaluations, it is not necessary to explicitly compute the mapping in the feature space. Indeed, in the case of one of the most commonly used kernel functions, the Gaussian kernel, the feature space has infinite dimensions.
IiB Unsupervised learning
Social network analysis, genes clustering and market research are among the most successful applications of unsupervised learning methods.
In the case of unsupervised learning the training dataset consists only of a set of input vectors
. While unsupervised learning can address different tasks, clustering or cluster analysis is the most common.
Clustering is the process of grouping data so that the intracluster similarity is high, while the intercluster similarity is low. The similarity is typically expressed as a distance function, which depends on the type of data. There exists a variety of clustering approaches. Here, we focus on two algorithms, kmeans and Gaussian mixture model as examples of partitioning approaches and modelbased approaches, respectively, given their wide area of applicability. The reader is referred to
[27] for a comprehensive overview of cluster analysis.kmeans is perhaps the most wellknown clustering algorithm (see chapter X in [27]
). It is an iterative algorithm starting with an initial partition of the data into k clusters. Then the centre of each cluster is computed and data points are assigned to the cluster with the closest centre. The procedure  centre computation and data assignment  is repeated until the assignment does not change or a predefined maximum number of iterations is exceeded. Doing so, the algorithm may terminate at a local optimum partition. Moreover, kmeans is well known to be sensitive to outliers. It is worth noting that there exists ways to compute k automatically
[26], and an online version of the algorithm exists.While kmeans assigns each point uniquely to one cluster, probabilistic approaches allow a soft assignment and provide a measure of the uncertainty associated with the assignment. Figure 3
shows the difference between kmeans and a probabilistic Gaussian Mixture Model (GMM). GMM, a linear superposition of Gaussian distributions, is one of the most widely used probabilistic approaches to clustering. The parameters of the model are the mixing coefficient of each Gaussian component, the mean and the covariance of each Gaussian distribution. To maximize the log likelihood function with respect to the parameters given a dataset, the expectation maximization algorithm is used, since no closed form solution exists in this case. The initialization of the parameters can be done using kmeans. In particular, the mean and covariance of each Gaussian component can be initialized to sample means and covariances of the cluster obtained by kmeans, and the mixing coefficients can be set to the fraction of data points assigned by kmeans to each cluster. After initializing the parameters and evaluating the initial value of the log likelihood, the algorithm alternates between two steps. In the expectation step, the current values of the parameters are used to determine the “responsibility” of each component for the observed data (i.e., the conditional probability of latent variables given the dataset). The maximization step uses these responsibilities to compute a maximum likelihood estimate of the model’s parameters. Convergence is checked with respect to the log likelihood function or the parameters.
IiC Semisupervised learning
Semisupervised learning methods are a hybrid of the previous two introduced above, and address problems in which most of the training samples are unlabeled, while only a few labeled data points are available. The obvious advantage is that in many domains a wealth of unlabeled data points is readily available. Semisupervised learning is used for the same type of applications as supervised learning. It is particularly useful when labeled data points are not so common or too expensive to obtain and the use of available unlabeled data can improve performance.
Selftraining is the oldest form of semisupervised learning [28]. It is an iterative process; during the first stage only labeled data points are used by a supervised learning algorithm. Then, at each step, some of the unlabeled points are labeled according to the prediction resulting for the trained decision function and these points are used along with the original labeled data to retrain using the same supervised learning algorithm. This procedure is shown in Fig. 4.
Since the introduction of selftraining, the idea of using labeled and unlabeled data has resulted in many semisupervised learning algorithms. According to the classification proposed in [28], semisupervised learning techniques can be organized in four classes: i) methods based on generative models^{4}^{4}4
Generative methods estimate the joint distribution of the input and output variables. From the joint distribution one can obtain the conditional distribution
, which is then used to predict the output values in correspondence to new input values. Generative methods can exploit both labeled and unlabeled data.; ii) methods based on the assumption that the decision boundary should lie in a lowdensity region; iii) graphbased methods; iv) twostep methods (first an unsupervised learning step to change the data representation or construct a new kernel; then a supervised learning step based on the new representation or kernel).IiD Reinforcement Learning
Reinforcement Learning (RL) is used, in general, to address applications such as robotics, finance (investment decisions), inventory management, where the goal is to learn a policy, i.e., a mapping between states of the environment into actions to be performed, while directly interacting with the environment.
The RL paradigm allows agents to learn by exploring the available actions and refining their behavior using only an evaluative feedback, referred to as the reward. The agent’s goal is to maximize its longterm performance. Hence, the agent does not just take into account the immediate reward, but it evaluates the consequences of its actions on the future. Delayed reward and trialanderror constitute the two most significant features of RL.
RL is usually performed in the context of Markov decision processes (MDP). The agent’s perception at time
is represented as a state , where is the finite set of environment states. The agent interacts with the environment by performing actions. At time the agent selects an action , where is the finite set of actions of the agent, which could trigger a transition to a new state. The agent will receive a reward as a result of the transition, according to the reward function . The agent’s goal is to find the sequence of stateaction pairs that maximizes the expected discounted reward, i.e., the optimal policy. In the context of MDP, it has been proved that an optimal deterministic and stationary policy exists. There exist a number of algorithms that learn the optimal policy both in case the state transition and reward functions are known (modelbased learning) and in case they are not (modelfree learning). The most used RL algorithm is Qlearning, a modelfree algorithm that estimates the optimal actionvalue function (see chapter VI in [19]). An actionvalue function, named Qfunction, is the expected return of a stateaction pair for a given policy. The optimal actionvalue function, , corresponds to the maximum expected return for a stateaction pair. After learning function , the agent selects the action with the corresponding highest Qvalue in correspondence to the current state.A tablebased solution such as the one described above is only suitable in case of problems with limited stateaction space. In order to generalize the policy learned in correspondence to states not previously experienced by the agent, RL methods can be combined with existing function approximation methods, e.g., neural networks.
IiE Overfitting, underfitting and model selection
In this section, we discuss a wellknown problem of ML algorithms along with its solutions. Although we focus on supervised learning techniques, the discussion is also relevant for unsupervised learning methods.
Overfitting and underfitting are two sides of the same coin: model selection. Overfitting happens when the model we use is too complex for the available dataset (e.g., a high polynomial order in the case of linear regression with polynomial basis functions or a too large number of hidden neurons for a neural network). In this case, the model will fit the training data too closely
^{5}^{5}5As an extreme example, consider a simple regression problem for predicting a realvalue target variable as a function of a realvalue observation variable. Let us assume a linear regression model with polynomial basis function of the input variable. If we have samples and we select as the order of the polynomial, we can fit the model perfectly to the data points., including noisy samples and outliers, but will result in very poor generalization, i.e., it will provide inaccurate predictions for new data points. At the other end of the spectrum, underfitting is caused by the selection of models that are not complex enough to capture important features in the data (e.g., when we use a linear model to fit quadratic data). Fig. 5 shows the difference between underfitting and overfitting, compared to an accurate model.Since the error measured on the training samples is a poor indicator for generalization, to evaluate the model performance the available dataset is split into two, the training set and the test set. The model is trained on the training set and then evaluated using the test set. Typically around of the samples are assigned to the training set and the remaining are assigned to the test set. Another option that is very useful in case of a limited dataset is to use crossvalidation so that as much of the available data as possible is exploited for training. In this case, the dataset is divided into subsets. The model is trained times using each of the subset for validation and the remaining subsets for training. The performance is averaged over the runs. In case of overfitting, the error measured on the test set is high and the error on the training set is small. On the other hand, in the case of underfitting, both the error measured on the training set and the test set are usually high.
There are different ways to select a model that does not exhibit overfitting and underfitting. One possibility is to train a range of models, compare their performance on an independent dataset (the validation set), and then select the one with the best performance. However, the most common technique is regularization. It consists of adding an extra term  the regularization term  to the error function used in the training stage. The simplest form of the regularization term is the sum of the squares of all parameters, which is known as weight decay and drives parameters towards zero. Another common choice is the sum of the absolute values of the parameters (lasso). An additional parameter, the regularization coefficient , weighs the relative importance of the regularization term and the datadependent error. A large value of heavily penalizes large absolute values of the parameters. It should be noted that the datadependent error computed over the training set increases with . The error computed over the validation set is high for both small and high values. In the first case, the regularization term has little impact potentially resulting in overfitting. In the latter case, the datadependent error has little impact resulting in a poor model performance. A simple automatic procedure for selecting the best consists of training the model with a range of values for the regularization parameter and select the value that corresponds to the minimum validation error. In the case of NNs with a large number of hidden units, dropout  a technique that consists of randomly removing units and their connections during training  has been shown to outperform other regularization methods [29].
Iii Motivation for using machine learning in optical networks and Systems
In the last few years, the application of mathematical approaches derived from the ML discipline have attracted the attention of many researchers and practitioners in the optical communications and networking fields. In a general sense, the underlying motivations for this trend can be identified as follows:

increased system complexity: the adoption of advanced transmission techniques, such as those enabled by coherent technology [11], and the introduction of extremely flexible networking principles, such as, e.g., the EON paradigm, have made the design and operation of optical networks extremely complex, due to the high number of tunable parameters to be considered (e.g., modulation formats, symbol rates, adaptive coding rates, adaptive channel bandwidth, etc.); in such a scenario, accurately modeling the system through closedform formulas is often very hard, if not impossible, and in fact “margins” are typically adopted in the analytical models, leading to resource underutilization and to consequent increased system cost; on the contrary, ML methods can capture complex nonlinear system behaviour with relatively simple training of supervised and/or unsupervised algorithms which exploit knowledge of historical network data, and therefore to solve complex crosslayer problems, typical of the optical networking field;

increased data availability: modern optical networks are equipped with a large number of monitors, able to provide several types of information on the entire system, e.g., traffic traces, signal quality indicators (such as BER), equipment failure alarms, users’ behaviour etc.; here, the enhancement brought by ML consists of simultaneously leveraging the plethora of collected data and discover hidden relations between various types of information.
The application of ML to physical layer use cases is mainly motivated by the presence of nonlinear effects in optical fibers, which make analytical models inaccurate or even too complex. This has implications, e.g., on the performance predictions of optical communication systems, in terms of BER, quality factor (Qfactor) and also for signal demodulation [30],[31], [32].
Moving from the physical layer to the networking layer, the same motivation applies for the application of ML techniques. In particular, design and management of optical networks is continuously evolving, driven by the enormous increase of transported traffic and drastic changes in traffic requirements, e.g., in terms of capacity, latency, user experience and Quality of Service (QoS). Therefore, current optical networks are expected to be run at much higher utilization than in the past, while providing strict guarantees on the provided quality of service. While aggressive optimization and trafficengineering methodologies are required to achieve these objectives, such complex methodologies may suffer scalability issues, and involve unacceptable computational complexity. In this context, ML is regarded as a promising methodological area to address this issue, as it enables automated network selfconfiguration and fast decisionmaking by leveraging the plethora of data that can be retrieved via network monitors, and allowing network engineers to build datadriven models for more accurate and optimized network provisioning and management.
Several use cases can benefit from the application of ML and data analytics techniques. In this paper we divide these use cases in physical layer and network layer use cases. The remainder of this section provides a highlevel introduction to the main applications of ML in optical networks, as graphically shown in Fig. 6, and motivates why ML can be beneficial in each case. A detailed survey of existing studies is then provided in Sections IV and V, for physical layer and network layer use cases, respectively.
Iiia Physical layer domain
As mentioned in the previous section, several challenges need to be addressed at the physical layer of an optical network, typically to evaluate the performance of the transmission system and to check if any signal degradation influences existing lightpaths. Such monitoring can be used, e.g., to trigger proactive procedures, such as tuning of launch power, controlling gain in optical amplifiers, varying modulation format, etc., before irrecoverable signal degradation occurs. In the following, a description of the applications of ML at the physical layer is presented.

QoT estimation.
Prior to the deployment of a new lightpath, a system engineer needs to estimate the Quality of Transmission (QoT) for the new lightpath, as well as for the already existing ones. The concept of Quality of Transmission generally refers to a number of physical layer parameters, such as received Optical SignaltoNoise Ratio (OSNR), BER, Qfactor, etc., which have an impact on the “readability” of the optical signal at the receiver. Such parameters give a quantitative measure to check if a predetermined level of QoT would be guaranteed, and are affected by several tunable design parameters, such as, e.g., modulation format, baud rate, coding rate, physical path in the network, etc. Therefore, optimizing this choice is not trivial and often this large variety of possible parameters challenges the ability of a system engineer to address manually all the possible combinations of lightpath deployment.As of today, existing (predeployment) estimation techniques for lightpath QoT belong to two categories: 1) “exact” analytical models estimating physicallayer impairments, which provide accurate results, but incur heavy computational requirements and 2) marginated formulas, which are computationally faster, but typically introduce high marginations that lead to underutilization of network resources. Moreover, it is worth noting that, due to the complex interaction of multiple system parameters (e.g., input signal power, number of channels, link type, modulation format, symbol rate, channel spacing, etc.) and, most importantly, due to the nonlinear signal propagation through the optical channel, deriving accurate analytical models is a challenging task, and assumptions about the system under consideration must be made in order to adopt approximate models. Conversely, ML constitutes a promising means to automatically predict whether unestablished lightpaths will meet the required system QoT threshold.
Relevant ML techniques: MLbased classifiers can be trained using supervised learning^{6}^{6}6Note that, specific solutions adopted in literature for QoT estimation, as well as for other physical and networklayer use cases, will be detailed in the literature surveys provided in Sections IV and V. to create direct inputoutput relationship between QoT observed at the receiver and corresponding lightpath configuration in terms of, e.g., utilized modulation format, baud rate and/or physical route in the network.

Optical amplifiers control.
In current optical networks, lightpath provisioning is becoming more dynamic, in response to the emergence of new services that require huge amount of bandwidth over limited periods of time. Unfortunately, dynamic setup and teardown of lightpaths over different wavelengths forces network operators to reconfigure network devices “on the fly” to maintain physicallayer stability. In response to rapid changes of lightpath deployment, Erbium Doped Fiber Amplifiers (EDFAs) suffer from wavelengthdependent power excursions. Namely, when a new lightpath is established (i.e., added) or when an existing lightpath is torn down (i.e., dropped), the discrepancy of signal power levels between different channels (i.e., between lightpaths operating at different wavelengths) depends on the specific wavelength being added/dropped into/from the system. Thus, an automatic control of preamplification signal power levels is required, especially in case a cascade of multiple EDFAs is traversed, to avoid that excessive postamplification power discrepancy between different lightpaths may cause signal distortion.Relevant ML techniques: Thanks to the availability of historical data retrieved by monitoring network status, ML regression algorithms can be trained to accurately predict postamplifier power excursion in response to the add/drop of specific wavelengths to/from the system.

Modulation format recognition (MFR).
Modern optical transmitters and receivers provide high flexibility in the utilized bandwidth, carrier frequency and modulation format, mainly to adapt the transmission to the required bitrate and optical reach in a flexible/elastic networking environment. Given that at the transmission side an arbitrary coherent optical modulation format can be adopted, knowing this decision in advance also at the receiver side is not always possible, and this may affect proper signal demodulation and, consequently, signal processing and detection.Relevant ML techniques: Use of supervised ML algorithms can help the modulation format recognition at the receiver, thanks to the opportunity to learn the mapping between the adopted modulation format and the features of the incoming optical signal.

Nonlinearity mitigation.
Due to optical fiber nonlinearities, such as Kerr effect, selfphase modulation (SPM) and crossphase modulation (XPM), the behaviour of several performance parameters, including BER, Qfactor, Chromatic Dispersion (CD), Polarization Mode Dispersion (PMD), is highly unpredictable, and this may cause signal distortion at the receiver (e.g., I/Q imbalance and phase noise). Therefore, complex analytical models are often adopted to react to signal degradation and/or compensate undesired nonlinear effects.Relevant ML techniques: While approximated analytical models are usually adopted to solve such complex nonlinear problems, supervised ML models can be designed to directly capture the effects of such nonlinearities, typically exploiting knowledge of historical data and creating inputoutput relations between the monitored parameters and the desired outputs.

Optical performance monitoring (OPM).
With increasing capacity requirements for optical communication systems, performance monitoring is vital to ensure robust and reliable networks. Optical performance monitoring aims at estimating the transmission parameters of the optical fiber system, such as BER, Qfactor, CD, PMD, during lightpath lifetime. Knowledge of such parameters can be then utilized to accomplish various tasks, e.g., activating polarization compensator modules, adjusting launch power, varying the adopted modulation format, reroute lightpaths, etc. Typically, optical performance parameters need to be collected at various monitoring points along the lightpath, thus large number of monitors are required, causing increased system cost. Therefore, efficient deployment of optical performance monitors in the proper network locations is needed to extract network information at reasonable cost.Relevant ML techniques: To reduce the amount of monitors to deploy in the system, especially at intermediate points of the lightpaths, supervised learning algorithms can be used to learn the mapping between the optical fiber channel parameters and the properties of the detected signal at the receiver, which can be retrieved, e.g., by observing statistics of power eye diagrams, signal amplitude, OSNR, etc.
IiiB Network layer domain
At the network layer, several other use cases for ML arise. Provisioning of new lightpaths or restoration of existing ones upon network failure require complex and fast decisions that depend on several quicklyevolving data, since, e.g., operators must take into consideration the impact onto existing connections provided by newlyinserted traffic. In general, an estimation of users’ and service requirements is desirable for an effective network operation, as it allows to avoid overprovisioning of network resources and to deploy resources with adequate margins at a reasonable cost. We identify the following main use cases.

Traffic prediction.
Accurate traffic prediction in the timespace domain allows operators to effectively plan and operate their networks. In the design phase, traffic prediction allows to reduce overprovisioning as much as possible. During network operation, resource utilization can be optimized by performing traffic engineering based on realtime data, eventually rerouting existing traffic and reserving resources for future incoming traffic requests.Relevant ML techniques: Through knowledge of historical data on users’ behaviour and traffic profiles in the timespace domain, a supervised learning algorithm can be trained to predict future traffic requirements and consequent resource needs. This allows network engineers to activate, e.g., proactive traffic rerouting and periodical network reoptimization so as to accommodate all users traffic and simultaneously reduce network resources utilization.
Moreover, unsupervised learning algorithms can be also used to extract common traffic patterns in different portions of the network. Doing so, similar design and management procedures (e.g., deployment and/or reservation of network capacity) can be activated also in different parts of the network, which instead show similarities in terms of traffic requirements, i.e., belonging to a same traffic profile cluster.
Note that, application of traffic prediction, and the relative ML techniques, vary substantially according to the considered network segment (e.g., approaches for intradatacenter networks may be different than those for access networks), as traffic characteristics strongly depend on the considered network segment.

Virtual topology design (VTD) and reconfiguration.
The abstraction of communication network services by means of a virtual topology is widely adopted by network operators and service providers. This abstraction consists of representing the connectivity between two endpoints (e.g., two data centers) via an adjacency in the virtual topology, (i.e., a virtual link), although the two endpoints are not necessarily physically connected. After the set of all virtual links has been defined, i.e., after all the lightpath requests have been identified, VTD requires solving a Routing and Wavelength Assignment (RWA) problem for each lightpath on top of the underlying physical network. Note that, in general, many virtual topologies can coexist in the same physical network, and they may represent, e.g., service required by different customers, or even different services, each with a specific set of requirements (e.g., in terms of QoS, bandwidth, and/or latency), provisioned to the same customer.VTD is not only necessary when a new service is provisioned and new resources are allocated in the network. In some cases, e.g., when network failures occur or when the utilization of network resources undergoes reoptimization procedures, existing (i.e., alreadydesigned) virtual topologies shall be rearranged, and in these cases we refer to the VT reconfiguration.
To perform design and reconfiguration of virtual topologies, network operators not only need to provision (or reallocate) network capacity for the required services, but may also need to provide additional resources according to the specific service characteristics, e.g., for guaranteeing service protection and/or meeting QoS or latency requirements. This type of service provisioning is often referred to as network slicing, due to the fact that each provisioned service (i.e., each VT) represents a slice of the overall network.
Relevant ML techniques: To address VTD and VT reconfiguration, ML classifiers can be trained to optimally decide how to allocate network resources, by simultaneously taking into account a large number of different and heterogeneous service requirements for a variety of virtual topologies (i.e., network slices), thus enabling fast decision making and optimized resources provisioning, especially under dynamicallychanging network conditions.

Failure management.
When managing a network, the ability to perform failure detection and localization or even to determine the cause of network failure is crucial as it may enable operators to promptly perform traffic rerouting, in order to maintain service status and meet Service Level Agreements (SLAs), and rapidly recover from the failure. Handling network failures can be accomplished at different levels. E.g., performing failure detection, i.e., identifying the set of lightpaths that were affected by a failure, is a relatively simple task, which allows network operators to only reconfigure the affected lightpaths by, e.g., rerouting the corresponding traffic. Moreover, the ability of performing also failure localization enables the activation of recovery procedures. This way, prefailure network status can be restored, which is, in general, an optimized situation from the point of view of resources utilization. Furthermore, determining also the cause of network failure, e.g., temporary traffic congestion, devices disruption, or even anomalous behaviour of failure monitors, is useful to adopt the proper restoring and traffic reconfiguration procedures, as sometimes remote reconfiguration of lightpaths can be enough to handle the failure, while in some other cases infield intervention is necessary. Moreover, prompt identification of the failure cause enables fast equipment repair and consequent reduction in Mean Time To Repair (MTTR).Relevant ML techniques: ML can help handling the large amount of information derived from the continuous activity of a huge number of network monitors and alarms. E.g., ML classifiers algorithms can be trained to distinguish between regular and anomalous (i.e., degraded) transmission. Note that, in such cases, semisupervised approaches can be also used, whenever labeled data are scarce, but a large amount of unlabeled data is available. Further, ML classifiers can be trained to distinguish failure causes, exploiting the knowledge of previously observed failures.

Traffic flow classification.
When different types of services coexist in the same network infrastructure, classifying the corresponding traffic flows before their provisioning may enable efficient resource allocation, mitigating the risk of under and overprovisioning. Moreover, accurate flow classification is also exploited for already provisioned services to apply flowspecific policies, e.g., to handle packets priority, to perform flow and congestion control, and to guarantee proper QoS to each flow according to the SLAs.Relevant ML techniques: Based on the various traffic characteristics and exploiting the large amount of information carried by data packets, supervised learning algorithms can be trained to extract hidden traffic characteristics and perform fast packets classification and flows differentiation.

Path computation.
When performing network resources allocation for an incoming service request, a proper path should be selected in order to efficiently exploit the available network resources to accommodate the requested traffic with the desired QoS and without affecting the existing services, previously provisioned in the network. Traditionally, path computation is performed by using costbased routing algorithms, such as Dijkstra, BellmanFord, Yen algorithms, which rely on the definition of a predefined cost metric (e.g., based on the distance between source and destination, the endtoend delay, the energy consumption, or even a combination of several metrics) to discriminate between alternative paths.Relevant ML techniques: In this context, use of supervised ML can be helpful as it allows to simultaneously consider several parameters featuring the incoming service request together with current network state information and map this information into an optimized routing solution, with no need for complex networkcost evaluations and thus enabling fast path selection and service provisioning.
IiiC A birdeye view of the surveyed studies
The physical and networklayer use cases described above have been tackled in existing studies by exploiting several ML tools (i.e., supervised and/or unsupervised learning, etc.) and leveraging different types of network monitored data (e.g., BER, OSNR, link load, network alarms, etc.).
In Tables I and II we summarize the various physical and networklayer use cases and highlight the features of the ML approaches which have been used in literature to solve these problems. In the tables we also indicate specific reference papers addressing these issues, which will be described in the following sections in more detail. Note that another recently published survey [33] proposes a very similar categorization of existing applications of artificial intelligence in optical networks.
Use Case  ML category  ML methodology  Input data  Output data  Training data  Ref. 
QoT estimation  supervised  kriging, norm minimization  OSNR (historical data)  OSNR  synthetic  [34] 
OSNR/Qfactor  BER  synthetic  [35, 36]  
OSNR/PMD/CD/SPM  blocking prob.  synthetic  [37]  
CBR  error vector magnitude, OSNR  Qfactor  real  [38]  
lightpath route, length, number of copropagating lightpaths  Qfactor  synthetic  [39, 40]  
RF  lightpath route, length, MF, traffic volume  BER  synthetic  [41]  
regression  SNR (historical data)  SNR  synthetic  [42]  
NN  lightpath route and length, number of traversed EDFAs, degree of destination, used channel wavelength  Qfactor  synthetic  [43, 44]  
knearest neighbor, RF, SVM  total link length, span length, channel launch power, MF and data rate  BER  synthetic  [45]  
NN  channel loadings and launch power settings  Qfactor  real  [46]  
NN  sourcedestination nodes, link occupation, MF, path length, data rate  BER  real  [47]  
OPM  supervised  NN  eye diagram and amplitude histogram param.  OSNR/PMD/CD  real  [48] 
NN, SVM  asynchronous amplitude histogram  MF  real  [49]  
NN  asyncrhonous constellation diagram and amplitude histogram param.  OSNR/PMD/CD  synthetic  [50, 51, 52, 53]  
Kernelbased ridge regression 
eye diagram and phase portraits param.  PMD/CD  real  [54]  
NN  Horizontal and Vertical polarized I/Q samples from ADC  OSNR, MF, symbol rate  real  [55]  
Gaussian Processes  monitoring data (OSNR vs )  Qfactor  real  [56]  
Optical amplifiers control  supervised  CBR  power mask param. (NF, GF)  OSNR  real  [57, 58] 
NNs  EDFA input/output power  EDFA operating point  real  [59, 60]  
Ridge regression, Kernelized Bayesian regr.  WDM channel usage  postEDFA power discrepancy  real  [61]  
unsupervised  evolutional alg.  EDFA input/output power  EDFA operating point  real  [62]  
MF recognition  unsupervised  6 clustering alg.  Stokes space param.  MF  synthetic  [63] 
kmeans  received symbols  MF  real  [64]  
supervised  NN  asynchronous amplitude histogram  MF  synthetic  [65]  
NN, SVM  asynchronous amplitude histogram  MF  real  [66],[67],[49]  
variational Bayesian techn. for GMM  Stokes space param.  MF  real  [68]  
Nonlinearity mitigation  supervised  Bayesian filtering, NNs, EM  received symbols  OSNR, Symbol error rate  real  [31, 32, 69] 
ELM  received symbols  selfphase modulation  synthetic  [70]  
knearest neighbors  received symbols  BER  real  [71]  
Newtonbased SVM  received symbols  Qfactor  real  [72]  
binary SVM  received symbols  symbol decision boundaries  synthetic  [73]  
NN  received subcarrier symbols  Qfactor  synthetic  [74]  
GMM  postequalized symbols  decoded symbols with impairment estimated and/or mitigated  real  [75]  
Clustering  received constellation with nonlinearities  nonlinearity mitigated constellation points  real  [76]  
NN  sampled received signal sequences  equalized signal with reduced ISI  real  [77, 78, 79, 80, 81, 82]  
unsupervised  kmeans  received constellation  densitybased spatial constellation clusters and their optimal centroids  real  [83]  
Use Case  ML category  ML methodology  Input data  Output data  Training data  Ref. 
Traffic prediction and virtual topology (re)design  supervised  ARIMA  historical realtime traffic matrices  predicted traffic matrix  synthetic  [84], [85] 
NN  historical endtoend maximum bitrate traffic  predicted endtoend traffic  synthetic  [86], [87]  
Reinforcement learning  previous solutions of a multiobjective GA for VTD  updated VT  synthetic  [88], [89]  
Recurrent NN  historical aggregated traffic at different BBU pools  predicted BBU pool traffic  real  [90]  
NN  historical traffic in intraDC network  predicted intraDC traffic  real  [91]  
unsupervised  NMF, clustering  CDR, PoI matrix  similarity patterns in base station traffic  real  [92]  
Failure management  supervised  Bayesian Inference  BER, received power  list of failures for all lightpaths  real  [93] 
Bayesian Inference, EM  FTTH network dataset with missing data  complete dataset  real  [94], [95]  
Kriging  previously established lightpaths with already available failure localization and monitoring data  estimate of failure localization at link level for all lightpaths  real  [96]  
(1) LUCIDA: Regression and classification
(2) BANDO: Anomaly Detection 
(1) LUCIDA: historic BER and received power, notifications from BANDO
(2) BANDO: maximum BER, threshold BER at setup, monitored BER 
(1) LUCIDA: failure classification
(2) BANDO: anomalies in BER 
real  [97]  
Regression, decision tree, SVM  BER, frequencypower pairs  localized set of failures  real  [98]  
SVM, RF, NN  BER  set of failures  real  [99]  
regression and NN  optical power levels, amplifier gain, shelf temperature, current draw, internal optical power  detected faults  real  [100]  
Flow classification  supervised  HMM, EM  packet loss data  loss classification: congestionloss or contentionloss  synthetic  [101] 
NN  source/destination IP addresses, source/destination ports, transport layer protocol, packet sizes, and a set of intraflow timings within the first 40 packets of a flow  classified flow for DC  synthetic  [102]  
Path computation  supervised  QLearning  traffic requests, set of candidate paths between each sourcedestination pair  optimum paths for each sourcedestination pair to minimize burstloss probability  synthetic  [103] 
unsupervised  FCM  traffic requests, path lengths, set of modulation formats, OSNR, BER  mapping of an optimum modulation format to a lightpath  synthetic  [104] 
Iv Detailed survey of machine learning in physical layer domain
Iva Quality of Transmission estimation
QoT estimation consists of computing transmission quality metrics such as OSNR, BER, Qfactor, CD or PMD based on measurements directly collected from the field by means of optical performance monitors installed at the receiver side [105] and/or on lightpath characteristics. QoT estimation is typically applied in two scenarios:

predicting the transmission quality of unestablished lightpaths based on historical observations and measurements collected from already deployed ones;

monitoring the transmission quality of alreadydeployed lightpaths with the aim of identifying faults and malfunctions.
QoT prediction of unestablished lightpaths relies on intelligent tools, capable of predicting whether a candidate lightpath will meet the required quality of service guarantees (mapped onto OSNR, BER or Qfactor threshold values): the problem is typically formulated as a binary classification problem, where the classifier outputs a yes/no answer based on the lightpath characteristics (e.g., its length, number of links, modulation format used for transmission, overall spectrum occupation of the traversed links etc.).
In [39] a cognitive Case Based Reasoning (CBR) approach is proposed, which relies on the maintenance of a knowledge database where information on the measured Qfactor of deployed lightpaths is stored, together with their route, selected wavelength, total length, total number and standard deviation of the number of copropagating lightpaths per link. Whenever a new traffic requests arrives, the most “similar” one (where similarity is computed by means of the Euclidean distance in the multidimensional space of normalized features) is retrieved from the database and a decision is made by comparing the associated Qfactor measurement with a predefined system threshold. As a correct dimensioning and maintenance of the database greatly affect the performance of the CBR technique, algorithms are proposed to keep it up to date and to remove old or useless entries. The tradeoff between database size, computational time and effectiveness of the classification performance is extensively studied: in [40]
, the technique is shown to outperform stateoftheart ML algorithms such as Naive Bayes, J48 tree and Random Forests (RFs). Experimental results achieved with data obtained from a real testbed are discussed in
[38].A databaseoriented approach is proposed also in [42] to reduce uncertainties on network parameters and design margins, where field data are collected by a software defined network controller and stored in a central repository. Then, a QTool is used to produce an estimate of the fieldmeasured SignaltoNoise Ratio (SNR) based on educated guesses on the (unknown) network parameters and such guesses are iteratively updated by means of a gradient descent algorithm, until the difference between the estimated and the fieldmeasured SNR falls below a predefined threshold. The new estimated parameters are stored in the database and yield to new design margins, which can be used for future demands. The tradeoff between database size and ranges of the SNR estimation error are evaluated via numerical simulations.
Similarly, in the context of multicast transmission in optical network, a NN is trained in [43, 44, 46, 47]
using as features the lightpath total length, the number of traversed EDFAs, the maximum link length, the degree of destination node and the channel wavelength used for transmission of candidate lightpaths, to predict whether the Qfactor will exceed a given system threshold. The NN is trained online with data minibatches, according to the network evolution, to allow for sequential updates of the prediction model. A dropout technique is adopted during training to avoid overfitting. The classification output is exploited by a heuristic algorithm for dynamic routing and spectrum assignment, which decides whether the request must be served or blocked. The algorithm performance is assessed in terms of blocking probability.
A random forest binary classifier is adopted in [41] to predict the probability that the BER of unestablished lightpaths will exceed a system threshold. As depicted in Figure 7, the classifier takes as input a set of features including the total length and maximum link length of the candidate lightpath, the number of traversed links, the amount of traffic to be transmitted and the modulation format to be adopted for transmission. Several alternative combinations of routes and modulation formats are considered and the classifier identifies the ones that will most likely satisfy the BER requirements. In [45], a random forest classifier along with two other tools namely knearest neighbor and support vector machine are used. The authors in [45] use three of the abovementioned classifiers to associate QoT labels with a large set of lightpaths to develop a knowledge base and find out which is the best classifier. It turns out from the analysis in [45], that the support vector machine is better in performance than the other two but takes more computation time.
Two alternative approaches, namely network kriging^{7}^{7}7Extensively used in the spatial statistics literature (see [106] for details), kriging is closely related to Gaussian process regression (see chapter XV in [20]). (first described in [107]) and norm minimization (typically used in network tomography [108]), are applied in [36, 37] in the context of QoT estimation: they rely on the installation of probe lightpaths that do not carry user data but are used to gather field measurements. The proposed inference methodologies exploit the spatial correlation between the QoT metrics of probes and datacarrying lightpaths sharing some physical links to provide an estimate of the Qfactor of already deployed or perspective lightpaths. These methods can be applied assuming either a centralized decisional tool or in a distributed fashion, where each node has only local knowledge of the network measurements. As installing probe lightpaths is costly and occupies spectral resources, the tradeoff between number of probes and accuracy of the estimation is studied. Several heuristic algorithms for the placement of the probes are proposed in [34]. A further refinement of the methodologies which takes into account the presence of neighbor channels appears in [35].
Additionally, a datadriven approach using a machine learning technique, Gaussian processes nonlinear regression (GPR), is proposed and experimentally demonstrated for performance prediction of WDM optical communication systems [49]. The core of the proposed approach (and indeed of any ML technique) is generalization: first the model is learned from the measured data acquired under one set of system configurations, and then the inferred model is applied to perform predictions for a new set of system configurations. The advantage of the approach is that complex system dynamics can be captured from measured data more easily than from simulations. Accurate BER predictions as a function of input power, transmission length, symbol rate and interchannel spacing are reported using numerical simulations and proofofprinciple experimental validation for a 24 28 GBd QPSK WDM optical transmission system.
Finally, a control and management architecture integrating an intelligent QoT estimator is proposed in [109] and its feasibility is demonstrated with implementation in a real testbed.
IvB Optical amplifiers control
The operating point of EDFAs influences their Noise Figure (NF) and gain flatness (GF), which have a considerable impact on the overall ligtpath QoT. The adaptive adjustment of the operating point based on the signal input power can be accomplished by means of ML algorithms. Most of the existing studies [59, 60, 57, 58, 62] rely on a preliminary amplifier characterization process aimed at experimentally evaluating the value of the metrics of interest (e.g., NF, GF and gain control accuracy) within its power mask (i.e., the amplifier operating region, depicted in Fig. 8).
The characterization results are then represented as a set of discrete values within the operation region. In EDFA implementations, stateoftheart microcontrollers cannot easily obtain GF and NF values for points that were not measured during the characterization. Unfortunately, producing a large amount of fine grained measurements is time consuming. To address this issue, ML algorithms can be used to interpolate the mapping function over nonmeasured points.
For the interpolation, authors of [59, 60] adopt a NN implementing both feedforward and backward error propagation. Experimental results with single and cascaded amplifiers report interpolation errors below 0.5 dB. Conversely, a cognitive methodology is proposed in [57], which is applied in dynamic network scenarios upon arrival of a new lightpath request: a knowledge database is maintained where measurements of the amplifier gains of already established lightpaths are stored, together with the lightpath characteristics (e.g., number of links, total length, etc.) and the OSNR value measured at the receiver. The database entries showing the highest similarities with the incoming lightpath request are retrieved, the vectors of gains associated to their respective amplifiers are considered and a new choice of gains is generated by perturbation of such values. Then, the OSNR value that would be obtained with the new vector of gains is estimated via simulation and stored in the database as a new entry. After this, the vector associated to the highest OSNR is used for tuning the amplifier gains when the new lightpath is deployed.
An implementation of realtime EDFA setpoint adjustment using the GMPLS control plane and interpolation rule based on a weighted Euclidean distance computation is described in [58] and extended in [62] to cascaded amplifiers.
Differently from the previous references, in [61] the issue of modelling the channel dependence of EDFA power excursion is approached by defining a regression problem, where the input feature set is an array of binary values indicating the occupation of each spectrum channel in a WDM grid and the predicted variable is the postEDFA power discrepancy. Two learning approaches (i.e., the Ridge regression and Kernelized Bayesian regression models) are compared for a setup with 2 and 3 amplifier spans, in case of singlechannel and superchannel adddrops. Based on the predicted values, suggestion on the spectrum allocation ensuring the least power discrepancy among channels can be provided.
IvC Modulation format recognition
The issue of autonomous modulation format identification in digital coherent receivers (i.e., without requiring information from the transmitter) has been addressed by means of a variety of ML algorithms, including kmeans clustering [64] and neural networks [66, 67]. Papers [63] and [68] take advantage of the Stokes space signal representation (see Fig. 9 for the representation of DPBPSK, DPQPSK and DP8QAM), which is not affected by frequency and phase offsets.
The first reference compares the performance of 6 unsupervised clustering algorithms to discriminate among 5 different formats (i.e. BPSK, QPSK, 8PSK, 8QAM, 16QAM) in terms of True Positive Rate and running time depending on the OSNR at the receiver. For some of the considered algorithms, the issue of predetermining the number of clusters is solved by means of the silhouette coefficient, which evaluates the tightness of different clustering structures by considering the inter and intracluster distances. The second reference adopts an unsupervised variational Bayesian expectation maximization algorithm to count the number of clusters in the Stokes space representation of the received signal and provides an input to a cost function used to identify the modulation format. The experimental validation is conducted over PSK (with ) and QAM (with ) modulated signals.
Conversely, features extracted from asynchronous amplitude histograms sampled from the eyediagram after equalization in digital coherent transceivers are used in
[66, 67, 65] to train NNs. In [66, 67], a NN is used for hierarchical extraction of the amplitude histograms’ features, in order to obtain a compressed representation, aimed at reducing the number of neurons in the hidden layers with respect to the number of features. In [65], a NN is combined with a genetic algorithm to improve the efficiency of the weight selection procedure during the training phase. Both studies provide numerical results over experimentally generated data: the former obtains 0% error rate in discriminating among three modulation formats (PMQPSK, 16QAM and 64QAM), the latter shows the tradeoff between error rate and number of histogram bins considering six different formats (NRZOOK, ODB, NRZDPSK, RZDQPSK, PMRZQPSK and PMNRZ16QAM).
IvD Nonlinearity mitigation
One of the performance metrics commonly used for optical communication systems is the dataratedistance product. Due to the fiber loss, optical amplification needs to be employed and, for increasing transmission distance, an increasing number of optical amplifiers must be employed accordingly. Optical amplifiers add noise and to retain the signaltonoise ratio optical signal power is increased. However, increasing the optical signal power beyond a certain value will enhance optical fiber nonlinearities which leads to Nonlinear Interference (NLI) noise. NLI will impact symbol detection and the focus of many papers, such as [71, 73, 31, 69, 32, 70, 72] has been on applying ML approaches to perform optimum symbol detection.
In general, the task of the receiver is to perform optimum symbol detection. In the case when the noise has circularly symmetric Gaussian distribution, the optimum symbol detection is performed by minimizing the Euclidean distance between the received symbol and all the possible symbols of the constellation alphabet, . This type of symbol detection will then have linear decision boundaries. For the case of memoryless nonlinearity, such as nonlinear phase noise, I/Q modulator and driving electronics nonlinearity, the noise associated with the symbol may no longer be circularly symmetric. This means that the clusters in constellation diagram become distorted (elliptically shaped instead of circularly symmetric in some cases). In those particular cases, optimum symbol detection is no longer based on Euclidean distance matrix, and the knowledge and full parametrization of the likelihood function,
, is necessary. To determine and parameterize the likelihood function and finally perform optimum symbol detection, ML techniques, such as SVM, kernel density estimator, knearest neighbors and Gaussian mixture models can be employed. A gain of approximately 3 dB in the input power to the fiber has been achieved, by employing Gaussian mixture model in combination with expectation maximization, for 14 Gbaud DP 16QAM transmission over a 800 km dispersion compensated link
[31].Furthermore, in [71] a distanceweighted knearest neighbors classifier is adopted to compensate system impairments in zerodispersion, dispersion managed and dispersion unmanaged links, with 16QAM transmission, whereas in [74] NNs are proposed for nonlinear equalization in 16QAM OFDM transmission (one neural network per subcarrier is adopted, with a number of neurons equal to the number of symbols). To reduce the computational complexity of the training phase, an Extreme Learning Machine (ELM) equalizer is proposed in [70]. ELM is a NN where the weights minimizing the inputoutput mapping error can be computed by means of a generalized matrix inversion, without requiring any weight optimization step.
SVMs are adopted in [73, 72]: in [73], a battery of binary SVM classifiers is used to identify decision boundaries separating the points of a PSK constellation, whereas in [72] fast Newtonbased SVMs are employed to mitigate intersubcarrier intermixing in 16QAM OFDM transmission.
All the above mentioned approaches lead to a 0.53 dB improvement in terms of BER/Qfactor.
In the context of nonlinearity mitigation or in general, impairment mitigation, there are a group of references that implement equalization of the optical signal using a variety of ML algorithms like Gaussian mixture models [75], clustering [76], and artificial neural networks [77, 78, 79, 80, 81, 82]. In [75], the authors propose a GMM to replace the soft/hard decoder module in a PAM4 decoding process whereas in [76], the authors propose a scheme for predistortion using the ML clustering algorithm to decode the constellation points from a received constellation affected with nonlinear impairments.
In references [77, 78, 79, 80, 81, 82] that employ neural networks for equalization, usually a vector of sampled receive symbols act as the input to the neural networks with the output being equalized signal with reduced intersymbol interference (ISI). In [77], [78], and [79]
for example, a convolutional neural network (CNN) would be used to classify different classes of a PAM signal using the received signal as input. The number of outputs of the CNN will depend on whether it is a PAM4, 8, or 16 signal. The CNNbased equalizers reported in
[77, 78, 79] show very good BER performance with strong equalization capabilities.While [77, 78, 79] report CNNbased equalizers, [81] shows another interesting application of neural network in impairment mitigation of an optical signal. In [81], a neural network approximates very efficiently the function of digital backpropagation (DBP), which is a wellknown technique to solve the nonlinear Schroedinger equation using splitstep Fourier method (SSFM) [110]. In [80] too, a neural network is proposed to emulate the function of a receiver in a nonlinear frequency division multiplexing (NFDM) system. The proposed NNbased receiver in [80]
outperforms a receiver based on nonlinear Fourier transform (NFT) and a minimumdistance receiver.
The authors in [82] propose a neuralnetworkbased approach in nonlinearity mitigation/equalization in a radiooverfiber application where the NN receives signal samples from different users in an RadiooverFiber system and returns a impairmentmitigated signal vector.
An example of unsupervised kmeans clustering technique applied on a received signal constellation to obtain a densitybased spatial constellation clusters and their optimal centroids is reported in [83]. The proposed method proves to be an efficient, lowcomplexity equalization technique for a 64QAM longhaul coherent optical communication system.
IvE Optical performance monitoring
Artificial neural networks are well suited machine learning tools to perform optical performance monitoring as they can be used to learn the complex mapping between samples or extracted features from the symbols and optical fiber channel parameters, such as OSNR, PMD, Polarizationdependent loss (PDL), baud rate and CD. The features that are fed into the neural network can be derived using different approaches relying on feature extraction from: 1) the power eye diagrams (e.g., Qfactor, closure, variance, rootmeansquare jitter and crossing amplitude, as in
[51, 50, 53, 52, 49, 69]); 2) the twodimensional eyediagram and phase portrait [54]; 3) asynchronous constellation diagrams (i.e., vector diagrams also including transitions between symbols [51]); and 4) histograms of the asynchronously sampled signal amplitudes [52, 53]. The advantage of manually providing the features to the algorithm is that the NN can be relatively simple, e.g., consisting of one hidden layer and up to 10 hidden units and does not require large amount of data to be trained. Another approach is to simply pass the samples at the symbol level and then use more layers that act as feature extractors (i.e., performing deep learning)
[48, 55]. Note that this approach requires large amount of data due to the high dimensionality of the input vector to the NN.Besides the artificial neural network, other tools like Gaussian process models are also used which are shown to perform better in optical performance monitoring compared to linearregressionbased prediction models [56]. The authors in [56] also claims that sometimes simpler ML tools like the Gaussian Process (compared to ANN) can prove to be robust under noise uncertainties and can be easy to integrate into a network controller.
V Detailed survey of machine learning in network layer domain
Va Traffic prediction and virtual topology design
Traffic prediction in optical networks is an important phase, especially in planning for resources and upgrading them optimally. Since one of the inherent philosophy of ML techniques is to learn a model from a set of data and ‘predict’ the future behavior from the learned model, ML can be effectively applied for traffic prediction.
For example, the authors in [84], [85] propose Autoregressive Integrated Moving Average (ARIMA) method which is a supervised learning method applied on time series data [111]. In both [84] and [85] the authors use ML algorithms to predict traffic for carrying out virtual topology reconfiguration. The authors propose a network planner and decision maker (NPDM) module for predicting traffic using ARIMA models. The NPDM then interacts with other modules to do virtual topology reconfiguration.
Since, the virtual topology should adapt with the variations in traffic which varies with time, the input dataset in [84] and [85] are in the form of timeseries data. More specifically, the inputs are the realtime traffic matrices observed over a window of time just prior to the current period. ARIMA is a forecasting technique that works very well with time series data [111] and hence it becomes a preferred choice in applications like traffic predictions and virtual topology reconfigurations. Furthermore, the relatively low complexity of ARIMA is also preferable in applications where maintaining a lower operational expenditure as mentioned in [84] and [85].
In general, the choice of a ML algorithm is always governed by the tradeoff between accuracy of learning and complexity. There is no exception to the above philosophy when it comes to the application of ML in optical networks. For example, in [86] and [87], the authors present traffic prediction in an identical context as [84] and [85], i.e., virtual topology reconfiguration, using NNs. A prediction module based on NNs is proposed which generates the sourcedestination traffic matrix. This predicted traffic matrix for the next period is then used by a decision maker module to assert whether the current virtual network topology (VNT) needs to be reconfigured. According to [87], the main motivation for using NNs is their better adaptability to changes in input traffic and also the accuracy of prediction of the output traffic based on the inputs (which are historical traffic).
In [91], the authors propose a deeplearningbased traffic prediction and resource allocation algorithm for an intradatacenter network. The deeplearningbased model outperforms not only conventional resource allocation algorithms but also a singlelayer NNbased algorithm in terms of blocking performance and resource occupation efficiency. The results in [91]
also bolsters the fact reflected in the previous paragraph about the choice of a ML algorithm. Obviously deep learning, which is more complex than a regular NN learning will be more efficient. Sometimes the application type also determines which particular variant of a general ML algorithm should be used. For example, recurrent neural networks (RNN), which best suits application that involve time series data is applied in
[90], to predict baseband unit (BBU) pool traffic in a 5G cloud Radio Access Network. Since the traffic aggregated at different BBU pool comprises of different classes such as residential traffic, office traffic etc., with different time variations, the historical dataset for such traffic always have a time dimension. Therefore, the authors in [90] propose and implement with good effect (a 7% increase in network throughput and an 18% processing resource reduction is reported) a RNNbased traffic prediction system.Reference [112] reports a cognitive network management module in relation to the ApplicationBased Network Operations (ABNO) framework, with specific focus on MLbased traffic prediction for VNT reconfiguration. However, [112] does not mention about the details of any specific ML algorithm used for the purpose of VNT reconfiguration. On similar lines, [113] proposes bayesian inference to estimate network traffic and decide whether to reconfigure a given virtual network.
While most of the literature focuses on traffic prediction using ML algorithms with a specific view of virtual network topology reconfigurations, [92] presents a general framework of traffic pattern estimation from call data records (CDR). [92] uses real datasets from service providers and operates matrix factorization and clustering based algorithms to draw useful insights from those data sets, which can be utilized to better engineer the network resources. More specifically, [92] uses CDRs from different base stations from the city of Milan. The dataset contains information like cell ID, time interval of calls, country code, received SMS, sent SMS, received calls, sent calls, etc., in the form of a matrix called CDR matrix. Apart from the CDR matrix, the input dataset also includes a pointofinterest (POI) matrix which contains information about different points of interests or regions most likely visited corresponding to each base station. All these input matrices are then applied to a ML clustering algorithm called nonnegative matrix factorization (NMF) and a variant of it called collective NMF (CNMF). The output of the algorithms factors the input matrices into two nonnegative matrices one of which gives the different types basic traffic patterns and the other gives similarities between base stations in terms of the traffic patterns.
While many of the references in the literature focus on one or few specific features when developing ML algorithms for traffic prediction and virtual topology (re)configurations, others just mention a general framework with some form of ‘cognition’ incorporated in association with regular optimization algorithms. For example, [88] and [89] describes a multiobjective Genetic Algorithm (GA) for virtual topology design. No specific machine learning algorithm is mentioned in [88] and [89], but they adopt adaptive fitness function update for GA. Here they use the principles of reinforcement learning where previous solutions of the GA for virtual topology design are used to update the fitness function for the future solutions.
VB Failure management
ML techniques can be adopted to either identify the exact location of a failure or malfunction within the network or even to infer the specific type of failure. In [96], network kriging is exploited to localize the exact position of failure along network links, under the assumption that the only information available at the receiving nodes (which work as monitoring nodes) of already established lightpaths is the number of failures encountered along the lightpath route. If unambiguous localization cannot be achieved, lightpath probing may be operated in order to provide additional information, which increases the rank of the routing matrix. Depending on the network load, the number of monitoring nodes necessary to ensure unambiguous localization is evaluated. Similarly, in [93]
the measured time series of BER and received power at lightpath end nodes are provided as input to a Bayesian network which individuates whether a failure is occurring along the lightpath and try to identify the cause (e.g., tight filtering or channel interference), based on specific attributes of the measurement patterns (such as maximum, average and minimum values, presence and amplitude of steps). The effectiveness of the Bayesian classifier is assessed in an experimental testbed: results show that only 0.8% of the tested instances were misclassified.
Other instances of application of Bayesian models to detect and diagnose failures in optical networks, especially GPON/FTTH, are reported in [94] and [95]. In [94], the GPON/FTTH network is modeled as a Bayesian Network using a layered approach identical to one of their previous works [114]
. The layer 1 in this case actually corresponds to the physical network topology consisting of ONTs, ONUs and fibers. Failure propagation, between different network components depicted by layer1 nodes, is modeled in layer 2 using a set of directed acyclic graphs interconnected via the layer 1. The uncertainties of failure propagation are then handled by quantifying strengths of dependencies between layer 2 nodes with conditional probability distributions estimated from network generated data. However, some of these network generated data can be missing because of improper measurements or nonreporting of data. An Expectation Maximization (EM) algorithm is therefore used to handle missing data for rootcause analysis of network failures and helps in selfdiagnosis. Basically, the EM algorithm estimates the missing data such that the estimate maximizes the expected loglikelihood function based on a given set of parameters. In
[95] a similar combination of Bayesian probabilistic models and EM is used for failure diagnosis in GPON/FTTH networks.In the context of failure detection, in addition to Bayesian networks, other machine learning algorithms and concepts have also been used. For example, in [97], two ML based algorithms are described based on regression, classification, and anomaly detection. The authors propose a BER anomaly detection algorithm which takes as input historical information like maximum BER, threshold BER at setup, and monitored BER per lightpath and detects any abrupt changes in BER which might be a result of some failures of components along a lightpath. This BER anomaly detection algorithm, which is termed as BANDO, runs on each node of the network. The outputs of BANDO are different events denoting whether the BER is above a certain threshold or below it or within a predefined boundary.
This information is then passed on to the input of another ML based algorithm which the authors term as LUCIDA. LUCIDA runs in the network controller and takes historic BER, historic received power, and the outputs of BANDO as input. These inputs are converted into three features that can be quantified by time series and they are as follows: 1) Received power above the reference level (PRXhigh); 2) BER positive trend (BERTrend); and 3) BER periodicity (BERPeriod). LUCIDA computes these features’ probabilities and the probabilities of possible failure classes and finally maps these feature probabilities to failure probabilities. In this way, LUCIDA detects the most likely failure cause from a set of failure classes.
Another notable use case for failure detection in optical networks using ML concepts appear in [98]. Two algorithms are proposed viz., Testing optIcal Switching at connection SetUp time (TISSUE) and FailurE causE Localization for optIcal NetworkinG (FEELING). The TISSUE algorithm takes the values of estimated BER calculated at each node across a lightpath and the measured BER and compares them. If the differences between the slopes of the estimated and theoretical BER is above a certain threshold a failure is anticipated. While it is not clear from [98] whether the estimation of BER in the TISSUE algorithm is based on ML methods, the FEELING algorithm applies two very wellknown ML methods viz., decision tree and SVM.
In FEELING, the first step is to process the input dataset in the form of ordered pairs of frequency and power for each optical signal and transform them into a set of features. The features include some primary features like the power levels across the central frequency of the signal and also the power around other cutoff points of the signal spectrum (interested readers are encouraged to look into
[98] for further details). In context of the FEELING algorithm, some secondary features are also defined in [98] which are linear combinations of the primary features. The featureextraction process is undertaken by a module named FeX. The next step is to input these features into a multiclass classifier in the form of a decision tree which outputs a predicted class among three options: ‘Normal’, ‘LaserDrift’ and ‘FilterFailure’; and ii) a subset of relevant signal points for the predicted class. Basically, the decision tree contains a number of decision rules to map specific combinations of feature values to classes. This decisiontreebased component runs in another module named signal spectrum verification (SSV) module. The FeX and SSV modules are located in the network nodes. There are two more modules called signal spectrum comparison (SSC) module and laser drift estimator (LDE) module which runs on the network controller.In the SSC module, a similar classification process takes place as in SSV. But here a signal is diagnosed based on the different classes of failures just due to filtering. Here the three classes are: Normal, FilterShift and TightFiltering. The SSC module uses Support Vector Machines to classify the signals based on the above three classes. First, the SVM classifies whether the signal is ‘Normal’ or has suffered a filterrelated failure. Next, the SVM classifies the signal suffering from filterrelated failures into two classes based on whether the failure is due to tight filtering or due to filter shift. Once these classifications are done, the magnitude of failures related to each of these classes are estimated using some linear regression based estimator modules for each of the failure classes. Finally, all these information provided by the different modules described so far, are used in the FEELING algorithm to return a final list of failures.
A similar multiML algorithm based framework like [98] for failure detection and classification is also proposed in [99] and [100]. In [99] several ML algorithms are used and, by tuning several model parameters, such as BER sampling time and amount of BER data needed to train the models, one or more proper optimized algorithm(s) is/are chosen from Binary and Multiclass SVMs, Random Forests and neural networks. Moreover, in paper [99], the authors propose the detection and cause identification algorithms suggesting that a network operator, able to earlydetect a failure (and identify its cause) before a critical BER threshold is reached, can proactively reroute the affected traffic onto a new lightpath, so as to minimize SLA violation and enhance (i.e., speed up) failure recovery procedures (see Fig. 11).
In [100], optical power levels, amplifier gain, shelf temperature, current draw, internal optical power are used to predict failures using statistical regression and neural network based algorithms that sit in the SDN controllers.
VC Flow classification
Another popular area of ML application for optical networks is flow classification. In [101]
for example, a framework is described that observes different types of packet loss in optical burstswitched (OBS) networks. It then classifies the packet loss data as congestion loss or contention loss using a Hidden Markov Model (HMM) and EM algorithms.
Another example of flow classification is presented in [102]. Here a NN is trained to classify flows in an optical data center network. The feature vector includes a 5tuple (source IP address, destination IP address, source port, destination port, transport layer protocol). Packet sizes and a set of intraflow timings within the first 40 packets of a flow, which roughly corresponds to the first 30 TCP segments, are also used as inputs to improve the training speed and to mitigate the problem of ‘disappearing gradients’ while using gradient descent for backpropagation.
The main outcome of the NN used in [102]
is the classification of mice and elephant flows in the data center (DC). The type of neural network used is a multilayer perceptron (MLP) with four hidden layers as MLPs are relatively simpler to implement. The authors of
[102] also mention the high levels of true negative classification associated with MLPs, and comment on importance of ensuring that mice do not flood the optical interconnections in the DC network. In general, mice flows do actually outnumber elephant flows in a practical DC network, and therefore the authors in [102] suggest to overcome this class imbalance between mice and elephant flows by training the NN with a nonproportional amount of mice and elephant flows.VD Path computation
Path computation or selection, based on different physical and network layer parameters, is a commonly studied problem in optical networks. In Section IV for example, physical layer parameters like QoT, modulation format, OSNR, etc. are estimated using ML techniques. The main aim is to make a decision about the best optical path to be selected among different alternatives. The overall path computation process can therefore be viewed as a crosslayer method with application of machine learning techniques in multiple layers. In this subsection we identify references [103] and [104] that addresses the path computation/selection in optical networks from a network layer perspective.
In [103] the authors propose a path and wavelength selection strategy for OBS networks to minimize burstloss probability. The problem is formulated as a multiarm bandit problem (MABP) and solved using Qlearning. An MABP problem comes from the context of gambling where a player tries to pull one of the arms of a slot machine with the objective to maximize sum of rewards over many such pulls of arms. In the OBS network scenario, the authors in [103] use the concept of path selection for each sourcedestination pair as pulling of one of the arms in a slot machine with the reward being minimization of burstloss probability. In general the MABP problem is a classical problem in reinforcement learning and the authors propose Qlearning to solve this problem because other methods does not scale well for complex problems. Furthermore, other methods of solving MABP, like dynamic programming, Gittins indices, and learning automata prove to be difficult when the reward distributions (i.e., the distributions of the burstloss probability in case of the OBS scenario) are unknown. The authors in [103] also argue that the Qlearning algorithm has a guaranteed convergence compared to other methods of solving the MABP problem.
In [104] a control plane decision making module for QoSaware path computation is proposed using a Fuzzy CMeans Clustering (FCM) algorithm. The FCM algorithm is added to the softwaredefined optical network (SDON) control plane in order to achieve better network performance, when compared with a noncognitive control plane. The FCM algorithm takes traffic requests, lightpath lengths, set of modulation formats, OSNR, BER etc., as input and then classifies each lightpath with the best possible parameters of the physical layer. The output of the classification is a mapping of each lightpath with a different physical layer parameter and how closely a lightpath is associated with a physical layer parameter in terms of a membership score. This membership score information is then utilized to generate some rules based on which real time decisions are taken to set up the lightpaths.
As we can see from the overall discussion in this section, different ML algorithms and policies can be used based on the use cases and applications of interest. Therefore, one can envisage a concise control plane for the next generation optical networks with a repository of different ML algorithms and policies as shown in Fig. 10. The envisaged control plane in Fig. 10 can be thought of as the ‘brain’ of the network that interacts constantly with the ‘network body’ (i.e., different components like transponders, amplifies, links etc.) and react to the ‘stimuli’ (i.e., data generated by the network) and perform certain ‘actions’ (i.e., path computation, virtual topology (re)configurations, flow classification etc.). A setup on similar lines as discussed above is presented in [115] where a MLbased monitoring module drives a Reconfigurable Optical Add/Drop Multiplexer (ROADM) controller which is again controlled by a SDN controller.
Vi Evaluation of machine learning algorithms in optical networks
In this section, we provide a more quantitative comparison of some of the ML applications described in Section III. To do this, we first provide an overview of the typical performance metrics adopted in ML. Then, we select some of the studies discussed in Sections IV and V, and we concentrate on how the ML algorithms used these papers are quantitatively compared using these performance metrics. For each paper, we also provide a quick description of the main outcome of this comparison.
Via Performance metrics
Use Case  Ref.  Adopted algorithms  Metrics  Outcome 

QoT estimation (BER classification)  [40]  Naive Bayes, Decision tree, RF, J4.8 tree, CBR  Accuracy, false positives  CBR has highest accuracy (above 99%) with low false positive (0.43%), decision tree reaches lowest false positive (0.02%) at the price of much lower accuracy (86%) 
QoT estimation (BER classification)  [41]  KNN, RF  Accuracy, AUC, running time  RF has higher AUC and accuracy than KNN, the training time of RF is higher than KNN but the testing time is at least one order of magnitude lower than RNN 
QoT estimation (BER classification)  [45]  KNN, RF, SVM  Accuracy, Confusion Matrix, ROC curves 
SVM has the best accuracy among all three ML algorithms, accuracy improves with size of Knowledge Base (KB) 
MF recognition in Stokes space  [63]  Kmeans, EM, DBSCAN, OPTICS, spectral clustering, Maximumlikelihood 
Running time, minimum OSNR to achieve 95% accuracy  Maximum likelihood requires lowest OSNR level and has very low running time (comparable to OPTICS, which has lowest running time but requires much higher OSNR level) 
Failure Management  [94], [95]  Bayesian Inference, EM  Confusion Matrix  The failure detection based on learning of the network parameters is more accurate compared to the case where an expert sets the parameters based on certain deterministic rules 
Failure Management  [99]  NN, RF, SVM  Accuracy versus model parameters (BER sampling time, amount of BER data etc.)  With right model parameters, binary SVM can reach up to 100% accuracy for failure detection 
Flow Classification (Loss classification in OBS networks)  [101]  HMM, EM  Misclassification probability (similar to FPR)  HMM has better accuracy and has lower misclassification probability for static traffic type compared to dynamic traffic, the misclassification probability also goes down with increasing number of wavelengths per link 
When applying ML to a classification problem, a common approach to evaluate the MLalgorithm performance is to show its classification accuracy and a meadure of the algorithm complexity, usually expressed in the form of trainingphase duration. Classification accuracy represents the fraction of the test samples which are correctly classified. Although this metric is intuitive, it turns out to be a poor metric in complex classification problems, especially when the available dataset contains an amount of samples largely unbalanced among the various classes (e.g., a binary dataset where 90% of samples belongs to one class). In these cases, the following and other measures can be used:

Confusion matrix: Given a binary classification problem, where samples in the test set belong to either a positive or a negative class, the confusion matrix gives a complete overview of the classifier performance, showing 1) the true positives () and true negatives (), i.e., the number of samples of the true and false class, respectively, which have been correctly classified, and 2) the false positives () and false negatives (), i.e., the number of samples of the true and false class, respectively, which have been misclassified. Note that, using these definitions, accuracy can be expressed as .

True Positive Rate, : This metric falls in the range and captures the ability of identifying actually positive samples in the test set (i.e., the larger, the better).

False Positive Rate, : Also this metric falls in the range, and it represents the fraction of negative samples in the test set that are incorrectly classified as positive (i.e., the lower, the better).

Receiver operating characteristic (ROC) curve: In a binary classifier, an arbitrary threshold can be set to distinguish between true and false instances; by increasing the value of , we reduce the number of instances that we classify as positive and increase the number of samples that we classify as negative; this has the effect of decreasing while correspondingly increasing , and increasing while correspondingly decreasing ; hence, both the TPR and the FPR are reduced. For different values of , the ROC curve plots the (on the vertical axis) against the (on the horizontal axis). For , all samples are classified as negative, therefore . Conversely, for , all samples are classified as positive, hence . For any classifier, its ROC curve always connects these two extremes. Classifiers capturing useful information yield a ROC curve above the diagonal in the plane, and aim at approaching the ideal classifier, which interconnects points (0,0), (0,1) and (1,1).

Area under the ROC curve (AUC): The AUC takes values in the range and captures how much a given classifier approaches the performance of an ideal classifier. While the ROC curve is an efficient graphical means to evaluate the performance of a classifier, the AUC is a synthetic numerical measure to indicate algorithm performance independently from the specific choice of the threshold .

Akaike Information Criteria (AIC): This is a metric that captures the goodness of fit for a particular model. It measures the deviation of a chosen statistical model from the ‘true model’ by defining a criteria which is a mathematical function of the number of estimated parameters by the model and the maximum likelihood function. The model with minimum AIC is considered as the best model to fit a given dataset [116].

Metrics from the optical networking field: Besides numerical and graphical metrics traditionally used in the ML context, measures from the networking field can be also adopted in combination with such metrics, in order to have a quantitative understanding of how the ML algorithm impacts on the optical network/system. E.g., an operator might be interested in the minimum number of optical performance monitors to deploy along a lightpath to correctly classify a degraded transmission with a given accuracy; similarly, the minimum OSNR and/or signal power level required at an optical receiver to correctly recognize the adopted MF. Furthermore, an operator might also wonder how often BER samples should be collected to predict or correctly localize an optical failure along a lightpath with a certain accuracy.
ViB Quantitative algorithms comparison
We now provide a schematic comparison of some ML algorithms focusing on some of the use cases discussed in Section III. To perform this comparison, we select, among the papers surveyed in Sections IV and V, those where different ML algorithms have been applied and compared with a same data set. Note that a fair quantitative comparison between algorithms in different papers is hard due to the fact that, in each paper, the various algorithms have been designed to fit with the specific available data set. As a consequence, a given algorithm may perform incredibly well if applied to a certain data set, but at the same time it may exhibit poor performance if the data set is changed, though not substantially.
Table III
provides such overview, highlighting, for each considered use case and corresponding reference, the ML algorithms and the evaluation metrics used for the comparison. In the table we also provide a synthetic description of the paper outcome.
Vii Discussion and future directions
In this section we discuss our vision on how this research area will expand in next years, focusing on some specific areas that we believe will require more attention during the next years.
ML methodologies. We notice how the vast majority of existing studies adopting ML in optical networks use offline supervised learning methods, i.e., assume that the ML algorithms are trained with historical data before being used to take decisions on the field. This assumption is often unrealistic for optical communication networks, where scenarios dynamically evolve with time due, e.g., to traffic variations or to changes in the behavior of optical components caused by aging. We thus envisage that, after learning from a batch of available past samples, other types of algorithms, in the field of semisupervised and/or unsupervised ML, could be implemented to gradually take in novel input data as they are made available by the network control plane. Under a different perspective, retraining of supervised mechanisms must be investigated to extend their applicability to, e.g., different network infrastructures (the training on a given topology might not be valid for a different topology) or to the same network infrastructure at a different point in time (the training performed in a certain week/month/year might not be valid anymore after some time). In a more general sense, novel ML techniques, developed adhoc for opticalnetworking problems might emerge. Consider, e.g., active ML algorithms, which can interactively ask the user to observe training data with specific characteristics. This way, the number of samples needed to build an accurate prediction model can be consistently reduced, which may lead to significant savings in case the dataset generation process is costly (e.g., when probe lightpaths have to be deployed).
Data availability. As of today, vendors and operators have not yet disclosed large set of field data to test the practicality of existing solutions. This problem might be partially addressed by emulating relevant events, as failures or signal degradations, over opticalnetwork testbeds, even though it is simply impossible to reproduce the diversity of scenarios of a real network in a lab environment. Moreover, even in situations of complete access to real data, for some of the use cases mentioned before, in practical assets it is difficult to collect extensive datasets during faulty operational conditions, since networks are typically dimensioned and managed via conservative design approaches which make the probability of failures negligible (at the price of underutilization of network resources).
Timescales. Scarce attention has so far been devoted to the fact that different applications might have very different timescales over which monitored data show observable and useful pattern changes (e.g., aging would make component behaviour vary slowly over time, while traffic varies quickly, and at different timescales, e.g., burst, daily, weekly, yearly level. Understading the right timescale for the monitoring of the parameters to be fed into ML algorithms is not only important to optimize the accuracy of the algorithm (and hence system performance), but it is fundamental to dimension the amout of control/monitoring bandwidth needed to actually implement the MLbased system. If a MLalgorithm works perfectly, but it requires a huge amount of data to be sampled extremely frequently, then the additional control bandwidth required will hinder the practical application of the algorithm.
A complete cognitive control system. Another important consideration is that all existing MLbased solutions have addressed specific and isolated issues in optical communications and networking. Considering that software defined networking has been demonstrated to be capable of successfully converging control through multiple network layers and technologies, such a unified control could also coordinate (orchestrate) several different applications of ML, to provide a holistic design for flexible optical networks. In fact, as seen in the literature, ML algorithms can be adopted to estimate different system characteristics at different layers, such as QoT, failure occurrences, traffic patterns, etc., some of which are mutually dependent (e.g., the QoT of a lightpath is highly related to the presence of failures along its links or in the traversed nodes), whereas others do not exhibit dependency (e.g., traffic patterns and fluctuations typically do not show any dependency on the status of the transmission equipment). More research is needed to explore the applicability and assess the benefits of MLbased unified control frameworks where all the estimated variables can be taken into account when making decisions such as where to route a new lightpath (e.g., in terms of spectrum assignment and core/mode assignment), when to reroute an existing one, or when to modify transmission parameters such as modulation format and baud rate.
Failure recovery. Another promising and innovative area for ML application paired with SDN control is network failure recovery. Stateoftheart optical network control tools are tipically configured as rulebased expert systems, i.e., a set of expert rules (IF conditions THEN actions) covering typical failure scenarios. Such rules are specialized and deterministic and usually in the order of a few tens, and cannot cover all the possible cases of malfunctions. The application of ML to this issue, in addition to its ability to take into account relevant data across all the layers of a network, could also bring in probabilistic characterization (e.g., making use of Gaussian processes, output probability distributions rather than single numerical/categorical values) thus providing much richer information with respect to currently adopted thresholdbased models.
Visualization. Developing effective visualization tools to make the informationrich outputs produced by ML algorithms immediately accessible and comprehensible to the end users is a key enabler for seamless integration of ML techniques in optical network management frameworks. Though some preliminary research steps in such direction have been done (see, e.g., [117], where bubble charts and spectrum color maps are employed to visualize network links experiencing high BER), design guidelines for intuitive visualization approaches depending on the specific aim of ML usage (e.g., network monitoring, failure identification and localization, etc.) have yet to be investigated and devised.
Commercialization and standardization. Though in its infancy, applications of ML to optical networking have already attracted the interest of network operators and optical equipment vendors, and it is expected that this attention will grow rapidly in the near future. Among the others, we notice some activities on QoT estimation optimization for margin reduction and erroraware rerouting [118], on lowmargin optical network design [119], on traffic prediction [120] and anomaly detection [121]. Furthermore, also standardization bodies have started looking at the application of ML for the resolution of networking problems. Although, to the best of our knowledge, no specific activity is currently undergoing with dedicated focus on optical networks, it is worth mentioning, e.g., ITUT focus group on ML [122], whose activities are concentrated on various aspects of future networking, such as architectures, interfaces, protocols, algorithms and data formats.
Optics for Machine Learning (vs. Machine Learning for optics). Finally, an interesting, though speculative, area of future research is the application of ML to alloptical devices and networks. Due to their inherent nonlinear behaviour, optical components could be interconnected to form structures capable of implementing learning tasks [123]. This approach represents an alloptical alternative to traditional software implementations. In [124], for example, semiconductor laser diodes were used to create a photonic neural network via timemultiplexing, taking advantage of their nonlinear reaction to power injection due to the coupling of amplitude and phase of the optical field. In [125], a ML method called “reservoir computing” is implemented via a nanophotonic reservoir constituted by a network of coupled crystal cavities. Thanks to their resonating behavior, power is stored in the cavities and generates nonlinear effects. The network is trained to reproduce periodic patterns (e.g., sums of sine waves).
To conclude, the application of ML to optical networking is a fastgrowing research topic, which sees an increasingly strong participation from industry and academic researchers. While in this section we could only provide a short discussion on possible future directions, we envisage that many more research topic will soon emerge in this area.
Viii Conclusion
Over the past decade, optical networks have been growing ‘smart’ with the introduction of software defined networking, coherent transmission, flexible grid, to name only few arising technical and technological directions. The combined progress towards highperformance hardware and intelligent software, integrated through an SDN platform provides a solid base for promising innovations in optical networking. Advanced machine learning algorithms can make use of the large quantity of data available from network monitoring elements to make them ‘learn’ from experience and make the networks more agile and adaptive.
Researchers have already started exploring the application of machine learning algorithms to enable smart optical networks and in this paper we have summarized some of the work carried out in the literature and provided insight into new potential research directions.
Glossary
ABNO  ApplicationBased Network Operations 
ANN  Artificial Neural Network 
API  Application Programming Interface 
AUC  Area Under the ROC Curve 
ARIMA  Autoregressive Integrated Moving Average 
BBU  Baseband Unit 
BER  Bit Error Rate 
BPSK  Binary Phase Shift Keying 
BVT  Bandwidth Variable Transponders 
CBR  Case Based Reasoning 
CD  Chromatic Dispersion 
CDR  Call Data Records 
CNN  Convolutional Neural Network 
CNMF  Collective Nonnegative Matrix Factorization 
CO  Central Office 
DBP  Digital Back Propagation 
DC  Data Center 
DP  Dual Polarization 
DQPSK  Differential Quadrature Phase Shift Keying 
EDFA  Erbium Doped Fiber Amplifier 
ELM  Extreme Learning Machine 
EM  Expectation Maximization 
EON  Elastic Optical Network 
FCM  Fuzzy CMeans Clustering 
FN  False negatives 
FP  False positives 
FTTH  Fibertothehome 
GA  Genetic Algorithm 
GF  Gain flatness 
GMM  Gaussian Mixture Model 
GMPLS  Generalized MultiProtocol Label Switching 
GPON  Gigabit Passive Optical Network 
GPR  Gaussian processes nonlinear regression 
HMM  Hidden Markov Model 
IP  Internet Protocol 
ISI  InterSymbol Interference 
LDE  Laser drift estimator 
MABP  Multiarm bandit problem 
MDP  Markov decision processes 
MF  Modulation Format 
MFR  Modulation Format Recognition 
ML  Machine Learning 
MLP  Multilayer perceptron 
MPLS  MultiProtocol Label Switching 
MTTR  Mean Time To Repair 
NF  Noise Figure 
NFDM  Nonlinear Frequency Division Multiplexing 
NFT  Nonlinear Fourier Transform 
NLI  Nonlinear Interference 
NMF  Nonnegative Matrix Factorization 
NN  Neural Network 
NPDM  Network planner and decision maker 
NRZ  NonReturn to Zero 
NWDM  Nyquist Wavelength Division Multiplexing 
OBS  Optical Burst Switching 
ODB  Optical Dual Binary 
OFDM  Orthogonal Frequency Division Multiplexing 
ONT  Optical Network Terminal 
ONU  Optical Network Unit 
OOK  OnOff Keying 
OPM  Optical Performance Monitoring 
OSNR  Optical SignaltoNoise Ratio 
PAM  Pulse Amplitude Modulation 
PDL  PolarizationDependent Loss 
PM  Polarizationmultiplexed 
PMD  Polarization Mode Dispersion 
POI  Point of Interest 
PON  Passive Optical Network 
PSK  Phase Shift Keying 
QAM  Quadrature Amplitude Modulation 
Qfactor  Quality factor 
QoS  Quality of Service 
QoT  Quality of Transmission 
QPSK  Quadrature Phase Shift Keying 
RF  Random Forest 
RL  Reinforcement Learning 
RNN  Recurrent Neural Network 
ROADM  Reconfigurable Optical Add/Drop Multiplexer 
ROC  Receiver operating characteristic 
RWA  Routing and Wavelength Assignment 
RZ  Return to Zero 
SBVT  Sliceable Bandwidth Variable Transponders 
SDN  Softwaredefined Networking 
SDON  Softwaredefined Optical Network 
SLA  Service Level Agreement 
SNR  SignaltoNoise Ratio 
SPM  SelfPhase Modulation 
SSC  Signal spectrum comparison 
SSFM  SplitStep Fourier Method 
SSV  Signal spectrum verification 
SVM  Support Vector Machine 
TCP  Transmission Control Protocol 
TN  True negatives 
TP  True positives 
VNT  Virtual Network Topology 
VT  Virtual Topology 
VTD  Virtual Topology Design 
WDM  Wavelength Division Multiplexing 
XPM  CrossPhase Modulation 
References
 [1] S. Marsland, Machine learning: an algorithmic perspective. CRC press, 2015.
 [2] A. L. Buczak and E. Guven, “A survey of data mining and machine learning methods for cyber security intrusion detection,” IEEE Communications Surveys & Tutorials, vol. 18, no. 2, pp. 1153–1176, Oct. 2015.
 [3] T. T. Nguyen and G. Armitage, “A survey of techniques for internet traffic classification using machine learning,” IEEE Communications Surveys & Tutorials, vol. 10, no. 4, pp. 56–76, 4th Q 2008.
 [4] M. Bkassiny, Y. Li, and S. K. Jayaweera, “A survey on machinelearning techniques in cognitive radios,” IEEE Communications Surveys & Tutorials, vol. 15, no. 3, pp. 1136–1159, Oct. 2012.
 [5] B. Mukherjee, Optical WDM networks. Springer Science & Business Media, 2006.
 [6] C. DeCusatis, “Optical interconnect networks for data communications,” Journal of Lightwave Technology, vol. 32, no. 4, pp. 544–552, 2014.
 [7] H. Song, B.W. Kim, and B. Mukherjee, “Longreach optical access networks: A survey of research challenges, demonstrations, and bandwidth assignment mechanisms,” IEEE communications surveys & tutorials, vol. 12, no. 1, 2010.
 [8] K. Zhu and B. Mukherjee, “Traffic grooming in an optical WDM mesh network,” IEEE Journal on Selected Areas in Communications, vol. 20, no. 1, pp. 122–133, Jan. 2002.
 [9] S. Ramamurthy and B. Mukherjee, “Survivable WDM mesh networks. Part IProtection,” in Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM) 1999, vol. 2, Mar. 1999, pp. 744–751.
 [10] [Online]. Available: http://www.lightreading.com/artificialintelligencemachinelearning/cspsembracemachinelearningandai/a/did/737973
 [11] [Online]. Available: http://wwwfile.huawei.com//media/CORPORATE/PDF/white%20paper/WhitePaperonTechnologicalDevelopmentsofOpticalNetworks.pdf
 [12] O. Gerstel, M. Jinno, A. Lord, and S. B. Yoo, “Elastic optical networking: A new dawn for the optical layer?” IEEE Communications Magazine, vol. 50, no. 2, Feb. 2012.
 [13] B. C. Chatterjee, N. Sarma, and E. Oki, “Routing and spectrum allocation in elastic optical networks: A tutorial,” IEEE Communications Surveys & Tutorials, vol. 17, no. 3, pp. 1776–1800, 2015.
 [14] S. Talebi, F. Alam, I. Katib, M. Khamis, R. Salama, and G. N. Rouskas, “Spectrum management techniques for elastic optical networks: A survey,” Optical Switching and Networking, vol. 13, pp. 34–48, 2014.
 [15] G. Zhang, M. De Leenheer, A. Morea, and B. Mukherjee, “A survey on OFDMbased elastic core optical networking,” IEEE Communications Surveys & Tutorials, vol. 15, no. 1, pp. 65–87, 2013.
 [16] C. M. Bishop, Pattern recognition and machine learning. Springer, 2006.
 [17] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification. John Wiley & Sons, 2012.
 [18] J. Friedman, T. Hastie, and R. Tibshirani, The elements of statistical learning. Springer series in statistics, New York, 2001, vol. 1.
 [19] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press Cambridge, 1998, vol. 1, no. 1.
 [20] K. P. Murphy, “Machine learning, a probabilistic perspective,” 2014.
 [21] I. Macaluso, D. Finn, B. Ozgul, and L. A. DaSilva, “Complexity of spectrum activity and benefits of reinforcement learning for dynamic channel selection,” IEEE Journal on Selected Areas in Communications, vol. 31, no. 11, pp. 2237–2248, Nov. 2013.
 [22] H. Ye, G. Y. Li, and B.H. Juang, “Power of deep learning for channel estimation and signal detection in OFDM systems,” IEEE Wireless Communications Letters, Sep. 2017.
 [23] T. J. O’Shea, T. Erpek, and T. C. Clancy, “Deep learning based MIMO communications,” arXiv preprint arXiv:1707.07980, July 2017.
 [24] A. Marinescu, I. Macaluso, and L. A. DaSilva, “A MultiAgent Neural Network for Dynamic Frequency Reuse in LTE Networks,” in IEEE International Conference on Communications (ICC), 2018.
 [25] K. Hornik, “Approximation capabilities of multilayer feedforward networks,” Neural networks, vol. 4, no. 2, pp. 251–257, 1991.
 [26] T. Hastie, R. Tibshirani, and J. Friedman, “Unsupervised Learning,” in The elements of statistical learning. Springer, 2009, pp. 485–585.
 [27] J. Han, J. Pei, and M. Kamber, Data mining: concepts and techniques. Elsevier, 2011.
 [28] O. Chapelle, B. Scholkopf, and A. Zien, Semisupervised learning. MIT Press, 2006.
 [29] N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” Journal of machine learning research, vol. 15, no. 1, pp. 1929–1958, June 2014.
 [30] J. Wass, J. Thrane, M. Piels, R. Jones, and D. Zibar, “Gaussian Process Regression for WDM System Performance Prediction,” in Optical Fiber Communication Conference (OFC) 2017. Optical Society of America, Mar. 2017.
 [31] D. Zibar, O. Winther, N. Franceschi, R. Borkowski, A. Caballero, V. Arlunno, M. N. Schmidt, N. G. Gonzales, B. Mao, Y. Ye, K. J. Larsen, and I. T. Monroy, “Nonlinear impairment compensation using expectation maximization for dispersion managed and unmanaged PDM 16QAM transmission,” Optics Express, vol. 20, no. 26, pp. B181–B196, Dec. 2012.
 [32] D. Zibar, M. Piels, R. Jones, and C. G. Schaeffer, “Machine Learning Techniques in Optical Communication,” IEEE/OSA Journal of Lightwave Technology, vol. 34, no. 6, pp. 1442–1452, Mar. 2016.
 [33] J. Mata, I. de Miguel, R. J. Durán, N. Merayo, S. K. Singh, A. Jukan, and M. Chamania, “Artificial intelligence (AI) methods in optical networks: A comprehensive survey,” Optical Switching and Networking, vol. 28, pp. 43–57, 2018.
 [34] M. Angelou, Y. Pointurier, D. Careglio, S. Spadaro, and I. Tomkos, “Optimized monitor placement for accurate QoT assessment in core optical networks,” IEEE/OSA Journal of Optical Communications and Networking, vol. 4, no. 1, pp. 15–24, Dec. 2012.
 [35] I. Sartzetakis, K. Christodoulopoulos, C. Tsekrekos, D. Syvridis, and E. Varvarigos, “Quality of transmission estimation in WDM and elastic optical networks accounting for space–spectrum dependencies,” IEEE/OSA Journal of Optical Communications and Networking, vol. 8, no. 9, pp. 676–688, Sep. 2016.
 [36] Y. Pointurier, M. Coates, and M. Rabbat, “Crosslayer monitoring in transparent optical networks,” IEEE/OSA Journal of Optical Communications and Networking, vol. 3, no. 3, pp. 189–198, Feb. 2011.
 [37] N. Sambo, Y. Pointurier, F. Cugini, L. Valcarenghi, P. Castoldi, and I. Tomkos, “Lightpath establishment assisted by offline QoT estimation in transparent optical networks,” IEEE/OSA Journal of Optical Communications and Networking, vol. 2, no. 11, pp. 928–937, Mar. 2010.
 [38] A. Caballero, J. C. Aguado, R. Borkowski, S. Saldaña, T. Jiménez, I. de Miguel, V. Arlunno, R. J. Durán, D. Zibar, J. B. Jensen et al., “Experimental demonstration of a cognitive quality of transmission estimator for optical communication systems,” Optics Express, vol. 20, no. 26, pp. B64–B70, Nov. 2012.
 [39] T. Jiménez, J. C. Aguado, I. de Miguel, R. J. Durán, M. Angelou, N. Merayo, P. Fernández, R. M. Lorenzo, I. Tomkos, and E. J. Abril, “A cognitive quality of transmission estimator for core optical networks,” IEEE/OSA Journal of Lightwave Technology, vol. 31, no. 6, pp. 942–951, Jan. 2013.
 [40] I. de Miguel, R. J. Durán, T. Jiménez, N. Fernández, J. C. Aguado, R. M. Lorenzo, A. Caballero, I. T. Monroy, Y. Ye, A. Tymecki et al., “Cognitive dynamic optical networks,” IEEE/OSA Journal of Optical Communications and Networking, vol. 5, no. 10, pp. A107–A118, Oct. 2013.
 [41] C. Rottondi, L. Barletta, A. Giusti, and M. Tornatore, “Machinelearning method for quality of transmission prediction of unestablished lightpaths,” IEEE/OSA Journal of Optical Communications and Networking, vol. 10, no. 2, pp. A286–A297, Feb 2018.
 [42] E. Seve, J. Pesic, C. Delezoide, and Y. Pointurier, “Learning process for reducing uncertainties on network parameters and design margins,” in Optical Fiber Communications Conference (OFC) 2017, Mar. 2017, pp. 1–3.
 [43] T. Panayiotou, G. Ellinas, and S. P. Chatzis, “A datadriven QoT decision approach for multicast connections in metro optical networks,” in International Conference on Optical Network Design and Modeling (ONDM) 2016, May 2016, pp. 1–6.
 [44] T. Panayiotou, S. Chatzis, and G. Ellinas, “Performance analysis of a datadriven qualityoftransmission decision approach on a dynamic multicastcapable metro optical network,” IEEE/OSA Journal of Optical Communications and Networking, vol. 9, no. 1, pp. 98–108, Jan. 2017.
 [45] S. Aladin and C. Tremblay, “Cognitive Tool for Estimating the QoT of New Lightpaths,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.

[46]
W. Mo, Y.K. Huang, S. Zhang, E. Ip, D. C. Kilper, Y. Aono, and T. Tajima, “ANNBased Transfer Learning for QoT Prediction in Real Time Mixed LineRate Systems,” in
Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.  [47] R. Proietti, X. Chen, A. Castro, G. Liu, H. Lu, K. Zhang, J. Guo, Z. Zhu, L. Velasco, and S. J. B. Yoo, “Experimental Demonstration of Cognitive Provisioning and Alien Wavelength Monitoring in Multidomain EON,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [48] T. Tanimura, T. Hoshida, J. C. Rasmussen, M. Suzuki, and H. Morikawa, “OSNR monitoring by deep neural networks trained with asynchronously sampled data,” in OptoElectronics and Communications Conference (OECC) 2016, Oct. 2016, pp. 1–3.
 [49] J. Thrane, J. Wass, M. Piels, J. C. M. Diniz, R. Jones, and D. Zibar, “Machine Learning Techniques for Optical Performance Monitoring From Directly Detected PDMQAM Signals,” IEEE/OSA Journal of Lightwave Technology, vol. 35, no. 4, pp. 868–875, Feb. 2017.
 [50] X. Wu, J. A. Jargon, R. A. Skoog, L. Paraschis, and A. E. Willner, “Applications of artificial neural networks in optical performance monitoring,” IEEE/OSA Journal of Lightwave Technology, vol. 27, no. 16, pp. 3580–3589, June 2009.
 [51] J. A. Jargon, X. Wu, H. Y. Choi, Y. C. Chung, and A. E. Willner, “Optical performance monitoring of QPSK data channels by use of neural networks trained with parameters derived from asynchronous constellation diagrams,” Optics Express, vol. 18, no. 5, pp. 4931–4938, Mar. 2010.
 [52] T. S. R. Shen, K. Meng, A. P. T. Lau, and Z. Y. Dong, “Optical performance monitoring using artificial neural network trained with asynchronous amplitude histograms,” IEEE Photonics Technology Letters, vol. 22, no. 22, pp. 1665–1667, Nov. 2010.

[53]
F. N. Khan, T. S. R. Shen, Y. Zhou, A. P. T. Lau, and C. Lu, “Optical performance monitoring using artificial neural networks trained with empirical moments of asynchronously sampled signal amplitudes,”
IEEE Photonics Technology Letters, vol. 24, no. 12, pp. 982–984, June 2012.  [54] T. B. Anderson, A. Kowalczyk, K. Clarke, S. D. Dods, D. Hewitt, and J. C. Li, “Multi impairment monitoring for optical networks,” IEEE/OSA Journal of Lightwave Technology, vol. 27, no. 16, pp. 3729–3736, Aug. 2009.
 [55] T. Tanimura, T. Hoshida, T. Kato, S. Watanabe, and H. Morikawa, “Dataanalyticsbased Optical Performance Monitoring Technique for Optical Transport Networks,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [56] F. Meng, S. Yan, K. Nikolovgenis, Y. Ou, R. Wang, Y. Bi, E. HuguesSalas, R. Nejabati, and D. Simeonidou, “Field Trial of Gaussian Process Learning of FunctionAgnostic Channel Performance Under Uncertainty,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [57] U. Moura, M. Garrich, H. Carvalho, M. Svolenski, A. Andrade, A. C. Cesar, J. Oliveira, and E. Conforti, “Cognitive methodology for optical amplifier gain adjustment in dynamic DWDM networks,” IEEE/OSA Journal of Lightwave Technology, vol. 34, no. 8, pp. 1971–1979, Jan. 2016.
 [58] J. R. Oliveira, A. Caballero, E. Magalhães, U. Moura, R. Borkowski, G. Curiel, A. Hirata, L. Hecker, E. Porto, D. Zibar et al., “Demonstration of EDFA cognitive gain control via GMPLS for mixed modulation formats in heterogeneous optical networks,” in Optical Fiber Communication Conference (OFC) 2013, Mar. 2013, pp. OW1H–2.
 [59] E. d. A. Barboza, C. J. BastosFilho, J. F. MartinsFilho, U. C. de Moura, and J. R. de Oliveira, “Selfadaptive erbiumdoped fiber amplifiers using machine learning,” in SBMO/IEEE MTTS International Microwave & Optoelectronics Conference (IMOC), 2013, Oct. 2013, pp. 1–5.
 [60] C. J. BastosFilho, E. d. A. Barboza, J. F. MartinsFilho, U. C. de Moura, and J. R. de Oliveira, “Mapping EDFA noise figure and gain flatness over the power mask using neural networks,” Journal of Microwaves, Optoelectronics and Electromagnetic Applications (JMOe), vol. 12, pp. 128–139, July 2013.
 [61] Y. Huang, C. L. Gutterman, P. Samadi, P. B. Cho, W. Samoud, C. Ware, M. Lourdiane, G. Zussman, and K. Bergman, “Dynamic mitigation of EDFA power excursions with machine learning,” Optics Express, vol. 25, no. 3, pp. 2245–2258, Feb. 2017.
 [62] U. C. de Moura, J. R. Oliveira, J. C. Oliveira, and A. C. César, “EDFA adaptive gain control effect analysis over an amplifier cascade in a DWDM optical system,” in SBMO/IEEE MTTS International Microwave & Optoelectronics Conference (IMOC) 2013, Oct. 2013, pp. 1–5.
 [63] R. Boada, R. Borkowski, and I. T. Monroy, “Clustering algorithms for Stokes space modulation format recognition,” Optics Express, vol. 23, no. 12, pp. 15 521–15 531, June 2015.
 [64] N. G. Gonzalez, D. Zibar, and I. T. Monroy, “Cognitive digital receiver for burst mode phase modulated radio over fiber links,” in European Conference on Optical Communication (ECOC) 2010, Sep. 2010, pp. 1–3.
 [65] S. Zhang, Y. Peng, Q. Sui, J. Li, and Z. Li, “Modulation format identification in heterogeneous fiberoptic networks using artificial neural networks and genetic algorithms,” Photonic Network Communications, vol. 32, no. 2, pp. 246–252, Feb. 2016.
 [66] F. N. Khan, Y. Zhou, A. P. T. Lau, and C. Lu, “Modulation format identification in heterogeneous fiberoptic networks using artificial neural networks,” Optics Express, vol. 20, no. 11, pp. 12 422–12 431, May 2012.
 [67] F. N. Khan, K. Zhong, W. H. AlArashi, C. Yu, C. Lu, and A. P. T. Lau, “Modulation format identification in coherent receivers using deep machine learning,” IEEE Photonics Technology Letters, vol. 28, no. 17, pp. 1886–1889, Sep. 2016.
 [68] R. Borkowski, D. Zibar, A. Caballero, V. Arlunno, and I. T. Monroy, “Stokes spacebased optical modulation format recognition for digital coherent receivers,” IEEE Photonics Technology Letters, vol. 25, no. 21, pp. 2129–2132, Nov. 2013.
 [69] D. Zibar, J. Thrane, J. Wass, R. Jones, M. Piels, and C. Schaeffer, “Machine learning techniques applied to system characterization and equalization,” in Optical Fiber Communications Conference (OFC) 2016, Mar. 2016, pp. 1–3.
 [70] T. S. R. Shen and A. P. T. Lau, “Fiber nonlinearity compensation using extreme learning machine for DSPbased coherent communication systems,” in OptoElectronics and Communications Conference (OECC) 2011, July 2011, pp. 816–817.
 [71] D. Wang, M. Zhang, M. Fu, Z. Cai, Z. Li, H. Han, Y. Cui, and B. Luo, “Nonlinearity Mitigation Using a Machine Learning Detector Based on Nearest Neighbors,” IEEE Photonics Technology Letters, vol. 28, no. 19, pp. 2102–2105, Apr. 2016.
 [72] E. Giacoumidis, S. Mhatli, M. F. Stephens, A. Tsokanos, J. Wei, M. E. McCarthy, N. J. Doran, and A. D. Ellis, “Reduction of Nonlinear Intersubcarrier Intermixing in Coherent Optical OFDM by a Fast NewtonBased Support Vector Machine Nonlinear Equalizer,” IEEE/OSA Journal of Lightwave Technology, vol. 35, no. 12, pp. 2391–2397, Mar. 2017.
 [73] D. Wang, M. Zhang, Z. Li, Y. Cui, J. Liu, Y. Yang, and H. Wang, “Nonlinear decision boundary created by a machine learningbased classifier to mitigate nonlinear phase noise,” in European Conference on Optical Communication (ECOC) 2015, Oct. 2015, pp. 1–3.
 [74] M. A. Jarajreh, E. Giacoumidis, I. Aldaya, S. T. Le, A. Tsokanos, Z. Ghassemlooy, and N. J. Doran, “Artificial neural network nonlinear equalizer for coherent optical OFDM,” IEEE Photonics Technology Letters, vol. 27, no. 4, pp. 387–390, Feb. 2015.
 [75] F. Lu, P.C. Peng, S. Liu, M. Xu, S. Shen, and G.K. Chang, “Integration of Multivariate Gaussian Mixture Model for Enhanced PAM4 Decoding Employing Basis Expansion,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [76] X. Lu, M. Zhao, L. Qiao, and N. Chi, “Nonlinear Compensation of MultiCAP VLC System Employing PreDistortion Base on Clustering of Machine Learning,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [77] P. Li, L. Yi, L. Xue, and W. Hu, “56 Gbps IM/DD PON based on 10GClass Optical Devices with 29 dB Loss Budget Enabled by Machine Learning,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [78] C.Y. Chuang, L.C. Liu, C.C. Wei, J.J. Liu, L. Henrickson, W.J. Huang, C.L. Wang, Y.K. Chen, and J. Chen, “Convolutional Neural Network based Nonlinear Classifier for 112Gbps High Speed Optical Link,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [79] P. Li, L. Yi, L. Xue, and W. Hu, “100Gbps IM/DD Transmission over 25km SSMF using 20G class DML and PIN Enabled by Machine Learning,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.

[80]
R. T. Jones, S. Gaiarin, M. P. Yankov, and D. Zibar, “Noise Robust Receiver for Eigenvalue Communication Systems,” in
Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.  [81] C. Hager and H. D. Pfister, “Nonlinear Interference Mitigation via Deep Neural Networks,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [82] S. Liu, Y. M. Alfadhli, S. Shen, H. Tian, and G.K. Chang, “Mitigation of Multiuser Access Impairments in 5G ARoFbased Mobile Fronthaul utilizing Machine Learning for an Artificial Neural Network Nonlinear Equalizer,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [83] J. Zhang, W. Chen, M. Gao, B. Chen, and G. Shen, “Novel LowComplexity FullyBlind DensityCentroid – Tracking Equalizer for 64QAM Coherent Optical Communication Systems,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [84] N. Fernández, R. J. Durán, I. de Miguel, N. Merayo, P. Fernández, J. C. Aguado, R. M. Lorenzo, E. J. Abril, E. Palkopoulou, and I. Tomkos, “Virtual Topology Design and reconfiguration using cognition: Performance evaluation in case of failure,” in 5th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT) 2013, Sep. 2013, pp. 132–139.
 [85] N. Fernández, R. J. D. Barroso, D. Siracusa, A. Francescon, I. de Miguel, E. Salvadori, J. C. Aguado, and R. M. Lorenzo, “Virtual topology reconfiguration in optical networks by means of cognition: Evaluation and experimental validation [Invited],” IEEE/OSA Journal of Optical Communications and Networking, vol. 7, no. 1, pp. A162–A173, Jan. 2015.
 [86] F. Morales, M. Ruiz, and L. Velasco, “Virtual network topology reconfiguration based on big data analytics for traffic prediction,” in Optical Fiber Communications Conference (OFC) 2016, Mar. 2016, pp. 1–3.
 [87] F. Morales, M. Ruiz, L. Gifre, L. M. Contreras, V. Lopez, and L. Velasco, “Virtual network topology adaptability based on data analytics for traffic prediction,” IEEE/OSA Journal of Optical Communications and Networking, vol. 9, no. 1, pp. A35–A45, Jan. 2017.
 [88] N. Fernández, R. J. Durán, I. de Miguel, N. Merayo, D. Sánchez, M. Angelou, J. C. Aguado, P. Fernández, T. Jiménez, R. M. Lorenzo, I. Tomkos, and E. J. Abril, “Cognition to design energetically efficient and impairment aware virtual topologies for optical networks,” in International Conference on Optical Network Design and Modeling (ONDM) 2012, Apr. 2012, pp. 1–6.
 [89] N. Fernández, R. J. Durán, I. de Miguel, N. Merayo, J. C. Aguado, P. Fernández, T. Jiménez, I. Rodríguez, D. Sánchez, R. M. Lorenzo, E. J. Abril, M. Angelou, and I. Tomkos, “Survivable and impairmentaware virtual topologies for reconfigurable optical networks: A cognitive approach,” in IV International Congress on Ultra Modern Telecommunications and Control Systems 2012, Oct. 2012, pp. 793–799.
 [90] W. Mo, C. L. Gutterman, Y. Li1, G. Zussman, and D. C. Kilper, “Deep Neural Network Based Dynamic Resource Reallocation of BBU Pools in 5G CRAN ROADM Networks,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [91] A. Yu, H. Yang, W. Bai, L. He, H. Xiao, and J. Zhang, “Leveraging Deep Learning to Achieve Efficient Resource Allocation with Traffic Evaluation in Datacenter Optical Networks,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [92] S. Troía, G. Sheng, R. Alvizu, G. A. Maier, and A. Pattavina, “Identification of tidaltraffic patterns in metroarea mobile networks via Matrix Factorization based model,” in International Conference on Pervasive Computing and Communications Workshop (PerCom) 2017, Mar. 2017, pp. 297–301.
 [93] M. Ruiz, F. Fresi, A. P. Vela, G. Meloni, N. Sambo, F. Cugini, L. Poti, L. Velasco, and P. Castoldi, “Servicetriggered failure identification/localization through monitoring of multiple parameters,” in European Conference on Optical Communication (ECOC) 2016, Sep. 2016, pp. 1–3.
 [94] S. R. Tembo, S. Vaton, J. L. Courant, and S. Gosselin, “A tutorial on the EM algorithm for Bayesian networks: Application to selfdiagnosis of GPONFTTH networks,” in International Wireless Communications and Mobile Computing Conference (IWCMC) 2016, Sep. 2016, pp. 369–376.
 [95] S. Gosselin, J. L. Courant, S. R. Tembo, and S. Vaton, “Application of probabilistic modeling and machine learning to the diagnosis of FTTH GPON networks,” in International Conference on Optical Network Design and Modeling (ONDM) 2017, May 2017, pp. 1–3.
 [96] K. Christodoulopoulos, N. Sambo, and E. M. Varvarigos, “Exploiting network kriging for fault localization,” in Optical Fiber Communication Conference (OFC) 2016, Mar. 2016, pp. W1B–5.
 [97] A. P. Vela, M. Ruiz, F. Fresi, N. Sambo, F. Cugini, G. Meloni, L. Potì, L. Velasco, and P. Castoldi, “Ber degradation detection and failure identification in elastic optical networks,” IEEE/OSA Journal of Lightwave Technology, vol. 35, no. 21, pp. 4595–4604, Nov. 2017.
 [98] A. P. Vela, B. Shariati, M. Ruiz, F. Cugini, A. Castro, H. Lu, R. Proietti, J. Comellas, P. Castoldi, S. J. B. Yoo, and L. Velasco, “Soft Failure Localization During Commissioning Testing and Lightpath Operation,” IEEE/OSA Journal of Optical Communications and Networking, vol. 10, no. 1, pp. A27–A36, Jan. 2018.
 [99] S. Shahkarami, F. Musumeci, F. Cugini, and M. Tornatore, “MachineLearningBased SoftFailure Detection and Identification in Optical Networks,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [100] D. Rafique, T. Szyrkowiec, A. Autenrieth, and J.P. Elbers, “AnalyticsDriven Fault Discovery and Diagnosis for Cognitive Root Cause Analysis,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [101] A. Jayaraj, T. Venkatesh, and C. S. Murthy, “Loss Classification in Optical Burst Switching Networks Using Machine Learning Techniques: Improving the Performance of TCP,” IEEE Journal on Selected Areas in Communications, vol. 26, no. 6, pp. 45–54, Aug. 2008.
 [102] H. Rastegarfar, M. Glick, N. Viljoen, M. Yang, J. Wissinger, L. Lacomb, and N. Peyghambarian, “TCP flow classification and bandwidth aggregation in optically interconnected data center networks,” IEEE/OSA Journal of Optical Communications and Networking, vol. 8, no. 10, pp. 777–786, Oct. 2016.
 [103] Y. V. Kiran, T. Venkatesh, and C. S. Murthy, “A Reinforcement Learning Framework for Path Selection and Wavelength Selection in Optical Burst Switched Networks,” IEEE Journal on Selected Areas in Communications, vol. 25, no. 9, pp. 18–26, Dec. 2007.
 [104] T. R. Tronco, M. Garrich, A. C. César, and M. d. L. Rocha, “Cognitive algorithm using fuzzy reasoning for softwaredefined optical network,” Photonic Network Communications, vol. 32, no. 2, pp. 281–292, Oct. 2016.
 [105] K. Christodoulopoulos et al., “ORCHESTRAOptical performance monitoring enabling flexible networking,” in 17th International Conference on Transparent Optical Networks (ICTON) 2015, July 2015, pp. 1–4.
 [106] P. Diggle and P. Ribeiro, “Modelbased Geostatistics,” 2007.
 [107] D. B. Chua, E. D. Kolaczyk, and M. Crovella, “Network kriging,” IEEE Journal on Selected Areas in Communications, vol. 24, no. 12, pp. 2263–2272, Dec. 2006.
 [108] R. Castro, M. Coates, G. Liang, R. Nowak, and B. Yu, “Network tomography: Recent developments,” Statistical science, pp. 499–517, Aug. 2004.
 [109] M. Bouda, S. Oda, O. Vasilieva, M. Miyabe, S. Yoshida, T. Katagiri, Y. Aoki, T. Hoshida, and T. Ikeuchi, “Accurate prediction of quality of transmission with dynamically configurable optical impairment model,” in Optical Fiber Communications Conference (OFC) 2017, Mar. 2017, pp. 1–3.
 [110] E. Ip and J. M. Kahn, “Compensation of dispersion and nonlinear impairments using digital backpropagation,” Journal of Lightwave Technology, vol. 26, no. 20, pp. 3416–3425, Oct. 2008.
 [111] “ARIMA models for time series forecasting,” Oct. [Online]. Available: http://people.duke.edu/~rnau/411arim.htm
 [112] L. Gifre, F. Morales, L. Velasco, and M. Ruiz, “Big data analytics for the virtual network topology reconfiguration use case,” in 18th International Conference on Transparent Optical Networks (ICTON) 2016, July 2016, pp. 1–4.
 [113] T. Ohba, S. Arakawa, and M. Murata, “A Bayesianbased approach for virtual network reconfiguration in elastic optical path networks,” in Optical Fiber Communications Conference (OFC) 2017, Mar. 2017, pp. 1–3.
 [114] S. R. Tembo, J. L. Courant, and S. Vaton, “A 3layered selfreconfigurable generic model for selfdiagnosis of telecommunication networks,” in 2015 SAI Intelligent Systems Conference (IntelliSys), Nov. 2015, pp. 25–34.
 [115] A. Sgambelluri, J.L. IzquierdoZaragoza, A. Giorgetti, L. Gifre, L. Velasco, F. Paolucci, N. Sambo, F. Fresi, P. Castoldi, A. C. Piat, R. Morro, E. Riccardi, A. D’Errico, and F. Cugini, “Fully Disaggregated ROADM White Box with NETCONF/YANG Control, Telemetry, and Machine Learningbased Monitoring,” in Optical Fiber Communications Conference (OFC) 2018, Mar. 2018.
 [116] H. Akaike, Akaike’s Information Criterion, M. Lovric, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011.

[117]
A. P. Vela, M. Ruiz, and L. Velasco, “Applying Data Visualization for Failure Localization,” in
Optical Fiber Communication Conference, Mar. 2018.  [118] J. Simsarian, S. Plote, M. Thottan, and P. J. Winzer, “How to use Machine Learning for Testing and Implementing Optical Networks,” North American Network Operators’ Group (NANOG), 2017.
 [119] “Evolving the awareness of optical networks,” Coriant white paper, June 2018.
 [120] G. Choudhury, D. Lynch, G. Thakur, and S. Tse, “Two Use Cases of Machine Learning for SDNEnabled IP/Optical Networks: Traffic Matrix Prediction and Optical Path Performance Prediction,” to appear in IEEE/OSA Journal of Optical Communications and Networking, 2018.
 [121] D. Cote, “Using Machine Learning in Communication Networks,” to appear in IEEE/OSA Journal of Optical Communications and Networking, 2018.
 [122] “ITUT Focus Group on Machine Learning for Future Networks including 5G.” [Online]. Available: https://www.itu.int/en/ITUT/focusgroups/ml5g/Pages/default.aspx
 [123] D. Woods and T. J. Naughton, “Optical computing: Photonic neural networks,” Nature Physics, vol. 8, no. 4, pp. 257–259, July 2012.
 [124] D. Brunner, M. Soriano, C. Mirasso, and I. Fischer, “High speed, high performance alloptical information processing utilizing nonlinear optical transients,” in The European Conference on Lasers and ElectroOptics, May 2013.
 [125] M. Fiers, K. Vandoorne, T. Van Vaerenbergh, J. Dambre, B. Schrauwen, and P. Bienstman, “Optical information processing: Advances in nanophotonic reservoir computing,” in 14th International Conference on Transparent Optical Networks (ICTON) 2012, Aug. 2012, pp. 1–4.
Comments
There are no comments yet.