Cyber-Physical Systems (CPS) integrate computation, communication and physical processes. The design of a *cps involves several fields including computer science, control theory, automation, networking and distributed systems. Skills from these domains are put together to ensure that a myriad of computing resources and physical elements get orchestrated via networking technologies. In addition, *cps integrate facilities for human-computer interaction. Examples of *cps interacting with humans include industrial control systems (e.g., workers operating industrial machines) and smart cities (involving thousands of nodes and citizens).
The key ingredient of *cps is the use of control. Control means making a dynamical system work as required. It integrates the notion of feedback, used to compute corrective control actions based on the distance between a reference signal and system output. A *cps relies on a feedback control system. The plant of the *cps represents the controllable and observable physical resources. The actuators represent an abstraction of all the devices that can be used to act upon the plant. The sensors correspond to measurement and observation apparatus. Traditional representations of a *cps use the to refer to summing junctions, i.e., linear elements that output the sum of a given number of signals. Using a reference model and signals produced by the sensors, the controller generates signals to the actuators.
Although the term *cps emerged recently111Coined by H. Gill at the National Science Foundation in 2006 ., it builds upon very well-established research fields, i.e., embedded computing, control theory and human-computer interaction. For instance, a *cps can be easily modelled as a *ncs. The major difference is that the controller is coupled with the actuators and sensors through a communication network (e.g, an Ethernet-like network). The use of this communication network to connect the components provides flexibility and low implementation costs.
Attacks exploiting *ncs vulnerabilities can be characterized according to three main aspects: (a) adversary’s a priori knowledge about the system and its protective measures, (b) class of disrupted resources (e.g., denial-of-service attacks targeting elements that are crucial to operation) and (c) analysis of control signals during perpetration of an attack (e.g., sensor outputs), that may be used to carry out more sophisticated attacks (e.g., attacks targeting the integrity or availability of the system).
The knowledge of adversaries in terms of, e.g., system dynamics, feedback predictability and system countermeasures, can be used to perpetrate attacks with severe security and safety implications, when they target the operations of, e.g., industrial systems and national infrastructures. They can lead to catastrophic consequences to businesses, governments and society at large. A growing number of attacks on cyber-physical infrastructures are reported in the world, targeting vital activities (e.g., water, energy and transportation) for intelligence or sabotage purposes. Some representative incidents are outlined in Table I.
A careful review of incidents as those in Table I reveals that they all have a common element: human adversarial actions forging system feedback measurements for disruption purposes. The underlying issue, hereinafter called the feedback truthfulness problem, refers to intentional situations perpetrated by human adversaries, forging physical observations in a stealthy manner. They are cyber-physical attacks generating anomalies. However, even if detected, the attacks appear as unintentional errors. Hence, they are leading to wrong resilience plans. How to distinguish an intentional attack from an unintentional fault? This a challenge because symptoms may almost be the same, but reactions should be different. Indeed, the correct response to a fault is a repair action that restores the state of the system. In case of an intentional attack, physical resources may not be faulty at all, but the adversary makes them appear faulty. A repair action will not help.
It is crucial to address the aforementioned challenge in a provable manner in order to prioritize appropriate responses and rapidly recover control to assure cyber-physical resilience. That is, to assure the persistence of the system when facing changes, either accidental or intentional. In terms of *cps design, cyber-physical resilience shall also deal with the management of operational functionality that is crucial for a system, and that cannot be stopped. In other words, system functionality that shall be properly accomplished. Regarding the incidents mentioned in Table I, the cooling service of reactor in a nuclear plant, or the safety controls of an autonomous navigation system, are proper examples of critical functionalities. Other system functionalities may be seen as less important; and even be temporarily stopped or partially completed. Such type of functions can be seen as secondary. A printing service for employees in a nuclear plant scenario is a proper example of a secondary function that one might accept to sacrifice, under graceful degradation.
When addressing resilience, two crucial elements to take into consideration are the severity of the actions disrupting the functionalities of a system and properly distinguishing accidental failures from intentional attacks. The objective is to use the proper security stacks and deploy resilience plans, including responses that mitigate the impact of undesirable actions. This includes the use of proactive, often short-term, tactical policies to handle failures; and reactive, usually long-term, strategies for attacks. Security stacks in both areas can include redundancy (e.g., use of additional system replicas), compartamentalization, segmentation, and activation of upgraded modes of protection (e.g., use of cryptography to enable secure handshakes, message signatures, and encryption). The inclusion of resilience plans shall always keep critical processes in a normal operating mode, while the system is confronted with incidents. The challenge of satisfying those requirements on automated *cps designs stresses the importance of determining the root nature of incidents, to drive the appropriate models (e.g., in terms of remediation) that the system must select and enforce in the end.
Organization — Section II provides a more thorough introduction to the concept of resilience and the use of security stacks to enable cyber-physical protection. Section III argues the necessity of a paradigm change and discusses our vision of how next generation resilient *cps will be addressing such a change. Section IV explores some promising techniques inline with our proposal. Section V closes the paper.
Ii Resilience and Security Stacks
Resilience emphasizes the capacity of a system to recover from disruptions. It can be seen as the underlying technique by which the system regulates its safety and security mechanisms, to recover from adverse events. Resilience includes actions and plans that must be conducted before, during, and after events take place. Resilience is a historical term used as a descriptor in complex fields, from psychology and medicine to civil and military engineering. The modern application of resilience relates to the idea of how a complex system bounces back from a disruption, as well as all the possible post-disruption strategies that may come after the events are identified.
Under the scope of complex systems theory, the concept of resilience may be confused with other traditional concepts such as robustness, fault tolerance and sustainability. However, there exist fundamental differences between such terms. For instance, while robustness stands for the ability to withstand or overcome adverse conditions (e.g., faults and attacks), resilience refers more to the capacity for a system to maintain functionality despite the occurrence of some internal or external disruptions, e.g., adversarial breach. Similarly, fault tolerance refers more to the maintenance of crucial services within a given time-period under the presence of failures and sustainability to similar metaphors in disciplines like environmental and socio-ecological processes.
Laprie  settled some key definitions when comparing resilience to dependability and fault tolerance. In his work, Laprie related the resilience and dependability terms as follows: Resilience is the persistence of dependability when facing changes. More recently, the relation between resilience and performance targets have been described by Meyer  as follows: Resilience is the persistence of Performability when facing changes. This can be accomplished by graceful degradation, i.e., by prioritizing some services over non-essential ones, for as long as possible.
The concept of resilience spans across several other disciplines. For instance, when talking about resilience in terms of network theory, resilience refers to the persistence of service delivery when the network faces changes. In terms of quality of service, resilience relates to the degree of stability of the services provided by the system. From a control-theoretic standpoint, resilience refers to the ability to reduce the magnitude and duration of deviations from optimal performance levels. Finally, resilience is also seen in disciplines such as medicine and psychology, as the ability to recover from a crucial trauma or crisis. The common element seen in all the aforementioned definitions relates to adaptation to confront change and significant adversities.
When we move to the specific context of cybersecurity, resilience means accepting that the system is vulnerable to attacks, in addition to faults and failures. It means to accept that there will be breach of security (e.g., by a collusion between insiders and outsiders, attacking and disrupting the system). Handling resilience in the cybersecurity context means holding an adversarial mindset and getting ready to lose some assets. This does not mean sacrificing the system, but deciding which parts of the system we can lose (accepting that we must lose some control over the system) while prioritizing those assets we must give up to assure that the system will remain functional during the disruptions.
To improve resilience from the cybersecurity standpoint relies on enforcing a traditional security stack, in terms of identifying the system weaknesses (e.g., in their software and infrastructure themselves) that could potentially be controlled by a skilled adversary with the purpose of disrupting the system. Management in terms of identifying vulnerabilities must be followed as well by assessment of incidents, service continuity and, in general, any risks affecting the system. These aforementioned management perspectives must be driven by resilience thinking in the form of bouncing back (or defending back) from disruptive or adverse events. In other words, attacks against the availability of a given service, as well as any incident leading to security breaches must be quickly solved (e.g., incidents must properly be absorbed). Representative examples of security stacks are presented next.
Ii-a Traditional Cybersecurity Stacks
Traditional cybersecurity literature associated to the resilience of a *cps includes the use of technological barriers designed to prevent unauthorized communications between the elements of the *cps and the outside world. This includes the use of firewalls, cryptography, intrusion detection systems and security information and event management (SIEMs). Beyond these traditional solutions, we can also encounter related cybersecurity literature to ensure attack tolerance.
Attack tolerance is an extended in-depth strategy proposed to defend a system against any particular attack using several independent methods. Several proposed security solutions for *cps focus in detection and attack prevention. However, preventing every single possible attack is hard to achieve. Despite the efforts, attacks can happen and be successful. Attack tolerance is the capability of a system to continue functioning properly with minimal degradation of performance, despite the presence of attacks. Some representative techniques proposed in the literature to achieve attack tolerance follow.
Redundancy and diversity are often combined together to ensure protection beyond breach. Redundancy assumes the use of extra reserved resources allocated to a system that are beyond its need in normal working conditions. If the system finds that the output values of a primary component are not correct, then the responsibility is transferred to one of the redundant components. Diversity means that a function should be implemented in multiple ways, differently at different times. For example, research has made it practical to automatically generate diverse functionality from the same source code or automatically change the configuration of a system from time to time to confuse the adversary.
Dynamic reconfiguration takes place after the detection of an attack. In traditional systems, reconfiguration is mostly reactive and generally performed manually by the administrator. Thus, it involves some downtime. Survivable systems need an adaptive reconfiguration to be proactive, instead. Reconfiguration under the context of decentralization can be achieved by dividing trust into separated shares. Decentralized strategies can be used to assure that the system needs to reach a given threshold prior granting authorization measures. Below the threshold, information gets concealed to the eyes of the adversary. Reaching a threshold allows reconfiguration to a state that ensures the correct provision of the required functions.
|Replay of cyber-physical control data ||Adversaries replay previous measurements (corresponding to normal operation conditions) and control inputs, to disrupt the system.|
|Dynamic false-data injection ||Adversaries drive the system to unstable states, by using system vulnerabilities.|
|Bias-data injection, using system identification techniques ||Adversaries disrupt the behavior of the system, by injecting faulty data constructed to evade feedback-control detectors.|
|Covert and Stealthy ||Adversaries hold complete knowledge about the system dynamics, to impersonate the feedback controller and evade fault detection.|
|Zero Dynamics ||Adversaries make unobservable an unstable state of the system using controller vulnerabilities.|
|Linear-quadratic-Gaussian (LQG) control using authentication challenge-response signals ||Watermarking injection of noise in the system that, under normal operation, must be reflected in the outputs, withoiut modifying the system,.|
|Signal-based correlation and Cross-layer identification ||Signal-based correlation using statistical properties of system dynamics, combining control-theoretic and adaptive security.|
|Auxiliary Systems and Digital Twins ||Use of auxiliary states, outputs and (optionally) inputs, evolving in parallel with the *cps, to identify the attacks.|
Ii-B Control-Theoretic Evolutions
From a control-theoretic point of view, the secure operation of a *cps requires the maintenance of three properties: observabilty, controllability and stability. Observability means that the controller must always be able to accurately estimate the state of the system. Controllability implies that the controller is all the time able to act upon the system. Stability is preserved when the controller manages to keep the system at or near the desired operating point. Cyber-physical attacks compromise observability, controllability and stability of a *cps. The adversary aim is to be invisible while attacks are being perpetrated. How to detect adversarial activities and mitigate their impact?
Tables II and III list, non-exhaustively, attack and detection techniques in control-theoretic literature for *cps. Control-theoretic literature represent as occasional disturbances the collateral damages resulting from attacks against a *cps. Mitigating the attack and its impact with the deployment of an appropriate resilience plan, may allow the *cps to continue offering the service under a graceful degradation mode. Mislabeling the attack and correcting it with unappropriated response would lead again to the series of incidents such as those contained in Table I, due to the application of wrong countermeasures.
The literature listed in Tables II and III also assumes powerful adversarial models. They suppose that an adversary can observe and change signals generated by sensors or signals provided to actuators. I.e., the adversary can operate with the information going through the network. The adversary may also know or learn the model of the system, e.g., using system identification tools. The adversary also can manipulate sensor measurements, to make the state change invisible, i.e., to evade detection.
Iii Futuristic Cyber-Physical Resilience
In the previous section, the meaning of resilience has been contrasted with related concepts such as dependability and fault tolerance, in the context of cybersecurity. We examined the relation between resilience and classical solutions implemented in today’s security stacks, including in-depth defense (e.g., firewalls and cryptography) and control-theoretic protection techniques. These approaches aim to prevent system breaches from happening. However, the scenarios cited in Table I are evidence that we must assume that security breaches will continue to occur in *cps. Solutions must be designed to defend the system beyond security breaches.
We argue that new security stacks must be included in tomorrow’s resilient *cps to manage the occurrence of breaches. New stacks must manage and take control over adversarial actions that will persistently occur in the future, such as the ones listed in Table I. The new stacks must be built taking on the adversary mindset, predicting its intentions and adequately mitigating the effects. For example, a *cps can be equipped with new learning capabilities that anticipate the adversary intentions and transform them into regular actions. These potential new stacks, hereinafter called the defense learning stack, will enable the evolution towards *cps that are resilient beyond breach. The management of feedback truthfulness (cf. Section I) is a typical example of the class of problems in the scope of a defense learning stack.
The defense learning approach must achieve profiling of control data to ensure that misbehaving *cps components can be detected and thwarted; as well as assessing the trustworthiness of all the exchanged information. Adversarial actions are signaled and the new defensive components start learning about the target (e.g., the source adversary). The goal is to discover the weaknesses of the adversary, to offensively mislead the adversary intentions and thwart the adversarial actions in the end. Once the new defense learning stack gets to know the adversary (i.e., the techniques used by the adversary to identify the system, discover services, disrupt the services, etc.) the defender starts offering to the adversary some assets (i.e., assets assumed as collateral damages, that need to be sacrificed to manage the breach) to gain the trust of the adversary, i.e., to make the adversary confident about the success of the perpetrated actions. In the sequel, an illustrative example is provided.
Illustrative Example. The availability of futuristic automated design technologies for resilient *cps is assumed, e.g., to synthesize controllers modelled as hybrid systems , i.e., by including continuous dynamics and discrete event components. The continuous dynamics of a *cps, shall be modelled by differential equations. Threat modeling, representing malfunctioning, faulty entry points, and powers of adversaries must be included in the design phases, via the incorporation of the security stacks mentioned in Section II (e.g., traditional cybersecurity stack solutions discussed in Section II-A and its control-theoretic evolution outlined in Section II-B), as well as the extended defense learning stack. Synthesized controllers must guarantee the correctness of observations and measurements to identify and mitigate system malfunctioning as well as intentional (human) attacks. It also provides the necessary mechanism for the defender components to conduct a profiling of adversarial actions, in order to thwart the attacks, i.e., synthesizing new *cps resilient designs in which adversaries do not lead the security game anymore.
This will radically change from current scenarios (e.g., attacking advantages with regard to resources), to the novel defense learning paradigm. The series of physical and adversarial laws provided in the *cps design, guide the rationale that controllers actions (e.g., acceleration and braking actions commanded to each train) satisfying the verification of networked-feedback observations to identify traces of system malfunctioning and intentional manipulation from networked-feedback observations. Controllers aim at guaranteeing collision avoidance. Adversaries aim at forcing collisions in a stealthy manner (i.e., taking control with minimal perturbation w.r.t. observations, to evade detection). A covert attack (cf. Table II) may be perpetrated. The adversary exploits the possibility of having a nonzero state trajectory, by conducting a covert misappropriation of the system via the adversarial entry points, and silently exciting the system to a perturbed state trajectory which, if detected, will be mitigated with malfunctioning correction, instead of an operational resilience plan to remediate the attack. The need to distinguish attacks and malfunctioning should be addressed carefully — a problem receiving very little attention in the literature.
Iv Moving Forward
In the previous section, we argued that modern *cps must change today’s adversarial paradigm, in which an increase in the resources of the adversaries always translates into higher likelihood of disruption. In this section, we survey some promising techniques that could potentially help to the dynamics of the game.
Machine Learning —
Artificial Intelligence (AI) by means of the subfields of machine learning and search provides a large set of techniques appropriate for cyber-physical resilient systems. There are three main machine learning paradigms, namely, supervised, unsupervised and reinforcement. In supervised machine learning, there are old and new data points. Old data points are labelled. A label represents a classification of data points. Comparing their similarity with old data points, supervised machine leaning assigns classes or labels to new data points. With unsupervised machine learning, the data points are unlabelled (i.e., learning is about extracting information from data). Data points are grouped together into classes according to similarity. The classes need to be labelled by a human expert. In contrast, reinforcement learning rewards or penalizes the learner following the validity of inferred classifications, i.e., there is no need for labelled data. Learning is inferred from the successes and failures.
Supervised and reinforcement machine learning is used for system identification and model fitting. Different alternative learning methods exist, based on different considerations on the type of model (e.g., rule-based, support-vector machines, deep learning models) and its properties (e.g., explainable models/decisions, efficiency). The perpetration of control-theoretic attacks discussed in SectionII-B may require performance of a system identification phase by the adversary. Kernels methods , a kind of machine learning, can be used for system identification.
Resilience plans build upon rational reactions. Their performance often requires rapid completion of search tasks. Their efficiency can be greatly improved when the search are informed, i.e., when it applies heuristics. The AI subfield ofsearch provides us with algorithms and methods for complex decision making problems. For example, systems based on Monte Carlo tree search have been proven successful in difficult games (e.g., AlphaGo and AlphaZero). Connection between Monte Carlo tree search and reinforcement learning exists in the AI literature [11, 14].
Fuzzy Decisional Systems — Fuzzy sets can be used to model imprecision and vagueness. A concept is said imprecise when several values satisfy it (e.g., the temperature is below zero). A concept is vague when it represents partial truth. For example, the fact that a temperature is near zero
can be a matter of degree and there is no value under which temperatures are near zero and over which it is completely false that the temperatures are near zero. Fuzzy systems are typically rule based systems in which concepts are represented by means of fuzzy sets. This permits that in particular situations terms are partially fulfilled and as a consequence rules are partially fired.
Fuzzy sets have been proven to be effective in modelling and control. Fuzzy control being one of the most successful application areas of fuzzy sets. In these applications, a control system is defined by a set of fuzzy rules that will be fired all at once. The set of consequents of all rules are then combined taking into account the partial fulfillment of each rule. Combination results into a fuzzy set that needs to be defuzzified to result into an actual value. When the number of variables in a system become large, construction of fuzzy sets systems need to deal with the course of dimensionality, as the number of rules are typically exponential on the number of variables. Hierarchical fuzzy systems have been developed to deal with this problem. Adaptive systems exist that modify the rules according to changes in the environment. Fuzzy rules can be learned from data and, thus, used for adversarial identification.
Fuzzy rule based systems can be efficiently deployed in real time systems. This is so because rules can be fired in parallel and inference can be also implemented in an efficient way. Fuzzy systems can also be used to model high-level decision making processes, as e.g. to reason about identification of adversarial actions, and the remediation to be taken. These decisions need to take into account high doses of uncertainty.
The Quantum Advantage — Quantum search techniques are data size independent. Quantum machine learning, i.e., the use of quantum computing for machine learning has potential because time complexity of classification is independent of the number of data points. Schuld and Killoran investigated the use of kernel methods , that can be used for system identification, for quantum machine learning. Encoding of classical data into a quantum format is involved. A similar approach has been proposed by Havlíček et al. . Schuld and Petruccione  discuss in details application of quantum machine learning classical data generation and quantum data processing. A translation procedure is required to map the classical data, i.e., the data points, to quantum data, enabling quantum data processing, i.e., quantum classification. However, there is a cost associated with translating classical data into the quantum form, which is comparable to the cost of classical machine learning classification. This is right now the main barrier. The approach resulting in real gains is quantum data generation and quantum data processing, there is no need to translate from classical to quantum data. Quantum data generation requires quantum sensing. Successful implementation of this approach will give the quantum advantage, to the adversary or *cps?
A *cps is a physical process observed and controlled through a computer network. Signals to actuators and feedback from sensors are exchanged with a controller using a network. The advantages of such an architecture are flexibility and relatively low deployment cost. A *cps will always be prone to failures and malicious attacks. The networking aspect of *cps opens the door to cyber-physical attacks. Analysis of past incidents highlights the advanced knowledge degree of the adversaries perpetrating the attacks. Adversaries are smart and they can learn. Their sophistication is such that they can fool the controllers forging false feedback. Hence, a fundamental *cps security problem is the feedback truthfulness.
The first burning question is the need to distinguish an unintentional failure from a malicious attack. The signs resulting for these undesirable situations may be the same, but the reactions should be different. A fault can be repaired. Against, an attack a *cps has to defend itself. Acknowledging that the operation of a *cps may be disrupted by a malicious attack, the second burning question is building a *cps with resilience. That is, it must be able to recognize the presence of an attack, recover and maintain operation. Several stories of attacks and disruption told in the media (see Table I) are evidence of the relevance of the problem and the increasing risks of major catastrophes in sectors such as industry, manufacturing, transport or power generation. Currently, *cps are in principle secure by design, in the sense that they implement state of the art cryptography and protection techniques. In the future, they need to be resilient by construction. We introduced the defense learning paradigm where knowledge is built about adversaries, their techniques are identified, weaknesses are discovered, actions are anticipated and transformed into regular actions.
We have presented our vision on how next generation resilient *cps will be. The same way that nobody can think about current *cps without perfect safety to argue resilience; we have claimed that in some years, nobody would think about a *cps without perfect cyber-physical protection, in which the adversarial paradigm would have to change and make sure that an increase in the adversarial resources does not translate into higher likelihood of *cps disruption. We have analyzed if this improvement would come from the evolution promoted by artificial intelligence (AI) and machine learning communities. We believe AI search, fuzzy decision and quantum machine learning will play roles in the design of *cps resilience.
The essence of the war between adversaries and defenders is knowledge.
On the one hand, supervised and reinforcement learning can be
used by an adversary for the purpose of system identification,
an enabler for covert attacks.
On the hand, the design of resilience plans can leverage AI heuristic
search to speedup decision taking during the execution of a resilience plan.
The adaptive control that resilience requires may be obtained
using the fuzzy decisional approach. Quantum techniques can eventually
perform searches with time complexity that is data size independent.
Acknowledgments— Support from the European Commission, under grant agreement 830892 (H2020 SPARTA project), and the Cyber CNI Chair of the Institut Mines-Télécom, supported by the Center of excellence in Cybersecurity, Airbus Defence and Space, Amossys, EDF, Nokia, BNP Paribas and the Regional Council of Brittany.
-  Yuan Chen, Soummya Kar, and Jose M. F. Moura. Dynamic Attack Detection in Cyber-Physical Systems with Side Initial State Information. IEEE Transactions on Automatic Control, PP(99):1–1, 2016.
-  Vojtěch Havlíček, Antonio D. Córcoles, Kristan Temme, Aram W. Harrow, Abhinav Kandala, Jerry M. Chow, and Jay M. Gambetta. Supervised learning with quantum-enhanced feature spaces. Nature, 567(7747):209–212, 2019.
-  Kyoung-Dae Kim and Panganamala R Kumar. Cyber–physical systems: A perspective at the centennial. Proceedings of the IEEE, 100(Special Centennial Issue):1287–1308, 2012.
-  Jean-Claude Laprie. From dependability to resilience. In 38th Annual IEEE/IFIP International Conference on Dependable Systems and Networks(DSN), pages G8–G9, 2008.
-  John F. Meyer. Defining and evaluating resilience: A performability perspective. In International Workshop on Performability Modeling of Computer and Communication Systems(PMCCS), pages 1–3, 2009.
-  Yilin Mo, Sean Weerakkody, and Bruno Sinopoli. Physical Authentication of Control Systems: Designing Watermarked Control Inputs to Detect Counterfeit Sensor Outputs. IEEE Control Systems, 35(1):93–109, February 2015.
-  Fabio Pasqualetti, Florian Dorfler, and Francesco Bullo. Control-Theoretic Methods for Cyberphysical Security: Geometric Principles for Optimal Cross-Layer Resilient Control Systems. IEEE Control Systems, 35(1):110–127, Feb 2015.
-  Christian Schellenberger and Ping Zhang. Detection of covert attacks on cyber-physical systems by extending the system dynamics with an auxiliary system. In 56th IEEE Annual Conference on Decision and Control (CDC), pages 1374–1379, December 2017.
-  Maria Schuld and Francesco Petruccione. Supervised Learning with Quantum Computers. Quantum science and technology. Springer, 2018.
-  John Shawe-Taylor and Nello Cristianini. Kernel Methods for Pattern Analysis. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
-  David Silver. Reinforcement learning and simulation-based search in computer Go. PhD thesis, University of Alberta, Edmonton, Alta., Canada, 2009.
-  Roy Smith. Covert Misappropriation of Networked Control Systems: Presenting a Feedback Structure. IEEE Control Systems, 35(1):82–92, Feb 2015.
-  André Teixeira, Iman Shames, Henrik Sandberg, and Karl Henrik Johansson. A secure control framework for resource-limited adversaries. Automatica, 51:135–148, 2015.
-  Tom Vodopivec, Spyridon Samothrakis, and Branko Ster. On monte carlo tree search and reinforcement learning. Journal of Artificial Intelligence Research, 60:881–936, 2017.