Real-time systems, especially nowadays Cyber-Physical Systems (CPS) Baheti & Gill (2011), entail digital devices which interact closely with analog ones, humans, and the surround world. These systems are increasingly used in our daily lives including autonomous vehicles, smart grids, automated health care, and many other life-critical applications Zheng et al. (2014); Eriksen et al. (2001); Zheng & Julien (2015); Wang et al. (2017); Zheng et al. (2017a). These real-time systems need to be reliable and secure and meet strict compliance check. There have been many research efforts to improve and ensure the reliability and security of such systems. For instance, formal methods Zheng et al. (2018); Cassez et al. (2017); Zheng et al. (2015b); Bouyer et al. (2011); Zheng (2014); Zeng et al. (2018); Xu et al. (2019); Pan et al. (2017); Radhappa et al. (2018); Zheng et al. (2017c) and testing have been widely used to improve the quality in real-time systems. However, the tigher coupling between physical processes and software components in the modern real-time systems along with the varying spatial and temporal runtime contexts made such system exhibit diverse behaviours across runs Zheng et al. (2017a). Thus, verifying and validating real-time systems are still complicated where validation assures that a real-time system meets the needs of the customers and verification assesses whether a real-time system complies with the given specifciation Zheng & Julien (2015) .
In industry, simulators have been used extensively in the planning, designing, implementation, and verification stage of real-time systems for decades Bélanger et al. (2010). With the above-mentioned complexity increased dramatically for modern real-time systems, the industry has a strong tendency of relying more heavily on simulaton tools Yamane et al. (2016). Nowadays the industry mainly uses real-time simulators to design and test various aspects of real-time systems including controls and protection schemes and devices. The simulators allow the designer to conduct a variety of test earlier at the planning and design stage and in a repetitive and safe fashion. At the same time, with the rapid development of computing technologies, the cost of running simulations is steadily reduced and the performance is increasingly improved, making the simulators available to more researchers and engineers for a wider variety of real-time systems Zheng (2015). The adoption of real-time simulation allows testing real-time sytems under faulty and extreme conditions without damaging equipments under test while maintaining sufficient flexibility in choosing test parameters and components. Other benefits of using real-time simulations also include maintaining a relatively safe testing environment for the engineers and other personnel Zheng et al. (2017a); Nguyen et al. (2017).
Though real-time simulation allows a tight coupling between a real hardware with a simulation tool to test hardware or software components under realistic conditions, the executation of the simulator in this case requires each time step execution to meet the real-time constraints of the correponding phsical target modelled. The costs of developing and running such simulators can be very prohibitive for a relatively complex real-time systems (e.g., autonomous vehicles) Zheng et al. (2017a). A recent study Zheng et al. (2017a) finds out that the truthfulness of the real-time systems behavior in simulation-based approaches is often uncertain, and in practice the uncertainty and random disturbances in the real-time sytems cannot be coped with by even extensive simulation. As a result, simulation-based approach often failes to verify mission-critical real-time systems and identify key failure points. Also in the same report, the high-learning curve associated with creating the models and scalability remain major issues for simulation-based approaches.
The objective of this chaper is to provide an introduction of simulation and more relevant give some definitions of real-time simulation. Then the chapter will explain the history of real-time simulation, which is followed by explaining real-time simulation support for real-time systems. Then the chaper will walk through the challenges in establishing accurate real-time simulation and some industry best practices. Finally, this paper concludes with the discussion of some promising research directions in exploring real-time simulation for more robust and accurate real-time systems.
2 What is Simulation and Real-Time Simulation
Simulation uses the operation of one system to represent the operation of another. Since digital computers are not able to record a continuous snapshot of a transient phenomena, but rather recording a sequence of snapshots at discrete intervals, as a result, the simulation discussed in the paper is also known as digial or numerical simulation with discrete time step. In such simulation, time usually moves forward in steps which are either of equal duration Dommel (1969) or variable duration Sanchez-Gasca et al. (1995). At each time step, some form of mathematical functions or equations (e.g., differential equations) are solved. While most of the linear systems can be simulated using fixed time step, variable time step simulation is more suitable for non-linear and high frequency dynamic systems to give the capabilities to study both fast and slow phenomena reflecting the observed system behavior Sanchez-Gasca et al. (1995). For instance, in power grid systems, analysis of such systems must cater for the voltage instability and unpredictable disturbances which can span hundreds or thousands of seconds. Using fixed time step simulation with small steps to solve equations and integrations in such system is not efficient. Conversely, using fixed time step simulation with large time steps fails to capture the possible fast transients associated with above-mentioned instability and disturbance. As a result, a simulation is required to adjust automatically the time step Sanchez-Gasca et al. (1995).
In discrete time simulation, the amount of real time required to solve the underlying mathematical equations, which represents the system at a specific time step, is known as the execution time . However, the specified step size in most of time is either shorter or longer than . In the above-mentioned simulation scenarios, which are also known as ”offline” simulation, the difference between and is irrelevant as the main objective of such simulation is to obtain simulation results as soon as possible. can be dedendent on many factors, the main contributors are the computation power of the host machine and complexity of the mathmatical equations (e.g., the system model) required to be solved in each time step.
In comparison, real-time simulation is an ”online” version of discrete-time simulation, where time moves forward in steps of pre-defined duration Sanchez-Gasca et al. (1995). In real-time simulation, the simulation results are dependent not only on the equations/models but also on the Bélanger et al. (2010). To solve the underlying mathematical equations (e.g., differential equations) at a specific time step, the model is solved using the input of the variables or states from the preceding time-step. In each time-step, the real-time simulator is required to execute the same batch of tasks which include 1) reading inputs from the last time-step and generating outputs for the next time-step; 2) solving the equations (e.g., differential equations) specified for each time step; 3) if necessary, exchanging data with other simulation nodes; 4) waiting (for the next time-step to start). This implies that for any externally connected devices or simulation nodes, the state of the simulated system can be exchanged only once at the begining of each time-step in the real-time simulation. This means, as compared to an “offline” version, the execution time required to solve the equations for a time step must be shorter than the specified step size .
To be more precise, for a real-time simulation to be valid, the simulator must calculate the values for those variables and states within the same time duration that the physical counterpart would require. For instance, if it takes minutes to fill in a physical water tank, then the corresponding real-time simulator shall use a value of , which is within minutes, to fill in a simulated water tank. Otherwise the real-time simulation is considered erroneous as discrepancies between the real-time simulation and its physical counterpart’s responses are observed. This kind of error is commonly known as an ”overrun” Bélanger et al. (2010). In Figure 1, (a) and (b) are examples of offline simulation where the actual execution time for the required mathematical equations is either shorter (a) (Accelerated Simulation) or longer (b) (Delayed Simulation) than a given simulation time step. The required time to solve the equations is largely dependent on the underlying mathematical function and corresponding variables. In comparision, (c) depicts the real-time simulation scenario where the underlying mathematical function (e.g., ) has to be done within the given discrete time-step (e.g., ) to avoid the ”overrun”. With the given discrete time-step, not only those equations inside the given function need to be solved, but also the input variables of the function need to be processed, and the results need to be output within and outside the current simultion unit (e.g., current real-time simulation unit can interact with other real-time simulation unit or real hardware).
3 Evolution of Real-Time Simulation
The state of the art real-time simulation can find its origin in physical simulation (i.e., analogue simulation). Early work in physical simulation in 1960’s utilises amplifiers, resistors, capacitors, diodes to simulate specific components of a physical system. This analogue simulation where physical components is physically connected to ech other in a manner similar to the real system is the basis for the Transient Network Analyses (TNA) and the High Voltage Direct Current (HVDC) simulators Bélanger et al. (2010); Kuffel et al. (1995). The advantage of this kind of simulation establishes a near one-to-one correlation between physical simulators and physical system components. The advantage allows system engineers to design the simulation by mapping with the parameters/components of the target physical systems with minimum efforts spent on mapping and translation Brennan & Linebarger (1964). However, users for physical simulation often struggle to solve those issues related to the computer rather than the simulation as a technique. For instance, users for physical simulation need to scale all the equations (e.g., differential equations) and shall grasp the necessary and often advanced knowledge of the transformation solution (e.g., transforming those equations from the problem domain into the physical component of the computer).
Digital simulations use mathematical representation to simulate the target physcial system. The algorithms created for such software based simulations were described as early as in 1969 and have been used in some quite well known programs Kuffel et al. (1995). Compared with their analogue simulators which are able to operate in real time, these early digital simulators operate, however, in non real-time fashion. As implied before, real-time operation requires that an event in the physical system which takes for one second shall be simulated on the corresponding simulator within one second. Unfortunately in the early digital simulators , those computations necessary to solve the required mathematical equations might take many seconds or even minutes rather than within one second to meet the real-time simulation requirement. With the quick development of microprocess, digital signal processing (DSP), together with the improvement of software modelling techniques, digital real-time simulators started to replace its physical counterparts in 1980s Bélanger et al. (2010).
The earlier pioneers in digital real-time simulation use relatively high-speed computers to achieve real-time operation for short period of time, but these simulators are restricted to quite simple systems Marti & Linares (1994); Kezunovic et al. (1994). The main issue of these simulators lies in the scalability. When the modelled system becomes realistic and more complex, the number of equations to be solved gets more complex. Correspondingly, the number of computations needs to be performed restricts the applicability of these simulators. To achieve real-time operations for digital simulators, many digital signal processors can work in parallel to share the computational tasks necessary to solve the underlying equations in the simulation models. This DSP-based real-time simulators are developed with proprietary hardware and became commercially available Kuffel et al. (1995). Since proprietary hardware imposes non-trivial limitation on its applicability, some other digital real-time simulators based on commericial supercomputers are created (e.g., HYPERSIM from Hydro-Quebec Do (1999)). There are also other attempts to use low-cost standard PCs to host real-time simulators Hollman & Martí (2003), and this attempt has been further accelerated by the introducing of low-cost Commerical Off-The-Shelf (COTS) multi-core processors. COTS enabled digital simulators are able to conduct complex parallel simulation by reducing dependence on inter-computer communcations and these simulators are widely used to simulate large-scale microgridas, aircarft and power systems Bélanger et al. (2007).
A recent advancement in real-time simulation lies in running simulation models directly on Field-Programmable Gate Arrays (FPGAs). This relatively new trend allows fully utilizing the parallel nature of FPGAs so that the time-step for the real-time simulation can be set to very small and a complex system can be simulated by many smaller models Saad et al. (2015); Chen & Dinavahi (2009).
4 Real-Time Simulation Support for Real-Time Systems
Real-Time Systems, especially recent Cyber-Physical Systems, entail complex software and exhibit sophisticated interactions among digital devices, analog components, and the surrounding world, including humans in that world. Such systems are often safety critical and must be reliable. However, Real-Time Systems contain both digital and analog components and must be modeled as hybrid systems, which are known to be hard to formally verify Henzinger et al. (1995). The state-of-the-practice in creating a repetitive and flexible test environment is to use real-time simulation, where computer models are used to accurately produce values of internal variables inside a real-time systems; these models are designed to operate on the same time-scale as the corresponding physical system Bélanger et al. (2010). The basic assumption to use real-time simulation is to consider in a real-time system, the process which needs to be verified composes of a plant with a controller acting upon it. Thus, though real-time simulation have been used in various real-time systems, these applications can be categorized as two tests: hardware-in-the-loop tests (HiL) Chen (2010); Zhang et al. (2013) and software-in-the-loop tests (SiL) Kwon & Choi (1999).
In HiL, a physical controller is connected to an executing real-time simulation representing a virtual plant, and this is used to verify the controller. Aircraft manufacturer Embraer used real-time simulation software platform (e.g., RT-LAB) to execute a highly accurate fighter plane model, which is connected with a real onboard aircraft computer and a real cockpit. The real-time simulation model can provide a variety of feedback including force (via flight control joystick), visual and sound De Resende (2004). Industrial electronic company Mitsubishi also used real-time simulation to design motor drives, where a physical motor was simulated to work with its related real-world controllers Harakawa et al. (2005). This simulation allows testing and verification of the whole system in a much earlier stage where a phsycal motor is not yet availalble for test. Real-time simulation also helps to understand the integration of microgrid devices with renewable energy resources (e.g., solar power and wind farms), where the overall stability and transient responses from the integrated power system can be thorougly studied and various statiscal studies can be conducted for optimization and worse-case scenario analysis Paquin et al. (2007, 2008).
In comparision, in SiL Kwon & Choi (1999), both controller and plant are simulated. SiL supports the embedded software in a real-time system to be tested as early as possible where the entire real-world platform including equipment (which might not be ready yet) and environment (which can not be thoroughly tested) is modeled in software and simulated. Compared with HiL which is often used during the testing phase of a real-time system, SiL can be used at all stages in a real-time system including design, development and testing Demers et al. (2007). In Bayha et al. (2012), SiL is used to model the controllers and the enviornment of a unmanned aerial vehicles (UAV) system and test the control software using a variety of test cases for fault-tolerance and robustness. In Demers et al. (2007), SiL is used to evaluate a policy-based network management software against a variety of network simualtors. In Muresan & Pitica (2012), SiL is used to evaluate the controller software for an electric motor and it was found the cost for implementing a SiL enviornment are about sixty times cheaper than HiL environment. Similarly in Russo et al. (2007), an SiL analysis is performed to evaluate a brake controller algorithm and good results are obtained by SiL with a cost effective way.
Though SiL is cost effective to produce repeatable results (as the randomness of the controller and plant both reside in the real-time simulator) and supports some basic forms fo testing (e.g., by visualizing the internal state of some variables), advanced debugging which allows tracabilities to software errors is not available and SiL does not provide any form of automation of test cases and test oracles neither Demers et al. (2007); Bayha et al. (2012).
5 Challenges and Best Practices in Industry
Since real-time systems often entail multiple physical processes, and each physical process may be modeled separately, in Industry, different real-time simulations must be able to coordinate, even potentially exchanging state information during a single time step . Different simulation models and platforms may have different time steps, depending on their physical laws (e.g., a dynamic electrical system has a fast time step while a dynamic thermal system may have a much slower one). Each real-time simulator (i.e., the executable implementing the simulation for a given model) has to execute a number of tasks within , including reading inputs, solving model equations, generating outputs, and exchanging results with other simulation models. All these tasks are important, and failures or inaccuracies in any of them can render the real-time simulation useless Bélanger et al. (2010). Accurate synchronization among different simulation models is crucial to ensuring simulation stability Bednar & Crosbie (2007).
The Functional Mockup Interface (FMI) Blochwitz et al. (2012) is an independent standard to create a co-simulation environment where C code for a specific dynamic system model is generated in the form of an input/output block, and two or more models (with different solvers) can be coupled. FMI requires each simulation platform provider (where each dynamic model is created) to explicitly support an FMI interface for model exchange so that it is possible to automatically generate a Functional Markup Unit (FMU) from the dynamic model. A FMU is a combination of C code and a helper XML specification that has definitions for all the variables in the given dynamic model. However, the two fundamental challenges in establishing real-time simulation, namely time synchronization and data integration among simulation models are left for developers to implement in the form of Master Algorithm. The MODELISAR Modelisar (2016)
project supports FMI and includes a prototypical implementation of a Master Algorithm. However, the existing implementation does not guarantee the efficiency and simulation speed, which largely depend on the problem to be solved (e.g., the size of the underlying ordinary differential equation or differential algebraic equation) and the host computer’s powerBastian et al. (2011). This kind of implementation of the Master Algorithm is not acceptable for an integrated verification environment where efficiency and speed of the real-time simulation must be optimized to guarantee necessary precision of the outputs; further, writing a suitable master algorithm is very error-prone and poses significant challenges for developers Bastian et al. (2011). Since numerical integrations deal with approximations, it is of vital importance to have an alternative automated solution that can guarantee efficiency and speed of the real-time simulation (instead of an interface or a requirement for data integration and time synchronization) to maintain a satisfactory balance between the simulation speed (i.e., latency) and precision (i.e., simulation errors) Khaled et al. (2014). In Al-Hammouri (2012), a co-simulation platform is proposed to integrate the ns-2 network simulator with the Modelica physical systems simulator. The simulation platform is able to support asynchronous events inside both physical and network systems. The main contribution of the work is to solve real-time synchronization to make sure both simulators will advance at the same wall-clock rate.
In industry, real-time system practitioners guarantee the simulation speed and precision by using dedicated machines and software to build the real-time simulation environments (i.e., NI PXI server PXI (2016) and LabVIEW real-time module LabVIEW RealTime (2016)). However, this approach is very expensive (e.g., a basic NI PXI server costs around USD PXIPrice (2016)) and is not scalable. Also this approach does not provide a solution for complex real-time systems where sub-system models must be created in different simulation platforms for a variety of reasons (e.g., knowledge and preference of the interdisciplinary team, different costs, and different built-in solvers). In Khaled et al. (2014), a fine-grained co-simulation method is explored that enables numerical integration speed-ups. The method is to partition the existing models into loosely coupled sub-systems with sparse communication between partitioned modules. The parallel execution is mainly to exploit multi-core processors to deal with originally sequential ordinary differential equations in real-time system’s sub-system models. In Kinsy et al. (2011), a time-predictable computer architecture for digital emulation is proposed for real-time systems. The architecture can be implemented on top of a Field Programmable Gate Array (FPGA) to provide low latency emulation. In Yan et al. (2012), an integrated platform is proposed to integrate Matlab/Simulink simulation tool with the DETERLab emulation testbed. The runtime environment provides time synchronization and data communication to coordinate two simulation platforms for security experiments.
6 Future Direction
Simulation based approaches are widely used in industry-scale real time systems, however, they are restrictive both in expressiveness (e.g., of quantitative properties) and coverage (i.e., the cyber part is modeled instead of testing the real implementation). Thus common but subtle bugs that result from the interaction of cyber and physical components in complex real-time systmes (e.g., Autonomous vehicles) are often not detectable.
On the other hand, runtime verification where properties are formally specified and checked at runtime, receives a lot of attention to verify real-time systems. In Zheng et al. (2015a), runtime monitors can check both qualitative (e.g., safety, liveness) and quantitative (e.g., bounded safety and liveness, responsiveness) properties. However, to detect the insidious real-time systems bugs that are manifest only in a specific deployment environment, runtime verification techniques require repetitive deployments that are either too expensive (e.g., in labor, time, and/or money), dangerous (e.g., involving autonomous vehicles), or infeasible. As an example, an unmanned rover deployed to the moon was unable to move after the first lunar night. A post hoc analysis found that the temperature on the moon is considerably lower than the rover’s components had accounted for; as a result, the rover effectively suffered from frostbite222Chen, Stephen. “Last-ditch efforts to salvage mission of China’s stricken Jane Rabbit Lunar rover.” South China Morning Post 18 April 2014, (http://tinyurl.com/oq5qnqx). Runtime verification of real-time systems in general requires a repetitive and flexible test environment where settings can be changed easily to determine whether the properties being checked will hold in all situations. In the case of the rover, there are relatively accurate models of involved physical processes (e.g., rover dynamics and moon environment). However these models are separate from the runtime verification of the system’s cyber components, ultimately leading to the failure.
In Zheng et al. (2017b), a combined verification approach that allows real-time system developers to opportunistically leverage real-time simulation to support runtime verification. The middleware, termed BraceBind, allows selecting, at runtime, between actual physical processes or simulations of them to support a running real-time system. BraceBind is a real-time simulation architecture to generate and manage multiple real-time simulation environments based on existing simulation models in a manner that ensures sufficient accuracy for verifying a real-time system Specifically, BraceBind aims to both improve simulation speed and minimize latency, thereby making it feasible to integrate simulations of physical processes into the running real-time system. BraceBind then integrates this real-time simulation architecture with an existing runtime verification approach that has low computational overhead and high accuracy. This integration uses an aspect-oriented adapter architecture that connects the variables in the cyber portion of the real-time system with either sensors and actuators in the physical world or the automatically generated real-time simulation. Their experimental results show that, with a negligible performance penalty, BraceBind is both efficient and effective in detecting program errors that are otherwise only detectable in a physical deployment.
Another promising direction to improve real-time simulation lies in the increasingly popular machine learning and deep learning models, which have been extensively in real-time systems to detect object locationPan et al. (2018), user activites Lu et al. (2019); Bhandari et al. (2017), driver drowsiness Zhang et al. (2019), road conditions for vehicles Zhou et al. (2019); Xie et al. (2018). As real-time systems are highly complex and essentially probalistic, deep learning models can be used along with real-time simulators to improve the robustness and accuracy of simulation models. There are already some pioneering work towards this path as in Vedaldi & Lenc (2015); Sivanandam & Deepa (2006)1994) can be used effectively with real-time simulation to improve robusness and safety of real-time systems.
- Al-Hammouri (2012) Al-Hammouri, Ahmad T. 2012. A comprehensive co-simulation platform for cyber-physical systems. Computer Communications, 36(1), 8–19.
- Baheti & Gill (2011) Baheti, Radhakisan, & Gill, Helen. 2011. Cyber-physical systems. The impact of control technology, 12(1), 161–166.
- Bastian et al. (2011) Bastian, J., Clauß, C., Wolf, S., & Schneider, P. 2011. Master for co-simulation using FMI. In: 8th International Modelica Conference. Citeseer.
- Bayha et al. (2012) Bayha, Andreas, Grüneis, Franziska, & Schätz, Bernhard. 2012. Model-based software in-the-loop-test of autonomous systems. Page 30 of: Proceedings of the 2012 Symposium on Theory of Modeling and Simulation-DEVS Integrative M&S Symposium. Society for Computer Simulation International.
- Bednar & Crosbie (2007) Bednar, R., & Crosbie, R. E. 2007. Stability of multi-rate simulation algorithms. Pages 189–194 of: Proc. of SCSC.
- Bélanger et al. (2010) Bélanger, J., Venne, P., & Paquin, J. N. 2010. The What, Where, and Why of Real-Time Simulation. In: Power and Energy Society.
- Bélanger et al. (2007) Bélanger, Jean, Lapointe, Vincent, Dufour, Christian, & Schoen, Loic. 2007. eMEGAsim: An open high-performance distributed real-time power grid simulator. Architecture and specification. Pages 12–24 of: Proceedings of the International Conference on Power Systems (ICPS 2007). Citeseer.
- Bhandari et al. (2017) Bhandari, Babin, Lu, JianChao, Zheng, Xi, Rajasegarar, Sutharshan, & Karmakar, Chandan. 2017. Non-invasive sensor based automated smoking activity detection. Pages 845–848 of: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE.
- Blochwitz et al. (2012) Blochwitz, T., Otter, M., Åkesson, J., Arnold, M., Clauss, C., Elmqvist, H., et al. . 2012. Functional mockup interface 2.0: The standard for tool independent exchange of simulation models. Pages 173–184 of: 9th International Modelica Conference. The Modelica Association.
- Bouyer et al. (2011) Bouyer, Patricia, Cassez, Franck, & Laroussinie, François. 2011. Timed modal logics for real-time systems. Journal of Logic, Language and Information, 20(2), 169–203.
- Brennan & Linebarger (1964) Brennan, Robert D, & Linebarger, Robert N. 1964. A survey of digital simulation: digital analog simulator programs. Simulation, 3(6), 22–36.
- Cassez et al. (2017) Cassez, Franck, Jensen, Peter Gjøl, & Larsen, Kim Guldstrand. 2017. Refinement of trace abstraction for real-time programs. Pages 42–58 of: International Workshop on Reachability Problems. Springer.
- Chen (2010) Chen, H. X. 2010. Simulink and VC-based Hardware-in-the-loop Real-time Simulation for EV. In: Proc. of EVS-25.
- Chen & Dinavahi (2009) Chen, Yuan, & Dinavahi, Venkata. 2009. FPGA-based real-time EMTP. IEEE Transactions on Power Delivery, 24(2), 892–902.
- De Resende (2004) De Resende, OC. 2004. The evolution of the aerodynamic design tools and transport aircraft wings at Embraer. Journal of the Brazilian Society of Mechanical Sciences and Engineering, 26(4), 379–389.
- Demers et al. (2007) Demers, Stephanie, Gopalakrishnan, Praveen, & Kant, Latha. 2007. A generic solution to software-in-the-loop. Pages 1–6 of: MILCOM 2007-IEEE Military Communications Conference. IEEE.
- Do (1999) Do, Van-Que. 1999. Hypersim, an integrated real-time simulator for power networks and control systems. ICDS’99.
- Dommel (1969) Dommel, Hermann W. 1969. Digital computer solution of electromagnetic transients in single-and multiphase networks. IEEE transactions on power apparatus and systems, 388–399.
- Eriksen et al. (2001) Eriksen, Charles C, Osse, T James, Light, Russell D, Wen, Timothy, Lehman, Thomas W, Sabin, Peter L, Ballard, John W, & Chiodi, Andrew M. 2001. Seaglider: A long-range autonomous underwater vehicle for oceanographic research. IEEE Journal of oceanic Engineering, 26(4), 424–436.
- Harakawa et al. (2005) Harakawa, Masaya, Yamasaki, Hisanori, Nagano, Tetsuaki, Abourida, Simon, Dufour, Christian, & Bélanger, Jean. 2005. Real-Time Simulation of a Complete PMSM Drive at 10 s Time Step. In: at the 2005 International Power Electronics Conference-Niigata (IPEC-Niigata 2005).
- Haykin (1994) Haykin, Simon. 1994. Neural networks. Vol. 2. Prentice hall New York.
- Henzinger et al. (1995) Henzinger, T. A., Kopke, P. W., Puri, A., & Varaiya, P. 1995. What’s decidable about hybrid automata? In: Proc. of STOC.
- Hollman & Martí (2003) Hollman, Jorge Ariel, & Martí, José Ramón. 2003. Real time network simulation with PC-cluster. IEEE transactions on power systems, 18(2), 563–569.
- Kezunovic et al. (1994) Kezunovic, M, Aganagic, M, Skendzic, V, Domaszewicz, J, Bladow, JK, Hamai, DM, & McKenna, SM. 1994. Transients computation for relay testing in real-time. IEEE Transactions on Power Delivery, 9(3), 1298–1307.
- Khaled et al. (2014) Khaled, A. B., Gaid, M. B., Pernet, N., & Simon, D. 2014. Fast multi-core co-simulation of Cyber-Physical Systems: application to internal combustion engines. Simulation Modelling Practice and Theory, 47, 79–91.
- Kinsy et al. (2011) Kinsy, M., Khan, O., Celanovic, I., Majstorovic, D., Celanovic, N., & Devadas, S. 2011. Time-predictable computer architecture for cyber-physical systems: Digital emulation of power electronics systems. Pages 305–316 of: Proc. of RTSS. IEEE.
- Kuffel et al. (1995) Kuffel, R, Giesbrecht, J, Maguire, T, Wierckx, RP, & McLaren, P. 1995. RTDS-a fully digital power system simulator operating in real time. Pages 498–503 of: Proceedings 1995 International Conference on Energy Management and Power Delivery EMPD’95, vol. 2. IEEE.
- Kwon & Choi (1999) Kwon, W. H., & Choi, S.-G. 1999. Real-time distributed software-in-the-loop simulation for distributed control systems. Pages 115–119 of: Proc. of CACCD.
- LabVIEW RealTime (2016) LabVIEW RealTime. 2016. LabVIEW RealTime. http://www.ni.com/labview/realtime/. Online; accessed 26 April 2016.
- Lu et al. (2019) Lu, Jianchao, Wang, Jiaxing, Zheng, Xi, Karmakar, Chandan, & Rajasegarar, Sutharshan. 2019. Detection of Smoking Events from Confounding Activities of Daily Living. Page 39 of: Proceedings of the Australasian Computer Science Week Multiconference. ACM.
- Marti & Linares (1994) Marti, José R, & Linares, Luis R. 1994. Real-time EMTP-based transients simulation. IEEE Transactions on Power Systems, 9(3), 1309–1317.
- Modelisar (2016) Modelisar. 2016. Modelisar. http://www.modelisar.org. Online; accessed 26 April 2016.
- Muresan & Pitica (2012) Muresan, Marius, & Pitica, Dan. 2012. Software in the loop environment reliability for testing embedded code. Pages 325–328 of: 2012 IEEE 18th International Symposium for Design and Technology in Electronic Packaging (SIITME). IEEE.
- Nguyen et al. (2017) Nguyen, Van Hoa, Besanger, Yvon, Tran, Quoc Tuan, Nguyen, Tung Lam, Boudinet, Cederic, Brandl, Ron, Marten, Frank, Markou, Achilleas, Kotsampopoulos, Panos, van der Meer, Arjen A, et al. . 2017. Real-time simulation and hardware-in-the-loop approaches for integrating renewable energy sources into smart grids: Challenges & actions. arXiv preprint arXiv:1710.02306.
- Noureen et al. (2017) Noureen, S, Shamim, N, Roy, V, & Bayne, S. 2017. real-time Digital Simulators: A Comprehensive Study on System Overview, Application, and Importance. Int. J. Res. Eng, 4, 266–277.
- Pan et al. (2017) Pan, Lei, Zheng, Xi, Chen, HX, Luan, T, Bootwala, Huzefa, & Batten, Lynn. 2017. Cyber security attacks to modern vehicular systems. Journal of information security and applications, 36, 90–100.
- Pan et al. (2018) Pan, Lei, Zheng, Xi, Kolar, Philip, & Bangay, Shaun. 2018. Object localisation through clustering unreliable ultrasonic range sensors. International journal of sensor networks, 27(4), 268–280.
- Paquin et al. (2007) Paquin, Jean-Nicolas, Moyen, Julien, Dumur, Guillaume, & Lapointe, Vincent. 2007. Real-time and off-line simulation of a detailed wind farm model connected to a multi-bus network. Pages 145–152 of: 2007 IEEE Canada Electrical Power Conference. IEEE.
- Paquin et al. (2008) Paquin, Jean-Nicolas, Dufour, Christian, & Bélanger, Jean. 2008. A hardware-in-the-loop simulation platform for prototyping and testing of wind generator controllers. In: CIGRÉ Canada Conference on Power Systems Winnipeg.
- PXI (2016) PXI. 2016. What’s PXI. http://www.ni.com/pxi/whatis/. Online; accessed 26 April 2016.
- PXIPrice (2016) PXIPrice. 2016. PXI Sample Price. http://sine.ni.com/nips/cds/view/p/lang/en/nid/210825. Online; accessed 26 April 2016.
- Radhappa et al. (2018) Radhappa, Harish, Pan, Lei, Xi Zheng, James, & Wen, Sheng. 2018. Practical overview of security issues in wireless sensor network applications. International journal of computers and applications, 40(4), 202–213.
- Russo et al. (2007) Russo, Riccardo, Terzo, Mario, & Timpone, Francesco. 2007. Software-in-the-loop development and validation of a cornering brake control logic. Vehicle System Dynamics, 45(2), 149–163.
- Saad et al. (2015) Saad, Hani, Ould-Bachir, Tarek, Mahseredjian, Jean, Dufour, Christian, Dennetière, Sébastien, & Nguefeu, Samuel. 2015. Real-time simulation of MMCs using CPU and FPGA. IEEE Transactions on Power Electronics, 30(1), 259–267.
- Sanchez-Gasca et al. (1995) Sanchez-Gasca, JJ, D’aquila, R, Price, WW, & Paserba, JJ. 1995. Variable time step, implicit integration for extended-term power system dynamic simulation. Pages 183–189 of: Proceedings of Power Industry Computer Applications Conference. IEEE.
- Sivanandam & Deepa (2006) Sivanandam, SN, & Deepa, SN. 2006. Introduction to neural networks using Matlab 6.0. Tata McGraw-Hill Education.
- Vedaldi & Lenc (2015) Vedaldi, Andrea, & Lenc, Karel. 2015. Matconvnet: Convolutional neural networks for matlab. Pages 689–692 of: Proceedings of the 23rd ACM international conference on Multimedia. ACM.
- Wang et al. (2017) Wang, Sen, Li, Xue, Yao, Lina, Sheng, Quan Z, Long, Guodong, et al. . 2017. Learning multiple diagnosis codes for ICU patients with local disease correlation mining. ACM Transactions on Knowledge Discovery from Data (TKDD), 11(3), 31.
- Xie et al. (2018) Xie, Haiming, Tian, Guangyu, Du, Guangqian, Huang, Yong, Chen, Hongxu, Zheng, Xi, & Luan, Tom H. 2018. A Hybrid Method Combining Markov Prediction and Fuzzy Classification for Driving Condition Recognition. IEEE Transactions on Vehicular Technology, 67(11), 10411–10424.
- Xu et al. (2019) Xu, Guangquan, Zhang, Yao, Sangaiah, Arun Kumar, Li, Xiaohong, Castiglione, Aniello, & Zheng, Xi. 2019. CSP-E2: An abuse-free contract signing protocol with low-storage TTP for energy-efficient electronic transaction ecosystems. Information Sciences, 476, 505–515.
- Yamane et al. (2016) Yamane, Amine, Abourida, Simon, Bouzid, Yahia, & Tempez, François. 2016. Real-time simulation of distributed energy systems and microgrids. IFAC-PapersOnLine, 49(27), 183–187.
- Yan et al. (2012) Yan, Wei, Xue, Yuan, Li, Xiaowei, Weng, Jiannian, Busch, Timothy, & Sztipanovits, Janos. 2012. Integrated simulation and emulation platform for cyber-physical system security experimentation. Pages 81–88 of: Proceedings of the 1st international conference on High Confidence Networked Systems. ACM.
- Zeng et al. (2018) Zeng, Xianjiao, Xu, Guangquan, Zheng, Xi, Xiang, Yang, & Zhou, Wanlei. 2018. E-AUA: An efficient anonymous user authentication protocol for mobile IoT. IEEE Internet of Things Journal.
- Zhang et al. (2019) Zhang, Chao, Wu, Xiaopei, Zheng, Xi, & Yu, Shui. 2019. Driver Drowsiness Detection using Multi-Channel Second Order Blind Identifications. IEEE Access.
- Zhang et al. (2013) Zhang, Z., et al. . 2013. Co-simulation framework for design of time-triggered cyber physical systems. In: Proc. of ICCPS.
- Zheng et al. (2015a) Zheng, X., Julien, C., Podorozhny, R., & Cassez, F. 2015a. BraceAssertion: Runtime Verification of Cyber-Physical Systems. In: Proc. of MASS.
- Zheng (2014) Zheng, Xi. 2014. Physically informed assertions for cyber physical systems development and debugging. Pages 181–183 of: 2014 IEEE International Conference on Pervasive Computing and Communication Workshops (PERCOM WORKSHOPS). IEEE.
- Zheng (2015) Zheng, Xi. 2015. Physically informed runtime verification for cyber physical systems. Ph.D. thesis.
- Zheng & Julien (2015) Zheng, Xi, & Julien, Christine. 2015. Verification and validation in cyber physical systems: research challenges and a way forward. Pages 15–18 of: 2015 IEEE/ACM 1st International Workshop on Software Engineering for Smart Cyber-Physical Systems. IEEE.
- Zheng et al. (2014) Zheng, Xi, Julien, Christine, Kim, Miryung, & Khurshid, Sarfraz. 2014. On the state of the art in verification and validation in cyber physical systems. The University of Texas at Austin, The Center for Advanced Research in Software Engineering, Tech. Rep. TR-ARiSE-2014-001, 1485.
- Zheng et al. (2015b) Zheng, Xi, Julien, Christine, Podorozhny, Rodion, & Cassez, Franck. 2015b. Braceassertion: Runtime verification of cyber-physical systems. Pages 298–306 of: 2015 IEEE 12th International Conference on Mobile Ad Hoc and Sensor Systems. IEEE.
- Zheng et al. (2017a) Zheng, Xi, Julien, Christine, Kim, Miryung, & Khurshid, Sarfraz. 2017a. Perceptions on the state of the art in verification and validation in cyber-physical systems. IEEE Systems Journal, 11(4), 2614–2627.
- Zheng et al. (2017b) Zheng, Xi, Julien, Christine, Chen, Hongxu, Podorozhny, Rodion, & Cassez, Franck. 2017b. Real-time simulation support for runtime verification of cyber-physical systems. ACM Transactions on Embedded Computing Systems (TECS), 16(4), 106.
- Zheng et al. (2017c) Zheng, Xi, Pan, Lei, & Yilmaz, Erdem. 2017c. Security analysis of modern mission critical android mobile applications. Page 2 of: Proceedings of the Australasian Computer Science Week Multiconference. ACM.
- Zheng et al. (2018) Zheng, Xi, Julien, Christine, Podorozhny, Rodion, Cassez, Franck, & Rakotoarivelo, Thierry. 2018. Efficient and scalable runtime monitoring for cyber–physical system. IEEE Systems Journal, 12(2), 1667–1678.
- Zhou et al. (2019) Zhou, Xun, Cai, Xuelian, Bu, Yuehang, Zheng, Xi, Jin, Jiong, Luan, Tom H, & Li, Changle. 2019. When Road Information Meets Data Mining: Precision Detection for Heading and Width of Roads. IEEE Access.