Modern production engineering is becoming increasingly complex elmaraghy2012complexity . The physical processes underlying cutting-edge production engineering cannot be appropriately expressed with simple models aastrom2000limitations . In response, new control methods have been introduced. Unfortunately, these new, more complex control methods are more challenging to design and apply, and require control engineers klatt2009perspectives . Limited by the number of control engineers and their cost, industry has widely chosen to instead retain simple, understandable linear controllers, despite their inability to appropriately model the involved processes. This choice comes at the cost of the controlled processes needing close monitoring and human assistance whenever the system changes in an unforeseen way. Both expert-designed control methods and closely-monitored simple controllers fail to scale.
One way to bridge this gap is the application of modern machine learning techniques wuest2016machine . Extending the capabilities of well- accepted and used controllers with machine learning yields high potential in manufacturing. In this paper, we will further investigate such an extension: the use of neural networks to adapt the parameters of a Proportional-Integral-Differential (PID) controller not only before deployment but online during the ongoing control.
The PID controller is one of the most widely used controllers ang_2005 . A PID controller calculates its control output based on the current error , the error derivative and the error, integrated over time. Each measure is multiplied with a constant (, , and ) and then summed up. While this controller is simple and well understood, its advantages come at the price of limited capabilities. PID controllers perform well only on linear systems or systems that are linearized. As they are usually adjusted before deployment, they handle neither disturbances nor varying system dynamics well.
Several prior studies have shown that neural networks can improve performance when used to tune the parameters of traditional PID controllers online. Simple feed-forward networks can adjust the PID parameters in multiple settings de2017modeling ; de2018discrete . More sophisticated neural networks like strictly recurrent, diagonal zhang_2006 and quasi-diagonal zhang_2010 recurrent networks have also been investigated. However, all these specific design choices result in very specific behaviour and learning bergstra2013making and are not expected to generalize well over different control environments. To evaluate whether the use of neural networks with all possible connections is applicable to control engineering tasks in general, a broad range of different challenges is required. Previous work demonstrated the applicability of neural-network—based parameter tuning for one well-defined control task at a time, e.g. pendulums de2018discrete , two-tank systems pirabakaran2002pid or magnetic systems de2017modeling . However, no single approach has been applied successfully to multiple representative control engineering problems in the literature.
Despite the potential benefits, neural networks are not used regularly for parameter tuning in real-world control systems; one reason for this lack of uptake is the challenge of explaining the control decisions or stability implications of previously demonstrated neural models. The domain of control engineering demands explainability to ensure system safety leveson2003systems . As neural networks introduce a black box to the system yang2019enhancing , their use for PID parameter tuning is not yet widely accepted. It is thus paramount to be able to check and explain the parameter choices a neural network outputs when granted access to a PID controllers parameters. Perhaps surprisingly, we suggest that the use of networks that allow all possible connections between internal units will in fact enable designers to achieve a better performance without significantly increasing the design complexity or design interpretability when deploying a controller.
In this paper, we advance the above point by investigating the use of General Dynamic Neural Networks (GDNNs) for online parameter tuning in PID controllers. These neural PID controllers are evaluated on four different closed loop control engineering tasks. Each task represents a different, common challenge in control engineering, namely non-linear behaviour, unstable equilibrium, dead time, and chaotic behaviour. We compare the performance of our neural approach with a standard PID controller, which acts as a baseline, and with a system-appropriate model-based controller, which should provide the best possible performance. To evaluate robustness, each comparison is performed with and without significant sensor noise and with and without disturbance. All controllers are evaluated quantitatively for all scenarios, making this study unique in its comprehensiveness. We furthermore investigate the stability of the neural PID controller as it tunes the parameters on one representative scenario. This assessment provides an explanation of when the system is stable with respect to the PID parameters, making the neural network outputs understandable to control engineers. The present paper therefore addresses key attributes of safe and explainable artificial intelligence.
2 General Dynamic Neural Network for PID parameter tuning
To extend the standard PID controller without adding complexity, the neural networks inputs are restricted to signals which are already available in the closed loop. This setup defines the size of the input- and output layers. As seen in Figure 3(a), the neural network inputs are the system output and the error . The neural network outputs are the three PID parameters , , and . For the purpose of this study, all tested neural networks had one hidden layer with neurons, as this size is sufficient for the network to be a universal approximator hornik1989multilayer . A neural network of this size is also cheap and fast to compute. The network architectures differed only in additional recurrent or feedback connections. An example can be seen in Figure 3(b). In this case, the simple feed-forward network was extended by feedback connections from each neuron in the hidden layers to the inputs with a time delay of
. This delay proved to be sufficient for the control task. For all neurons, the activation function tanh was chosen, following the rationale ofkalman1992tanh
. Weights are initialized randomly with a mean of zero and a standard deviation of one.
A standard approach to train neural networks is backpropagationrumelhart_1986 . However, in the present setting, backpropagation cannot be applied naively. The neural network outputs are the PID parameters , , and
while the error to be minimized is the difference between the setpoint and the system output. Therefore, a loss function with respect to the neural network output cannot be formulated directly, as the ideal solution, i.e. the ideal PID parameters, is not available. For this reason, the neural network is trained using a numerical Levenberg-Marquard algorithmmarquardt_1963 . At each time step, the Jacobian matrix is computed and the control error with respect to each weight is evaluated. Each weight is then adapted to decrease the control error. To train the neural network, data is collected using the differential equations and an excitation signal as described in guenther_2018 . The system is excited with controller inputs of various lengths and amplitudes to provide sample system outputs. This follows the idea of system identification using a Dirac impulse aastrom1971system . We collect data samples that are split into training, validation, and test sets with a ratio of , , and , respectively.
. The double-lined arrows indicate that the associated variable could be a vector, while the single-lined arrows indicate scalar variables. An example neural network (b) shows one possible set of connections. All networks haveneurons in three layers. All hidden neurons have delayed feedback connections, denoted , to the input layer to capture temporal information.
To evaluate the performance of the neural PID controller, we use four typical control problems. Each system offers a different control challenge. Each individual system is controlled with and without noise. The noise is Gaussian white noise with a signal to noise ratio (SNR) ofdB and corresponds to noisy measurements from the sensors. The noise is added to the system output.
To further evaluate the robustness of the controller, we inflict upon each system a suitable disturbance. For each system, the disturbance is increased in size from zero until only one control approach is still able to stabilize the system. Disturbances of this magnitude are then used in the experiments. These disturbances are not included in the training data for the neural PID controller. All experiments are run over independent runs to ensure statistical relevance. The system differential equations are solved using the Dormand-Prince solver. To simulate real-world conditions of a discrete sample time, the controller output can be adapted every s. This intervention time is chosen to represent the limitations of real-world actuators, which cannot adjust their values on an arbitrarily small timescale. The solver is run iteratively for s, using the result of the former step as starting conditions. During each s intervention time window, the controller output is kept constant.
The training is done on an Intel Core i5-4570 with a GHz clock rate, MB of shared L cache, GB DDR3 RAM. Once learned, the neural networks run on a Raspberry Pi Model A+.
Two-Tank System: The first system is a nonlinear two-tank system, as seen in Figure 8(a). The controller has access to a pump, regulating the input, while the measured output is the water level in the second tank. This system is a standard benchmark system in control theory. It corresponds to various industrial processes, e.g. bio-reactors, filtration, and nuclear power plants. There exist a number of control approaches for this system, including direct control via neural networks majstorovic_2008 , adaptive output feedback Takagi_2014 , and backstepping pan_2005 , which will be used as comparison.
To evaluate the robustness of the compared control approaches, the two-tank system is disturbed continuously between s and s. As a disturbance, the controller output is set to zero, which would correspond to a drain of the water supply. The water levels in the tank are therefore independent from the control inputs for s. The voltage for the pump, , was limited to the range to simulate a pump appropriate to the tank dimensions.
Inverted Pendulum on a Cart: The second system is a nonlinear inverted pendulum on a cart. The control task is to stabilize the inverted pendulum at its unstable equilibrium by applying a force on the cart. The cart’s movement is restricted to m in either direction. This system is a widespread benchmark system in control theory due to its nonlinearity and unstable equilibrium wang_2011 ; kim_2017 . Practical applications for inverted pendulums include, rocket control during initial stages of flight or keeping a walking robot in an upright position. For comparison, a linear–quadratic regulator (LQR) jaleel_2013 ; li_2012 and a double PID controller li_2012 are used.
The system is disturbed by a force of N to the pendulum at time s. This disturbance can be interpreted as a strong and unexpected wind condition during the launch of a rocket. The controller ouptut was bounded within , which corresponds to a typical actuator of that size.
System with Non-negligible Time Delay: The third system is a first order linear time invariant (LTI) system with a non-negligible time delay. Time delay is a problem in control theory that is not always considered, while designing controllers wang_2013 . Time delays can result in decreased performance and system instability if the controller is not able to cope with it. The benchmarks for this system are a PID controller tavakoli_2003 and a smith-predictor stojic_2001 .
This system is disturbed by a (dimensionless) disturbance of between s and s continuously. Such a disturbance can be thought of as a temporary blockage in a fluid transport system. The exact system specifications can be found in tavakoli_2003 . The controller output was bounded between the range of .
Chaotic Thermal Convection Loop: The fourth system is a chaotic thermal convection loop, as shown in 8(d). Its dynamics are described by the equations
with the and as appropriate constants creveling1975stability . Chaotic behavior may lead to vibrations, oscillations and failure in systems and is therefore an important aspect of control theory. As chaotic behavior is unpredictable, mathematical models are only sufficient to a certain point, hence closed loop control is a desirable approach wang_1992 . Usual control approaches for the chaotic thermal convection loop are nonlinear feedback controllers boskovic_2000 and backstepping vazquez_2006 ; vazquez_2010 .
To evaluate the robustness of the applied control approaches, the system is disturbed with a force of W continuously between s and s. This perturbation can be interpreted as a temporary change in the cooling water temperature. To simulate real actuators with a limited capacity, the controller output is limited between to simulate an appropriate heating element.
4 Results and Discussion
The results for all experiments can be found in Table LABEL:tab:ctrl_results
. For each system the mean and variance are shown for all controllers in all tested scenarios. The best control approach is highlighted in bold. For all values, a two-sample t-test was performed and a control approach is only considered to be superior for. From Table LABEL:tab:ctrl_results, it can be seen that the neural PID controller performs best in scenarios, pairs with the standard PID controller in one and performs second best in two scenarios. To compare control approaches, there are common measurements that are used in control engineering, e.g. rise time, overshoot, settling time ang_2005 . However, these measurements all address the error between the setpoint and the actual systems output with different emphases. We therefore used the root-mean-squared-error (RMSE) between the setpoint and the system output to summarize these error measures in a single number without losing information.
|Control Benchmark||RMSE on test data over 50 independent runs|
|SNR||-||20 dB||-||20 dB|
|LTI system with input delay|
|Chaotic thermal convection loop|
Two-Tank System: For the two-tank system, a backstepping controller is chosen for a comparison, as this approach takes the nonlinear behavior of the system into account and has demonstrated to be well suited for this system pan_2005 . The PID controller is parameterized with the constants and , found by the MATLAB control system toolbox. All controllers are able to control the system, while the neural PID controller exhibited the smallest error for all scenarios. However, the advantage the neural PID controller yields is relatively small, as can be seen in Table LABEL:tab:ctrl_results. As this system is the easiest to control, it can be expected that the standard PID controller and backstepping perform on a similar scale.
Inverted Pendulum: The inverted pendulum on a cart is controlled by a standard PID controller stack wang_2011 , and a LQR li_2012 for benchmarking. The PID controller, responsible for the position, has the values are and and the controller for the angle is set to and wang_2011 . All three approaches are able to initially stabilize the system. For the scenario without disturbance, the neural PID controller performs equally to the standard PID controller when there is no noise present. For the scenario with only noise, the neural PID controller is the best control approach. Furthermore, only the neural PID controller is able to stabilize the disturbed system, resulting in a substantially lower error as can be seen in Table LABEL:tab:ctrl_results.
System with Non-negligible Time Delay: For the system with non-negligible time delay, the standard PID controller is set to and . The neural PID controller is superior in all scenarios, when compared to the standard PID controller. When compared to the Smith predictor smith_1957 , the neural PID controller performs better only for the scenarios without noise. However, as the Smith predictor has knowledge about the exact time delay, it has a significant advantage over the neural PID controller.
Chaotic Thermal Convection Loop: The PID parameters yielding the lowest error for the chaotic thermal convection loop are , and —the controller is therefore a PD controller. The system is initialized outside of its inherently stable region (region of attraction) with the initial conditions . Without control, the system will therefore not converge to the desired steady state .
All controllers are capable of stabilizing the system, as can be seen in Figure 9(a). The backstepping approach has the least overshoot but takes a long time to reach the steady state. The standard PID controller is more aggressive, resulting in a higher overshoot but still a smaller error. The neural PID controller performs best as it finds a good balance between settling time and overshoot.
Out of four scenarios, the neural PID controller demonstrates superior control performance. Only in the scenario with noise but without disturbance does backstepping perform slightly better ( vs ). This can be explained by backstepping being designed using the differential equations. It therefore knows the underlying systems dynamics and is less influenced by the sensor noise.
Between the time s and s, the control output is set to W to simulate the disturbance described earlier. The standard PD controller becomes meta stable and its controller output iterates between the maximum value of W and the minimum value of W. Although backstepping is proven to be globally, asymptotically stable in the Lyapunov sense mascolo1999controlling , it also becomes meta stable. This can be explained by the real world conditions. As the controller can change its control output only every s the backstepping approach fails, resulting in switching inputs between the maximum value and the minimum value, as seen in Figure 9(b). Both controllers (PID and backstepping) use excessive amounts of energy without being able to stabilize the system.
The neural PID controller is able to stabilize the system after the disturbance. Figure 9(c) shows how the neural network changes the PID parameters in response to the system output. When is far from the setpoint, the parameter has a high absolute value to force the system towards its steady state. To further increase the controller output at s, where the system reaches its furthest distance from the set point is increased. After the system reaches its steady state again, all PID parameters are adjusted back to their stationary value to ensure asymptotic performance. The neural PID controller furthermore uses significantly less energy to control the system.
Despite the experimental evidence that suggests the enormous benefit of using neural networks to adapt PID parameters online, this approach is not yet used on real-world system. This is due to the black box character of neural networks and the stringent safety requirements for control processes. One of the most important safety requirements of a closed loop control approach is input-output stability. It describes whether the system output is bounded for all bounded inputs. A system can be evaluated for stability by analysing the closed loop transfer function, i.e. the relation between the system output to its input. For the chaotic thermal convection loop, this closed loop transfer function is dependent on the systems transfer function, linearized around a steady state and the controller transfer function, characterized by the PID parameters. As the PID parameters are changed by the neural network, stability has to be evaluated for every set of parameters. The change in PID parameters furthermore renders a Lyapunov stability analysis impractical, as a well-behaved controller is not guaranteed for changing parameters.
As an important contribution, we therefore perform an analysis of the input-output stability for the controller. This analysis can be seen in Figure 10. The Figure shows the systems output and the stability, with respect to its linearized steady state over the experiment. The closed loop transfer function is not stable in the beginning, during settling and after the disturbance. This can be expected, as the system is far away from its steady state for which stability is evaluated. However, as the systems output gets close to the set point, i.e. the steady state, the closed loop transfer function becomes stable. Knowing about the relationship between chosen parameters and stability allows to include this knowledge into the training. When used as a regularization term, the parameters for the PID controller could be driven towards stable solutions. Furthermore, the input-output stability evaluation is an important insight for control engineers and makes the neural PID controllers understandable for humans, thus emphasising its applicability for safety critical systems.
In this paper, we conduct an extensive and rigorous investigation into the use of general dynamic neural networks for online PID parameter adaption. We perform experiments on four different systems, with and without sensor noise as well as with and without disturbance, resulting in experiments in total. These scenarios cover the most important challenges in control engineering. This study is therefore unique in its extensiveness. The neural-network-based approach outperforms a standard PID controller in of scenarios and outperforms a model-based controller in of scenarios. These results showcase the potential of extending existing systems by machine learning in general and neural networks in particular. To the best of our knowledge, this is the first investigation that uses general neural networks that include feedback between layers with arbitrary delays, extending the state of the art for using neural networks to tune PID parameters. We perform a detailed analysis for one representative scenario, highlighting the superior control performance of our approach over both the traditional PID controller and model-based backstepping.
Although the significant potential of neural networks for PID parameter tuning is known de2017modeling ; de2018discrete ; zhang_2006 , this technique has not been used in real-world applications to date. As the functioning of a neural network in this setting is not understood, control engineers refrain from using them. In a first attempt to solve this problem, we perform a input-output stability analysis to interpret how neural networks function within the suggested framework. Tying the neural network outputs back to stability makes this neural-network-based approach understandable to humans. It is thus an important step to increase the acceptance of machine learning based approaches for real-world systems and is an important step towards explainable and safe applied artificial intelligence.
- (1) W. ElMaraghy, H. ElMaraghy, T. Tomiyama, and L. Monostori, “Complexity in engineering design and manufacturing,” CIRP annals, vol. 61, no. 2, pp. 793–814, 2012.
- (2) K. J. Åström, “Limitations on control system performance,” European Journal of Control, vol. 6, no. 1, pp. 2–20, 2000.
- (3) K.-U. Klatt and W. Marquardt, “Perspectives for process systems engineering—personal views from academia and industry,” Computers & Chemical Engineering, vol. 33, no. 3, pp. 536–550, 2009.
- (4) T. Wuest, D. Weimer, C. Irgens, and K.-D. Thoben, “Machine learning in manufacturing: advantages, challenges, and applications,” Production & Manufacturing Research, vol. 4, no. 1, pp. 23–45, 2016.
- (5) K. H. Ang, G. Chong, and Y. Li, “PID control system analysis, design, and technology,” IEEE Transactions on Control Systems Technology, vol. 13, no. 4, pp. 559–576, 2005.
- (6) J. de Jesús Rubio, L. Zhang, E. Lughofer, P. Cruz, A. Alsaedi, and T. Hayat, “Modeling and control with neural networks for a magnetic levitation system,” Neurocomputing, vol. 227, pp. 113–121, 2017.
- (7) J. de Jesús Rubio, “Discrete time control based in neural networks for pendulums,” Applied Soft Computing, vol. 68, pp. 821–832, 2018.
- (8) M.-g. Zhang, X.-G. Wang, and W.-h. Li, “The self-tuning PID decoupling control based on the diagonal recurrent neural network,” in Proceedings of the International Conference on Machine Learning and Cybernetics, pp. 3016–3020, 2006.
- (9) K. Zhang and X. An, “Design of multivariable self-tuning PID controllers via quasi-diagonal recurrent wavelet neural network,” in Proceedings of the International Conference on Intelligent Human-Machine Systems and Cybernetics, vol. 2, pp. 95–99, 2010.
J. Bergstra, D. Yamins, and D. D. Cox, “Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures,”JMLR, 2013.
- (11) K. Pirabakaran and V. M. Becerra, “PID autotuning using neural networks and model reference adaptive control,” IFAC Proceedings Volumes, vol. 35, no. 1, pp. 451–456, 2002.
- (12) N. Leveson, M. Daouk, N. Dulac, and K. Marais, “A systems theoretic approach to safety engineering,” Dept. of Aeronautics and Astronautics, Massachusetts Inst. of Technology, Cambridge, pp. 16–17, 2003.
- (13) Z. Yang, A. Zhang, and A. Sudjianto, “Enhancing explainability of neural networks through architecture constraints,” arXiv preprint arXiv:1901.03838, 2019.
- (14) K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural networks, vol. 2, no. 5, pp. 359–366, 1989.
B. L. Kalman and S. C. Kwasny, “Why tanh: choosing a sigmoidal function,” in[Proceedings 1992] IJCNN International Joint Conference on Neural Networks, vol. 4, pp. 578–581, IEEE, 1992.
- (16) D. E. Rumelhart, G. E. Hintont, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, p. 9, 1986.
D. W. Marquardt, “An algorithm for least-squares estimation of nonlinear parameters,”Journal of the society for Industrial and Applied Mathematics, vol. 11, no. 2, pp. 431–441, 1963.
- (18) J. Günther, Machine intelligence for adaptable closed loop and open loop production engineering systems. Dissertation, Technische Universität München, München, 2018.
- (19) K. J. Åström and P. Eykhoff, “System identification—a survey,” Automatica, vol. 7, no. 2, pp. 123–162, 1971.
- (20) M. Majstorovic, I. Nikolic, J. Radovic, and G. Kvascev, “Neural network control approach for a two-tank system,” in Proceedings of the International Conference on Neural Network Applications in Electrical Engineering, pp. 215–218, IEEE, 2008.
- (21) T. Takagi, I. Mizumoto, and J. Tsunematsu, “Performance-driven adaptive output feedback control and its application to two-tank system,” in Proceedings of the International Conference on Advanced Mechatronic Systems, pp. 254–259, IEEE, 2014.
- (22) H. Pan, H. Wong, V. Kapila, and M. S. de Queiroz, “Experimental validation of a nonlinear backstepping liquid level controller for a state coupled two tank system,” Control Engineering Practice, vol. 13, no. 1, pp. 27–40, 2005.
- (23) J.-J. Wang, “Simulation studies of inverted pendulum based on PID controllers,” Simulation Modelling Practice and Theory, vol. 19, no. 1, pp. 440–449, 2011.
- (24) S. Kim and S. Kwon, “Nonlinear optimal control design for underactuated two-wheeled inverted pendulum mobile platform,” IEEE/ASME Transactions on Mechatronics, vol. 22, no. 6, pp. 2803–2808, 2017.
- (25) J. A. Jaleel and R. M. Francis, “Simulated annealing based control of an inverted pendulum system,” in Proceedings of the International Conference on Control Communication and Computing, pp. 204–209, IEEE, 2013.
- (26) W. Li, H. Ding, and K. Cheng, “An investigation on the design and performance assessment of double-PID and LQR controllers for the inverted pendulum,” in Proceedings of the International Conference on Control, pp. 190–196, 2012.
- (27) C. Wang, W. Fu, and Y. Shi, “Fractional order proportional integral differential controller design for first order plus time delay system,” in Proceedings of the International Conference on Control and Decision Conference, pp. 3259–3262, IEEE, May 2013.
- (28) S. Tavakoli and M. Tavakoli, “Optimal tuning of PID controllers for first order plus time delay models using dimensional analysis,” in Proceedings of the International Conference on Control and Automation, 2003, pp. 942–946, 2003.
- (29) M. Stojic, M. Matijevic, and L. Draganovic, “A robust smith predictor modified by internal models for integrating process with dead time,” IEEE Transactions on Automatic Control, vol. 46, no. 8, pp. 1293–1298, 2001.
- (30) H. Creveling, J. De Paz, J. Baladi, and R. Schoenhals, “Stability characteristics of a single-phase free convection loop,” Journal of Fluid Mechanics, vol. 67, no. 1, pp. 65–84, 1975.
- (31) Y. Wang, J. Singer, and H. H. Bau, “Controlling chaos in a thermal convection loop,” Journal of Fluid Mechanics, vol. 237, pp. 479–498, 1992.
- (32) D. M. Boskovic and M. Krstic, “Global stabilization of a thermal convection loop,” in Proceedings of the International Conference on Decision and Control, vol. 5, pp. 4777–4782, IEEE, 2000.
- (33) R. Vazquez and M. Krstic, “Explicit integral operator feedback for local stabilization of nonlinear thermal convection loop pdes,” Systems & Control Letters, vol. 55, no. 8, pp. 624–632, 2006.
- (34) R. Vazquez and M. Krstic, “Boundary observer for output-feedback stabilization of thermal-fluid convection loop,” IEEE Transactions on Control Systems Technology, vol. 18, no. 4, pp. 789–797, 2010.
- (35) O. J. M. Smith, “Closed control of loops with dead time,” Chemical Engineering Progress, vol. 53, no. 5, pp. 217–219, 1957.
- (36) S. Mascolo and G. Grassi, “Controlling chaotic dynamics using backstepping design with application to the lorenz system and chua’s circuit,” International Journal of Bifurcation and Chaos, vol. 9, no. 07, pp. 1425–1434, 1999.