A time-varying complex network with fractional dynamics has node activities over time influenced by its own history and also by activities of the other nodes. The systems with such attributes enables the modeling of coupled non-stationary processes that exhibit long-term memory [1, 2, 3, 4, 5]. A multitude of examples can be found in several application domains such as biology, and engineering, e.g. bacteria dynamics [6, 7, 8] and swarm robotics [9, 10], respectively. Nonetheless, their applicability becomes even more useful in the context of heterogeneous networks that interact geographically and across different time-scales, as it often occurs in the context of cyber-physical systems (CPS).
In the context of the present manuscript, we are motivated by the recent success of fractional order dynamics in modeling spatiotemporal properties of physiological signals, such as electrocardiogram (ECG), electromyogram (EMG), electroencephalogram (EEG) and blood oxygenation level dependent (BOLD) just to mention a few [11, 12]. Despite these modeling capabilities, there is one main limitation that continues to elude scientists and engineers alike. Specifically, complex networks such as the brain, whose nodes will dynamically evolve using fractal order dynamics, are often observed locally. Meaning that some of the dynamics assessed by the models are not only due to the local interaction, but might be constrained by unknown sources, i.e., stimuli that are external to the considered network. Consequently, we propose a model that enables us to account for the existence of such unknown stimuli, and determine the model that best captures the local dynamics under such stimuli. Observe that this enhances the analysis of these systems once we have an additional feature (i.e., the stimuli) that can be the main driver of a possible abnormal behavior of the network .
In the context of neuroscience and medical applications, the ability to determine unknown inputs is crucial in the retrieval of the model under the so-called input artifacts  captured in the EEG readings due to a pulse that causes a higher variation of the potential than the baseline. Alternatively, the existence of a model to cope with the presence of unknown inputs enable the modeling of stimuli that are originated in the subcortical structures of the brain that are often neglected in the current EEG and functional magnetic resonance imaging (fMRI) that leverages the BOLD signals . Thus, it is imperative to develop such framework, as well as tools that enable us to perform the analysis and design of the systems modeled by such setup, which we introduce in the current paper.
To the best of the authors’ knowledge, such framework has not been advanced in the context of discrete-time fractional order dynamics. In the domain of continuous-time fractional dynamics, works like [16, 17] exist for the design of observer in the presence of unknown inputs. The closest work for the discrete-time case is , which does not consider the case of unknown inputs. Nonetheless, the usefulness of accounting for unknown inputs in the context of linear time invariant (LTI) systems is an old topic [19, 20, 21, 22, 23, 24]. Specifically, the closest papers to the results proposed in this paper are as follows: observer with unknown inputs , delayed systems with unknown inputs , estimation of unknown inputs with sparsity constraints . Notwithstanding, LTI systems are not good approximations for fractional order dynamical systems due to their limited memory capabilities. Yet, due to the numerical approximation of the highly nonlinear infinite dimension fractional order system used, we are able to obtain finite dimension closed-form description that will enable us to derive results alike those attained in the context of LTI systems.
The main contributions of the present paper are threefold: (i) we present an alternating scheme that enables to determine the best estimate of the model parameters and unknown stimuli; and (ii) we provide necessary and sufficient conditions to ensure that it is possible to retrieve the state and unknown stimuli; and (iii) upon these conditions we determine a small subset of variables that need to be measured to ensure that both state and input can be recovered, while establishing sub-optimality guarantees with respect to the smallest possible subset.
The remaining of the paper is organized as follows. Section II introduces the model considered in this paper and the main problems studied in this manuscript. Next, in Section III and IV, we present the solution to these problems. Finally, in Section V, we present an illustrative example of the main results using real EEG data from a wearable technology.
Ii Problem Formulation
In this section, we first describe the time-varying complex network model having fractional order dynamical growth under unknown excitations. Next, upon this model, we propose two main problems regarding analysis and design to be addressed in the present paper.
Ii-a System Model
We consider a linear discrete time fractional-order dynamical model described as follows:
where is the state, is the unknown input and
is the output vector. Also, we can describe the system by its matrices tupleof appropriate dimensions. In what follows, we assume that the input size is always strictly less than the size of state vector, i.e., . The difference between a classic linear time-invariant and the above model is the inclusion of fractional-order derivative whose expansion and discretization  for any th state can be written as
where is the fractional order corresponding to the th state and with denoting the gamma function.
Having defined the system model, the system identification i.e. estimation of model parameters from the given data is an important step. It becomes nontrivial when we have unknown inputs since one has to be able to differentiate which part of the evolution of the complex network is due to its intrinsic dynamics and what is due to the unknown inputs. Subsequently, one of the first problems we need to address is that of system identification from the data, as described next.
Ii-B Data-driven Model Estimation
The fractional-order dynamical model takes care of long-range memory which often is the property of many physiological signals. The estimation of the spatiotemporal parameters when there are no inputs to the system was addressed in . But as it happens, ignoring the inputs inherently assume that the system is isolated from the external surrounding. Hence, for a model to be able to capture the system dynamics, the inclusion of unknown inputs is necessary. Therefore, the first problem that we consider is as follows.
Problem-1: Given the input coupling matrix , and measurements of all states across a time horizon of length , we aim to estimate the model parameters and the unknown inputs .
Ii-C Sensor Selection
For the system model described by (1), where the system parameters were determined as part of the solution to the Problem-1, we consider that the output measurements are collected only by a subset of sensors. In numerous applications (for example physiological systems) it happens that the sensors are dedicated, i.e., each sensor capture an individual state , so the measurement model can be written as
where is the matrix constructed by selecting rows indexed by set S of the identity matrix . As an example, if all sensors are selected, i.e., , then . For selecting the best set of sensors , with knowledge of the system matrices and the given observations, we would resort to the constraint of perfect observability that is defined as follows.
Definition 1 (Perfect Observability).
A system described by (1) is called perfectly observable if given the system matrices and observations , it is possible to recover the initial state and the unknown inputs .
Subsequently, the second problem that we consider is as follows. Problem-2: Determine the minimum number of sensors such that the system whose dynamics is captured by is perfectly observable from the collection of measurements collected by a sub-collection of dedicated outputs, i.e.,
Iii Model Estimation
We consider the problem of estimating , and inputs from the given limited observations , , which due to the dedicated nature of sensing mechanism is same as and under the assumption that the input matrix is known. The realization of can be application dependent and is computed separately using experimental data. For the simplicity of notation, let us denote with chosen appropriately. The pre-factors in the summation in (2) grows as and, therefore, for the purpose of computational ease we have limited the summation in (2) to the first values, where is sufficiently large. Therefore, can be written as
with the assumption that for . Using the above introduced notations and the model definition in (1), the given observations are written as
is assumed to be Gaussian noise independent across space and time. For simplicity, we have assumed that each noise component has same variance, i.e.,. Also, let us denote the system matrices as and . The vertical concatenated states and inputs during an arbitrary window of time as , respectively, and for any th state we have . For the sake of brevity, we would be dropping the time horizon subscript from the above matrices as it is clear from the context.
Since the problem of joint estimation of the different parameters is highly nonlinear, we proceed as follows: (i) we estimate the fractional order using the wavelet technique described in ; and (ii) with known, the in (5) can be computed under the additional assumption that the system matrix is known. Therefore, the problem now reduces to estimate and the inputs
. Towards this goal, we propose the following sequential optimization algorithm similar to an expectation-maximization (EM) algorithm. Briefly, the EM algorithm is used for maximum likelihood estimation (MLE) of parameters subject to hidden variables. Intuitively, in our case, in Algorithm 1, we estimate in the presence of hidden variables or unknown unknowns . Therefore, the ‘E-step’ is performed to average out the effects of unknown unknowns and obtain an estimate of , where due to the diversity of solutions, we control the sparsity of the inputs (using the parameter ). Subsequently, the ‘M-step’ can then accomplish MLE estimation to obtain an estimate of . The solution provided in  can be related to the proposed technique as follows.
It is worthwhile to mention the following result regarding EM algorithm.
Theorem 1 ().
The incomplete data (without unknown unknowns) likelihood is non-decreasing after an EM iteration.
Hence, the proposed algorithm being EM (detailed formulation in the Appendix A) has non-decreasing likelihood. Additionally, we have the following result about the incomplete data likelihood.
The incomplete data likelihood is bounded at each iteration .
We can comment about the convergence of the Algorithm 1 as follows.
The Algorithm 1 is convergent in the likelihood sense.
It should be noted that convergence in likelihood does not always guarantee convergence of the parameters. But as emphasized in , from numerical viewpoint the convergence of parameters is not as important as convergence of the likelihood. Also the EM algorithm can converge to saddle points as exemplified in .
Iv Sensor Selection
Before defining the problem of sensor selection, we review the properties of fractional-order growth system with closed-form expressions for state vectors. Using the expansion of fractional order derivative in (2), we can write the state evolving equation as
where . Alternatively, (7) can be written as
where and for . With this definition of , we can define the matrices as follows :
Thus, we can obtain the following result.
Iv-a System Observability
To achieve perfect observability, i.e., to retrieve the initial state and the unknown inputs, we need system matrices and observations. While any observation matrix is sufficient for defining the perfect observability, if we set as introduced in Section II, then and are intertwined. Simply speaking, by increasing we will have more measurements acquired across time which could be used to compensate the number of measurements at each instance of time ruled by the set of sensors used.
Given the observations from to , we can represent the initial state and the unknown inputs using (10) by defining the following matrices
where and are the observation and input matrices from (1) respectively, and is as defined in (9). Having and defined and using (1), we can write the initial state and inputs in terms of the observations as
where and .
The system described by (1) is perfectly observable after measurements if and only if .
The proof follows from re-writing equation (13) as
and, therefore, and first inputs from can be recovered if and only if . ∎
Proposition 2 will be key to formulate the constraints in the sensor selection problem as detailed in the next section.
Iv-B Sensor Selection Problem
Given the system matrices and first observations, the problem of sensor selection is defined as selecting the minimum number of sensors such that the system is perfectly observable. It can be mathematically written as
where denotes the rank of matrix when and are constructed from (11) and (12) with . An analogous problem of sensor selection with no inputs is studied in  and it was shown to be NP-hard; hence, (15) is at least as computationally challenging since it contains the former as a special case when .
Consequently, we propose a sub-optimal solution to the above problem, while providing optimality guarantees within constant factor. For the discrete set , a function is called submodular if for all sets
Also, the marginal of an element w.r.t. set is defined as . Alternatively, a set function is referred as submodular if and only if it satisfies the diminishing returns property, i.e., for all and . The monotone submodular functions have a remarkable property of performance through greedy selection within constant factor of the optimality .
With the motivation to borrow such attributes, let us define a discrete set function as, .
The discrete set function is submodular in .
Since is submodular, we will be using a greedy selection of sensors to maximize the rank of ; in other words, greedily select sensors such that . The sensor selection algorithm is described as Algorithm 2.
The complexity of Algorithm 2 with a total of sensors and length time horizon is i.e polynomial.
With the computation of would require at most . The algorithm being forward greedy selection has at most executions and hence the complexity of the algorithm is at most . ∎
We demonstrate the application of the results derived in the previous sections on physiological signals. In particular we have taken a -channel EEG signal which records the brain activity of 109 subjects. The -channel/electrode distribution with the corresponding labels and numbers are shown in Figure 1. The subjects were asked to perform motor and imagery tasks. The data was collected by BCI system with sampling rate of 160Hz [35, 36].
V-a Model parameters estimation
The parameters of the system model and , are estimated by the application of Algorithm 1. The performance of EM algorithm like any iterative algorithm is crucially dependent on its initial conditions. For the considered example of EEG dataset, it was observed that convergence of the algorithm is fast. Further, even a single iteration was sufficient to reach the point of local maxima of the likelihood. This shows that the choice of the initial point for EM algorithm is considerably good. The input coupling matrix can be easily determined through experiments. The values predicted by the model in comparison with actual data are shown in Figure 2. The one step prediction follows very closely the actual data, but there is small mismatch in the five step prediction. The ratio of square root of mean squared error of the prediction by model with and without inputs  is shown in Figure 3 for total of subjects. As observed, the error ratio is less than one-third in the case when unknown inputs is considered. Therefore, the fractional-order dynamical model with unknown inputs fits the EEG data much better than the case of no inputs. In the next part, we will use these estimated parameters to compute the set of sensors for perfect observability.
V-B Sensor selection
The estimated parameters are used to construct and as written in (11) and (12) for the greedy sensor selection Algorithm 2. Upon the application of Algorithm 2, we found that roughly half of the sensors out of are sufficient enough to retrieve the initial state and unknown inputs uniquely. The selected sensors for a particular subject are marked in the Figure 1. Due to the large size and sparsity of
matrix, some values of the initial state may blow up due to the presence of very small eigenvalues. In such cases, we can first remove the unknown inputs from (13) by multiplying both sides by , i.e., projecting signals into the orthogonal space of . We can then enforce the norm constraint of into the least squares estimation of
or, in other words, use RIDGE regression, i.e.,
Upon the knowledge of the initial state, the unknown inputs are recovered in the similar fashion or by enforcing sparsity constraint. Figure 4 shows a comparison between the actual and retrieved initial states when using the sensor set as output of the Algorithm 2. The retrieved initial states are close to the actual values provided they are estimated in the presence of numerical errors and sparsity.
The presented experimental results show how the proposed schemes are useful in mapping the complex dynamics of brain in the presence of unknown stimuli. The same framework can be easily applied to the study of other complex time evolving networks such as the physiological dynamics systems, for example BOLD signals, EMG, ECG etc.
In this paper, we introduced the framework of discrete fractional order dynamical systems under unknown inputs. Also, we provided tools to perform analysis and design of such systems. Specifically, for the analysis, we presented an alternating scheme that enables to determine the best estimate of the model parameters and unknown stimuli. Also, we provided necessary and sufficient conditions to ensure that it is possible to retrieve the state and unknown stimuli. Furthermore, we enable the design of sensing capabilities of such systems, and provided a mechanism to determine a small subset of variables that need to be measured to ensure that both state and input can be recovered, while establishing sub-optimality guarantees with respect to the smallest possible subset.
Future research will focus on exploiting the structure of fractional order dynamical systems in the context of multi-case scenarios under quantitative description of the estimation quality of the states and inputs. Such extension will enable to determine the confidence in the model obtained that would permit formal claims about the mechanism underlying in the process under study. Additionally, some of the algorithms need to be improved to be deployed in real-time applications when energy and computational resources are limited.
The authors are thankful to the reviewers for their valuable feedback. G.G. and P.B. gratefully acknowledge the support by the U.S. Army Defense Advanced Research Projects Agency (DARPA) under grant no. W911NF-17-1-0076 and DARPA Young Faculty Award under grant no. N66001-17-1- 4044.
Appendix A EM Formulation
We present the detailed construction of EM like algorithm in this section. In our formulation, the observed (incomplete) data is and while is the hidden data, therefore the complete data would be . Let us consider, and at the th iteration we denote
We can enforce Laplacian prior for for sparsity (any other prior could also be used) such that . Therefore, is then derived as
We have approximated the conditional distribution as . In the final step of expectation, we can write
where is used to signify the likelihood of the complete data, and constants are ignored. For the Maximization step, we can simply write
or in other words,
where any th component of is .
Appendix B Proof of Proposition 1
We show that the likelihood for incomplete (observed) data is bounded at each EM update step. Let us denote the likelihood of the observed data in relation to the parameter as
which is further written as
where is the normality constant. Therefore is bounded for every iteration index . ∎
Appendix C Proof of Theorem 2
For a given matrix , let denote the span of rows of indexed by set . Let be the rank of matrix composed by rows indexed by set , therefore . For given , and , we can write
where the last equality is written using the fact that dimension of intersection of and orthogonal space of are number of linearly independent rows in which are not in i.e. . ∎
-  F. Moon, Chaotic and Fractal Dynamics: An Introduction for Applied Scientists and Engineers. Wiley, 1992.
-  B. N. Lundstrom, M. H. Higgs, W. J. Spain, and A. L. Fairhall, “Fractional differentiation by neocortical pyramidal neurons,” Nature Neuroscience, vol. 11, no. 11, pp. 1335–1342, Nov 2008.
-  G. Werner, “Fractals in the nervous system: Conceptual implications for theoretical neuroscience,” Frontiers in Physiology, vol. 1, p. 15, 2010.
-  R. G. Turcott and M. C. Teich, “Fractal character of the electrocardiogram: Distinguishing heart-failure and normal patients,” Annals of Biomedical Engineering, vol. 24, no. 2, pp. 269–293, 1996.
-  S. Thurner, C. Windischberger, E. Moser, P. Walla, and M. Barth, “Scaling laws and persistence in human brain activity,” Physica A: Statistical Mechanics and its Applications, vol. 326, no. 3-4, pp. 511–521, 2003.
-  A. A. M. Arafa, “Fractional differential equations in description of bacterial growth,” Differential Equations and Dynamical Systems, vol. 21, no. 3, pp. 205–214, Jul 2013.
-  F. A. Rihan, D. Baleanu, S. Lakshmanan, and R. Rakkiyappan, “On fractional sirc model with salmonella bacterial infection,” in Abstract and Applied Analysis, vol. 2014. Hindawi Publishing Corporation, 2014.
-  H. Koorehdavoudi, P. Bogdan, G. Wei, R. Marculescu, J. Zhuang, R. W. Carlsen, and M. Sitti, “Multi-fractal characterization of bacterial swimming dynamics: a case study on real and simulated serratia marcescens,” Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, vol. 473, no. 2203, 2017.
-  M. S. Couceiro, R. P. Rocha, N. M. F. Ferreira, and J. A. T. Machado, “Introducing the fractional-order darwinian pso,” Signal, Image and Video Processing, vol. 6, no. 3, pp. 343–350, Sep 2012.
M. Couceiro and S. Sivasundaram, “Novel fractional order particle swarm optimization,”Applied Mathematics and Computation, vol. 283, pp. 36 – 54, 2016.
-  R. L. Magin, “Fractional calculus in bioengineering,” in 2012 13th International Carpathian Control Conference, ICCC 2012, 2012.
-  D. Baleanu, J. A. T. Machado, and A. C. Luo, Fractional dynamics and control. Springer Science & Business Media, 2011.
-  A. D. Norden and H. Blumenfeld, “The role of subcortical structures in human epilepsy,” Epilepsy & Behavior, vol. 3, no. 3, pp. 219–231, 2002.
A. Delorme, T. Sejnowski, and S. Makeig, “Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis,”Neuroimage, vol. 34, no. 4, pp. 1443–1449, 2007.
-  A. S. Shah, S. L. Bressler, K. H. Knuth, M. Ding, A. D. Mehta, I. Ulbert, and C. E. Schroeder, “Neural dynamics and the fundamental mechanisms of event-related brain potentials,” Cerebral cortex, vol. 14, no. 5, pp. 476–483, 2004.
-  S. Kong, M. Saif, and B. Liu, “Observer design for a class of nonlinear fractional-order systems with unknown input,” Journal of the Franklin Institute, vol. 354, no. 13, pp. 5503 – 5518, 2017.
-  I. N’Doye, M. Darouach, H. Voos, and M. Zasadzinsk, “Design of unknown input fractional-order observers for fractional-order systems,” International Journal of Applied Mathematics and Computer Science, vol. 23, no. 3, pp. 491–500, 2013.
-  Y. Xue, S. Rodriguez, and P. Bogdan, “A spatio-temporal fractal model for a CPS approach to brain-machine-body interfaces,” in DATE, 2016.
-  B. A. Charandabi and H. J. Marquez, “A novel approach to unknown input filter design for discrete-time linear systems,” Automatica, vol. 50, no. 11, pp. 2835–2839, 2014.
-  S. Sundaram and C. N. Hadjicostis, “Optimal state estimators for linear systems with unknown inputs,” in Decision and Control, 2006 45th IEEE Conference on. IEEE, 2006, pp. 4763–4768.
-  J. N. Yang and H. Huang, “Sequential non-linear least-square estimation for damage identification of structures with unknown inputs and unknown outputs,” International Journal of Non-linear mechanics, vol. 42, no. 5, pp. 789–801, 2007.
-  Y. Park and J. L. Stein, “Closed-loop, state and input observer for systems with unknown inputs,” International Journal of Control, vol. 48, no. 3, pp. 1121–1136, 1988.
C.-S. Hsieh, “On the optimality of two-stage kalman filtering for systems with unknown inputs,”Asian Journal of Control, vol. 12, no. 4, pp. 510–523, 2010.
-  S. Sundaram and C. N. Hadjicostis, “Delayed observers for linear systems with unknown inputs,” Automatic Control, IEEE Transactions on, vol. 52, no. 2, pp. 334–339, 2007.
-  S. Sefati, N. J. Cowan, and R. Vidal, “Linear systems with sparse inputs: Observability and input recovery,” in American Control Conference (ACC), 2015. IEEE, 2015, pp. 5251–5257.
-  A. Dzielinski and D. Sierociuk, “Adaptive feedback control of fractional order discrete state-space systems,” in CIMCA-IAWTIC, 2005.
-  Y. Xue, S. Pequito, J. R. Coelho, P. Bogdan, and G. J. Pappas, “Minimum number of sensors to ensure observability of physiological systems: a case study,” in Allerton, 2016.
-  P. Flandrin, “Wavelet analysis and synthesis of fractional brownian motion,” IEEE Transactions on Information Theory, vol. 38, no. 2, pp. 910–917, March 1992.
-  G. McLachlan and T. Krishnan, The EM Algorithm and Extensions. John Wiley & Sons, New York, 1996.
-  A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” Journal of the Royal Statistical Society. Series B (Methodological), vol. 39, no. 1, pp. 1–38, 1977.
-  C. F. J. Wu, “On the convergence properties of the EM algorithm,” Ann. Statist., vol. 11, no. 1, pp. 95–103, Mar 1983.
-  S. Guermah, S. Djennoune, and M. Bettayeb, “Controllability and observability of linear discrete-time fractional-order systems,” Applied Mathematics and Computer Science, vol. 18, no. 2, pp. 213–222, 2008.
-  F. Bach, “Learning with submodular functions: A convex optimization perspective,” 2013, arXiv:1111.6453.
-  M. Conforti and G. Cornuejols, “Submodular set functions, matroids and the greedy algorithm: tight worst-case bounds and some generalizations of the Rado-Edmonds theorem,” Discrete Applied Math, vol. 7, no. 3, pp. 251–274, 1984.
-  G. Schalk, D. J. McFarland, T. Hinterberger, N. Birbaumer, and J. R. Wolpaw, “Bci2000: a general-purpose brain-computer interface (BCI) system,” IEEE Transactions on Biomedical Engineering, vol. 51, no. 6, pp. 1034–1043, June 2004.
-  A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley, “Physiobank, physiotoolkit, and physionet,” Circulation, vol. 101, no. 23, pp. e215–e220, 2000.
-  K. P. Murphy, Machine Learning: A Probabilistic Perspective. The MIT Press, 2012.