Learning and Leveraging Environmental Features to Improve Robot Awareness

09/13/2021 ∙ by Tahiya Salam, et al. ∙ University of Pennsylvania 0

This paper studies how global dynamics can inform path planning and decision-making for robots. Specifically, we investigate how coherent sets, an environmental feature found in flow-like environments, informs robot awareness within these scenarios. We compute coherent sets online with techniques from machine learning, and design a framework for robot behavior that uses coherent sets. We demonstrate the effectiveness of online methods over offline methods. Notably, we apply these online methods for robot monitoring of urban environments and robot navigation through water. Environmental features such as coherent sets provide rich context to robots for smarter, more efficient behavior.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 5

page 6

page 7

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Robots operating in flow-like environments benefit from a greater understanding of the characteristics of their surroundings. Flow-like environments, such as ocean and air currents, movements of vehicles and pedestrians, or transport of blood in the heart, are often complex and hard to model. However, robots conducting environmental monitoring, autonomous driving, crowd monitoring, or surgical procedures all require knowledge of these flow-like environments in their tasks. For robots to successfully operate in these situations, they need to be able to deduce important features of these environments and incorporate these features into their planning and decision-making. In this work, we propose a strategy for robots to learn high-level features in flow-like environments and develop planning frameworks based on these features.

(a)
(b)
Fig. 1: Time-dependent double gyre flow at times (a) and (b) . The double gyre model is a prototypical model in the study of coherent sets. The black particles, not in a region of coherence, disperse quickly. The pink particles within a coherent structure follow the flow and stay within a region.

Coherent sets are a global characteristic of turbulent flows that describe groups of particles that do not disperse very much and move together robustly [12]. This means particles within coherent sets tend to stay within the sets, as in Fig. 1. Lagrangian coherent structures (LCS) are separatices that delineate dynamically distinct regions in dynamical systems [29]. LCS provide different but complementary information when compared to coherent sets [39]. Knowledge of coherent sets and LCS allows robots to compactly represent key characteristics of their environment and leverage them in application-specific ways. For example, surgical robots could understand vasculature conditions such as flow stagnation and separation during medical procedures [37]. Aerial vehicles are affected by coherent features during takeoff and landing [40]. Robots used for environmental monitoring may be interested in atmospheric and oceanic transport pathways that play a role in the movement of pollutants [34]. Marine vehicles can plan energy efficient trajectories in the ocean by leveraging coherent structures [27, 24, 36] and maintaining sensors in their desired monitoring regions [26, 41].

There are many methods for studying coherent sets through transfer operators which provide a framework for analyzing the global behaviors of dynamical systems. These methods identify which geometric structures are the strongest barriers to global mixing [11, 15, 16, 17]. There are also methods for identifying LCS relying on detailed flow field data and integration over a parametrized time interval [22].

While the mentioned approaches elucidate coherence that can be used in planning and control of robots, neither take into account the complex interdependencies within the data sets. This presents the need for online algorithms for robotic applications, especially when the time parameterization of the system is unknown in the absence of prior models or historical data. In recent years, there has been a surge of machine learning approaches relying on kernels (or feature maps) and transfer operators to estimate dynamical systems and their key characteristics

[42, 23, 7, 32, 30, 38, 31]. Kernels map the environmental data to a higher dimensional feature space with greater expressivity for learning models. To the best of our knowledge, these advances at the intersection of machine learning and dynamical systems have not been studied in a robotics context, despite the field’s growing interest in transfer operators [1, 2, 8, 13].

New advances in the detection of coherent sets leverage techniques from machine learning [32, 30]. Though these techniques highlight the expressiveness of data-driven, kernel-based methods, they still suffer from the same limitations in their assumptions about centralized, offline data, and time parameterization. Our work builds on these recent advances and develops strategies that allow robots to compute coherent sets online and contextualize them in the robots’ environment to increase awareness and facilitate decision-making.

We propose using the coherent set detection framework as a map for greater situational awareness for operating within flow-like environments that exhibit this notion of coherence. First, we provide the relevant background in dynamical systems and machine learning related to coherent sets and show how kernel methods can represent coherent sets from data. Next, we demonstrate the usefulness of computing coherent sets in an online setting with simulations and analysis of known benchmark. We apply these online methods for computing coherent sets for overhead data in an urban monitoring environment, where the coherent sets correspond to regions of interest for situational awareness. Lastly, we show how coherent sets in a fluid flow can be leveraged for energy-efficient navigation of surface vehicles.

Ii Methodology

Fig. 2: Diagram of interplay between data, transfer operators, kernel methods, and environmental features. Transfer operators represent dynamical systems, where a state is lifted to a space and provides physical properties of the system. Many systems are defined by data exhibiting complex patterns, such as two nested rings, flows in oceans, taxi trajectories, and biological behaviors. Kernel methods transform this data to an alternative space with the use of kernel functions. Data is then easier to interpret, such as by separating two nested rings or by creating a Gram matrix for use in a kernel algorithm. Transfer operators are represented through kernel methods by embedding dynamical systems into a kernel space, . Kernel algorithms extract environmental features from transfer operators, such as where humans tend to congregate in crowds, areas of gyres in oceans, or patterns of blood flow in hearts.

Ii-a Dynamical Systems and Transfer Operators

Coherent sets are global properties of a dynamical system, extracted through constructing transfer operators. In this section, we discuss the connection between dynamical systems and transfer operators.

Define a discrete dynamical system as where is a measurable map. Transfer operators allow for a systematic study of the global properties of dynamical systems and alleviates computing the map , which is often difficult to model and estimate for unknown environments exhibiting complex, nonlinear dynamics. For example, the Koopman operator is an infinite dimensional linear transfer operator that fully captures the nonlinear dynamics of the system by acting on the space of scalar-valued functions operating on the observables of the system, . In terms of the dynamical systems map , the Koopman operator is also often written as .

Transfer operators can more generally be defined for stochastic dynamical systems to infer statistical aspects of a dynamical system. Define as a stochastic process on the state space .

is a stationary and ergodic Markov decision process. A transition density function,

, is then defined by the relation for some measurable set .

Given a probability density

, an observable of the system , and a specified lag time

, we can define the following transfer operators that provides information about iterated maps. Specifically, the Perron-Frobenius and Koopman transfer operators serve as adjoints to one another. Furthermore, the eigenvalues of transfer operators are real and their eigenfunctions form an orthogonal basis with respect to the corresponding scalar product. The Perron-Frobenius,

, and Koopman, , transfer operators are defined as follows: where , while where .

Transfer operators and their decompositions are often approximated using methods acting directly on observations of the dynamical systems, i.e., data [33, 32, 9, 28].

Ii-B Kernel Methods

Instead of constructing models using data directly, kernel methods rely on transforming data into a feature representation that elucidates interesting, meaningful patterns. Here, we present the relevant background for kernel methods, which later serves as the foundation for estimating transfer operators.

Kernel functions are applied to elements of some space, , to measure the similarity between any pairs of elements. However, this similarity metric can be found by computing the inner product of features acting on elements of the input space in some high-dimensional, possibly infinite feature space in . Thus, kernels allow us to compare objects based on their features. Formally, for inputs , a feature map , and some valid inner product , a kernel is defined as

(1)

By using the relation in Eq. (1), commonly referred to as the kernel trick, operations in the high-dimensional feature space can be computed without ever explicitly computing the coordinates of and instead using a kernel function in inner product space. Remarkably, the inner product can be efficiently computed by the kernel function, alleviating the need to assess the high-dimensional coordinates, and

, while still receiving a metric for similarity between the vectors

and in the rich feature space [25].

Kernels on features spaces that are positive definite can be used to define a function on . The space of such functions , such that the evaluation of at can be represented as the inner product in feature space, is referred to as the Reproducing Kernel Hilbert Space (RKHS). An RKHS is a space of functions mapping to with two defining features:

(2)
(3)

These two properties show that in an RKHS the feature map of every point is in the feature space as in (2) and that the space exhibits the reproducing property as in (3). The reproducing property defines a functional evaluation at a point as the inner product between the function and evaluated feature map [19].

Let be the feature map associated with kernel defined on and be the feature map associated with kernel defined on . Then, define feature matrices as and , with Gram matrices and . is then the covariance operator and is the cross-covariance operator, which can be thought of as a nonlinear generalization of standard covariance and cross-covariance matrices [32]. and can be estimated as

(4)
(5)

The covariance and cross-covariance operators are one of the most important and established tools in RKHS theory. These operators are used in the computation of kernel principal components analysis (PCA), the kernel Fisher discriminant, kernel partial least squares, the kernel canonical correlation analysis (CCA), and many other learning algorithms

[20]. Some learning algorithms relying on the use of (4) and (5), can be written in terms of their corresponding Gram matrices. For Gram matrices, only the inner products between and and and need to be computed. Thus, by computing Gram matrices on a finite number of data points, we can alleviate the need to explicitly compute the possibly infinite-dimensional covariance and cross-covariance operator. The ability to construct algorithms in the inner product space using kernels, and more specifically the kernel trick, lends them to be particularly useful in variety of learning scenarios [25]. In the next section, we will lay the framework for use of transfer operator theory in the RKHS and later show how this expression in RKHS can be used as a learning technique for modeling coherent sets and planning with this awareness.

Ii-C Connecting Transfer Operators to Kernel Methods

Transfer operator approximation in the RKHS allows for the use of kernel-based methods, acting on feature maps of the data, while still capturing the properties of the transfer operators . Transfer operators can be expressed in terms of the covariance and cross-covariance operators defined on some RKHS [32]. Assume the input and output space are the same, i.e., , then the kernels and resulting Hilbert spaces are also the same. In this scenario, we assume , meaning full-state observables are simply the states themselves.

The kernel Koopman operator is defined as

(6)

and pushes forward the observables. Using the approximation of from (4) and from the transpose of (5), the empirical estimate of the Koopman operator is

(7)

This value can be calculated empirically from training data. Similarly, the kernel Perron-Frobenius operator is defined as

(8)

with the empirical estimate of the Perron-Frobenius

(9)

where .

We now have a representation of the transfer operators in terms of the covariance and cross-covariance operators that can be estimated using the kernel versions (4) and (5). Thus, we now have a framework unifying existing techniques for approximating transfer operators and their eigendecompositions in RKHS [32], as shown in Fig. 2. In our work, we can use additional techniques from statistics to extract the dominant characteristics from the transfer operator representation of the environment, specifically coherent sets.

Ii-D Kernel Canonical Correlation Analysis

Kernel CCA applied to the kernel transfer operators produces the coherent sets in the environment. The techniques outlined in this section were first presented in [30]. Traditional CCA seeks to find two sets of basis such that the correlations between projections of two multidimensional variables and onto each set of basis vectors are maximized. Through the use of kernels, kernel CCA seeks to more generally find two nonlinear mappings and , for and , such that the correlation between the mappings is maximized.

Formally, the kernel CCA problem is

(10)

The solution to this problem is the largest eigenvalue, , corresponding to the following eigenvalue problem

(11)

In practice, we estimate covariance and cross covariance operators from finite samples of the environment, so eigenfunctions and are approximated from data as and .

In general, the inverses of the covariance and cross covariance operators do not exist. Thus, the regularized inverses are used, specifically we use and . Using these regularized inverses to solve (11) gives

(12)

Comparing with Eq. (7), we see the quantity is an approximation of the kernel Koopman operator. Similarly, from Eq. (9), we see the quantity as a reweighted Perron-Frobenius operator.

By replacing the covariance and cross-covariance operators with their empirical estimates, the eigenvalue problem of the kernel canonical correlation analysis in Eq. (12) becomes

(13)

for . Instead of solving this eigenvalue problem directly, we can instead solve a surrogate eigenvalue problem

(14)

To solve for , we can compute

(15)

In practice, we will solve this eigenvalue problem to compute the coherent sets of a system. In order to compute the coherent sets, we first compute the dominant eigenfunctions, as in (14). Then, a -means clustering is applied to the dominant eigenfunctions to determine the coherent sets.

We will now show how the formulation of the eigenvalue problem as (13) is related to the detection of coherent sets, as presented in [5]. Let be the forward operator, for some that is a reference density of interest. Given as the image density obtained by mapping the indicator function on forward in time. If we normalize with respect to , we can define a new operator and its adjoint

As per the properties of adjoint, we have that . Here, is the analogue of the reweighted Perron-Frobenius operator, or the Koopman operator with the roles of and reversed, and is the analogue of the Koopman operator. The operator is used to detect coherent sets. We want to show that the operator is equivalent to the operator in Eq. (12). With this equivalence, we can compute the coherent sets using the kernel representation of transfer operators and applying kernel CCA.

Given that , we have that . If we define the RKHS approximation of as . Then, the eigenvalue problem (12) can be reformulated as

(16)
(17)

where the kernel Koopman operator in (16) is replaced with its analogue in (17).

The property of adjointness and for can be verified as follows:

This shows the eigenvalue problem for computing coherent sets and the eigenvalue problem for kernel CCA are equivalent.

Ii-E Online Computation of Coherent Sets

Robots collect data in real time and knowledge of the dynamics of the environment is limited; thus, we desire a way to detect the coherent sets online using the and solving the eigenvalue problem, estimated as Eq. (17). The operator can be represented as a function of covariance operators, and , and cross covariance operators, and . Instead of using the covariance and cross covariance operators directly, the eigenvalue problem is solved using a surrogate formulation (14). The surrogate formulation relies on Gram matrices and , where captures the correlations between the initial positions of the tracked objects and captures the correlations between the final positions of the objects. However, robots collect sensor measurements in real time. Other works assume knowledge of the fixed lag time , such that analyzing the operator with data at and computes the coherent sets [30]. Our paper analyzes how coherence can be computed in an online setting with no prior assumptions about .

For coherence at several time instances, studies have shown it is sufficient to average the operators for all the different time instances and compute the dominant eigenfunctions of the resulting operator [17, 18, 5]. Let correspond to the Koopman operator and correspond to the Perron-Frobenius operator for the dynamics from to . Then, we are interested in the time-averaged quantity

(18)

Averaging the operator by using the representations of the operators in (12) and the corresponding surrogate eigenvalue problem in (14), we are interested, at each time step , the eigenvalue decomposition of

which can be rewritten as

(19)

The inner sum is computed as the standard optimal online averaging solution with the addition of more data. Then, the eigenvalue decomposition of the full resulting operator for the surrogate problem, Eq. (19), is computed at each time step. Finally, -means clustering is applied, as before.

Iii Simulations and Analysis

We present two examples of fluid flows, where the coherent sets are well-studied. We perform an analysis of our online method compared to offline methods.

(a)
(b)
Fig. 3: (a) particles are initialized in a environment. Particles are colored by their -coordinate to visualize their movement. (b) The position of the particles are tracked under a time-dependent double gyre flow at time . Two dynamically distinct regions can be seen as a result of this type of flow, which correspond to coherent sets.

Iii-a Time-Dependent Double Gyre

A simple model of the wind-driven, time-dependent double gyre flow, as in [14], is described by

For this paper, the parameters are set to . We simulate points uniformly sampled on grid . From time with step size , we use a differential equations solver based on an explicit Runge-Kutta (4,5) formula on the velocities to calculate the trajectories of the points, as shown in Fig. 3.

With a Gaussian radial basis function kernel

for , we compute the Gramian matrices for for the initial position of the particles and for the position of the particles at time , as in Eq. (19). In an online setting, only the inner sum needs to be recomputed as a moving average. The total product and its eigenvalue decomposition is performed. A -means clustering for is computed on the dominant eigenvalues of the estimated time-averaged matrix Eq. (18).

Iii-B Bickley Jet

(a)
(b)
Fig. 4: The Bickley jet is an idealized model for fluid flows, such as in the ocean or in the atmosphere. (a) particles are initialized in a environment. (b) The movement of the particles are tracked under a Bickley jet model until , shown here at .

The Bickley jet model is a prototypical model in the study of coherence that is a meandering zonal jet, flanked both above and below by counter rotating vertices. The Bickley jet is used as an idealized model for the Gulf Stream in the ocean and polar night jets in the atmosphere [10, 6].

The stream function for the Bickley jet model is

with . The velocities can be computed as and . We use scaled parameters from [21] resulting in .

We sample points uniformly on a grid . From time with step size , we calculate the trajectories of the points using a variable-step, variable-order Adams-Bashforth-Moulton solver of orders 1 to 13 on the velocity , as shown in Fig. 4. Some particles escape the grid , and the positions of these particles wrap around such that they stay within the grid. The true time lag parameter , as introduced in Sec. II-E, for this map is known to be for this simulation.

For this example, we also use a Gaussian kernel, with . We compute the Gramian matrices for for the initial position of the particles and , as before for the time-varying Eq. (19) and perform -means clustering for with dominant eigenvalues of the operator, similar to [30].

Iii-C Analysis of Online Coherent Sets Computation

Environment
Method
Evaluation Metric
RI H V
Time-Dependent
Double Gyre
Offline
Online
0.981 0.965 0.964
0.967 0.940 0.945
Bickley Jet
Offline
Online
0.696 0.559 0.628
0.745 0.585 0.663
TABLE I: Comparison of Evaluation Metrics on Methods for Computing Coherent Sets
(a)
(b)
(c)
(d)
Fig. 5: Comparison of offline and online clustering algorithms for the time-dependent double gyre flow at time in (a) and (b) and Bickley jet at time in (c) and (d). (a) The offline version does not take into account prior measurements of the double gyre. (b) The online version incorporates historical data about the flow and consistently computes the true coherent sets. (c) The offline clustering of the Bickley jet incorrectly computes the coherent sets, even when close to the true time parameter. (d) The online method produces robust estimates of the coherent sets.

We compare the online algorithm, using Eq. (19), to the offline algorithm, where we naively compute surrogate eigenvalues of the operator at each snapshot. We compare methods for external cluster validation, where evaluation is based on comparing the online and offline clustering to a known true clustering. The true clustering is found by constructing the operator for the true time parameter and using the -means algorithm for the known number of clusters over multiple random initializations of cluster centers and averaging the clusters. The results are shown in Table I. We use the Rand index adjusted for chance (RI), homogeneity (H), and the V-measure (V). The Rand index measures similarity of the assignments, homogeneity indicates if each cluster contains only members of a single class, and V-measure is a combination of homogeneity and completeness, where completeness indicates if all members of a given class are assigned to the same cluster. The evaluation metric is found by comparing clusters from each time step against the true clusters. This is then averaged across all time steps.

Across all metrics, the online method for the double gyre performs slightly worse than the offline method. In both the offline and online clustering, the clustering are qualitatively similar to the true clustering in size and shape. For the more complex and realistic Bickley jet, the online method outperforms the offline method across all metrics. In both scenarios, the online method is less sensitive to noise and invalid clustering, as shown in Fig. 5. The simulations and experiments conducted in this paper rely on the online method, which requires less knowledge about the environment and works better in complicated, real world scenarios due to its incorporation of data over time.

Iv Path Planning in Flow-like Environments with Aerial and Surface Vehicles 111Accompanying video can be found at https://youtu.be/D-4S6hnFr2E

In this section, we discuss how computing coherent sets in real time facilitates robots’ awareness. We then suggest how this awareness can be incorporated into a planning scheme.

Iv-a Aerial vehicle based crowd monitoring

Robots operating in dense environments, such as urban landscapes or roadways, rely on persistent monitoring of crowds for scene understanding and safe navigation. Much of the literature in this area surrounds tracking individual trajectories of pedestrians or other objects. However, understanding global information about pedestrian movement allows for greater awareness of high-level behaviors and facilitates informed planning. With aerial vehicles monitoring an environment overhead, we can collect trajectory information of pedestrians, which can be thought of as a flow-like environment

[3]. This information can be used within our framework to elucidate key features of the underlying persistent crowd behaviors.

Overhead data of pedestrians at a train station stop was collected from the th floor of an hotel in Bahanhofstr, Zurich [35]. The original sequence is processed with a time step of . We found and analyzed the dataset using tools from OpenTraj [4]. We compute coherent sets using trajectories of pedestrians over time steps, with a -degree polynomial kernel and -means clusters on eigenvalues.

Fig. 6: Aerial view of pedestrians walking through train station with coherent sets overlaid. Coherent sets correspond to regions of interest including train platform, entry areas, and exit areas, indicated by different colored circles. Regions could be used for decision-making in urban or dense environments.

In Fig. 6, we see coherent sets illustrate areas of congregation or high movement for pedestrians. Pedestrian data can be collected in real time to track coherent sets online, where sets would then be used to navigate through crowds, such as by avoiding the busy train platform, or deploy sensors for situational awareness, such as by monitoring the entrance for anomalous behavior or notable changes in coherence patterns.

Iv-B Energy-efficient trajectories for surface vehicles

Currents in rivers, oceans, and in the atmosphere are other examples of flow fields. While global information is complex and hard to analyze, local information does not provide an accurate description of environment features. Here, we leverage coherent sets from a single gyre to demonstrate the usefulness of path planning under awareness of these global features.

Fig. 7: Experimental test bed for surface vehicle navigation in controlled flows within a tank. The mCoSTe consists of a mASV and a water tank, where two flow driving cylinders were placed horizontally in the tank, circled in blue, to create a single gyre.

We create flows and deploy surface vehicles using the multirobot Coherent Structure Testbed (mCoSTe) which consists of a micro Autonomous Surface Vehicle (mASV) and a Multi Robot flow tank (MR tank). The MR tank is a tank of water equipped with an OptiTrack motion capture system providing localization of the mASV at . The mASV is a differential drive vehicle with a maximum forward speed of approximately . Two flow driving cylinders placed horizontally and rotating at approximately creates a single gyre flow in the MR tank, as shown in Fig 7.

We assume one or more robots are collecting velocity or trajectory data. This may be aerial vehicles collecting overhead imagery of the flow field or passive gliders directly sampling the velocity. This information is then processed in real time to compute the coherent sets. Instead of surface vehicles tracking and modeling individual trajectories within the flow field, the robot maintains the coherent sets as a global description of the dynamics of its environment. The coherent sets are calculated using an approximate flow model for the gyres in the tank for simulated particles over time steps using a Gaussian kernel with and -means clusters on eigenvalues. Then, the sets are used to construct energy-efficient paths. We plan paths from a specified start location to a goal location by picking waypoints near coherent sets, exerting control effort to travel to and escape from coherent sets, and remaining in coherent sets until the next waypoint is close.

Fig. 8: Comparing trajectories of a robot operating in a single gyre flow and using a coherent set in decision-making, with region of entry (blue), region of exit (green), coherent set (pink), and selected waypoints (stars). The trajectory of the mASV is lightest where it began and darkest where it ends. (Left) The trajectory of an mASV stuck in the single gyre while attempting to reach the goal. (Right) The trajectory of an mASV leveraging the coherent set to move through the environment efficiently and escape when close to the goal.

The robot navigates the flow induced by a single gyre model and completes its mission when planning a trajectory based on the coherent set. During execution, the robot saves energy by turning off its motors when the boundary of the coherent set is reached and floating through the minimal dispersion region. However, a boat unaware of the coherent set drives into the gyre, causing the robot to become stuck and ultimately fail to reach its goal, as seen in Fig. 8. Modeling coherent sets gives the robot a descriptive framework to reason about flows in the environment, without maintaining detailed knowledge.

V Conclusion

Robots operating in complex scenarios necessitates greater awareness of their environment. In flow-like environments, global dynamics may be more useful than tracking individual trajectories. Transfer operator theory provides a systematic framework for reasoning about global dynamics. Recent advances in machine learning and transfer operators combine powerful data representation with physical system knowledge. Here, we present a framework where coherent sets, defined by transfer operators, are learned with kernel methods, estimated online, and integrated into decision-making for robots. We foresee robotics applications incorporating awareness of environments in novel, meaningful ways with these connections. For example, our framework allows scene segmentation according to pedestrian movement instead of static objects and efficient navigation of strong flows with little information. Future work includes automated kernel function selection, a distributed framework for sensor information collected by multi-robot teams, development of more sophisticated decision-making schemes, and analysis in real environments.

References

  • [1] I. Abraham, G. De La Torre, and T. D. Murphey (2017-09) Model-Based Control Using Koopman Operators. Robotics: Science and Systems 13. External Links: Document Cited by: §I.
  • [2] I. Abraham and T. D. Murphey (2019-10) Active Learning of Dynamics for Data-Driven Control Using Koopman Operators. IEEE Transactions on Robotics 35 (5), pp. 1071–1083. External Links: Document Cited by: §I.
  • [3] S. Ali and M. Shah (2007) A Lagrangian particle dynamics approach for crowd flow segmentation and stability analysis.

    Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

    .
    External Links: Document Cited by: §IV-A.
  • [4] J. Amirian, B. Zhang, F. V. Castro, J. J. Baldelomar, J. Hayet, and J. Pettre (2020) OpenTraj: Assessing Prediction Complexity in Human Trajectories Datasets. In Asian Conference on Computer Vision (ACCV), Cited by: §IV-A.
  • [5] R. Banisch and P. Koltai (2017) Understanding the geometry of transport: Diffusion maps for lagrangian trajectory data unravel coherent sets. Chaos 27 (3). External Links: Document, ISSN 10541500 Cited by: §II-D, §II-E.
  • [6] F. J. Beron-Vera, M. J. Olascoaga, M. G. Brown, H. Koçak, and I. I. Rypina (2010) Invariant-tori-like Lagrangian coherent structures in geophysical flows. Chaos 20 (1). External Links: Document Cited by: §III-B.
  • [7] J. Bouvrie and B. Hamzi (2017-04) Kernel methods for the approximation of some key quantities of nonlinear systems. Journal of Computational Dynamics 4 (1). External Links: Document, ISSN 21582505 Cited by: §I.
  • [8] D. Bruder, B. Gillespie, C. David Remy, and R. Vasudevan (2019) Modeling and Control of Soft Robots Using the Koopman Operator and Model Predictive Control. arXiv preprint arXiv:1902.02827. External Links: Document Cited by: §I.
  • [9] S. Das and D. Giannakis (2020) Koopman spectra in reproducing kernel Hilbert spaces. Applied and Computational Harmonic Analysis 49, pp. 573–607. External Links: Link Cited by: §II-A.
  • [10] D. Del-Castillo-Negrete and P. J. Morrison (1992) Chaotic transport by Rossby waves in shear flow. Physics of Fluids A 5 (4), pp. 948–965. External Links: Document Cited by: §III-B.
  • [11] M. Dellnitz and O. Junge (1999) On the approximation of complicated dynamical behavior. SIAM Journal on Numerical Analysis 36 (2), pp. 491–515. External Links: Document, ISSN 00361429 Cited by: §I.
  • [12] H. E. Fiedler (1988-01) Coherent structures in turbulent flows. Progress in Aerospace Sciences 25 (3), pp. 231–269. External Links: Document, ISSN 0376-0421 Cited by: §I.
  • [13] C. Folkestad, D. Pastor, and J. W. Burdick (2020-05) Episodic Koopman Learning of Nonlinear Robot Dynamics with Application to Fast Multirotor Landing. Proceedings - IEEE International Conference on Robotics and Automation, pp. 9216–9222. External Links: Document Cited by: §I.
  • [14] E. Forgoston, L. Billings, P. Yecko, and I. B. Schwartz (2011) Set-based corral control in stochastic dynamical systems: Making almost invariant sets more invariant. Chaos. External Links: Document, ISSN 10541500 Cited by: §III-A.
  • [15] G. Froyland, S. Lloyd, and A. Quas (2009-09) Coherent structures and isolated spectrum for Perron–Frobenius cocycles. Ergodic Theory and Dynamical Systems 30 (3), pp. 729–756. External Links: Document, ISSN 1469-4417 Cited by: §I.
  • [16] G. Froyland, S. Lloyd, and N. Santitissadeekorn (2010) Coherent sets for nonautonomous dynamical systems. Physica D: Nonlinear Phenomena. External Links: Document, ISSN 01672789 Cited by: §I.
  • [17] G. Froyland, N. Santitissadeekorn, and A. Monahan (2010) Transport in time-dependent dynamical systems: Finite-time coherent sets. Chaos 20 (4), pp. 043116. External Links: Document, ISSN 1054-1500 Cited by: §I, §II-E.
  • [18] G. Froyland (2013) An analytic framework for identifying finite-time coherent sets in time-dependent dynamical systems. Physica D: Nonlinear Phenomena 250, pp. 1–19. External Links: Document Cited by: §II-E.
  • [19] A. Gretton (2013) Introduction to rkhs, and some simple kernel algorithms. Adv. Top. Mach. Learn. Lecture Conducted from University College London 16, pp. 5–3. Cited by: §II-B.
  • [20] A. Gretton (2015) Notes on mean embeddings and covariance operators. Cited by: §II-B.
  • [21] A. Hadjighasem, M. Farazmand, D. Blazevski, G. Froyland, and G. Haller (2017) A critical comparison of Lagrangian methods for coherent structure detection. Chaos 27 (5), pp. 1–25. External Links: Document, ISSN 10541500 Cited by: §III-B.
  • [22] G. Haller (2002) Lagrangian coherent structures from approximate velocity data. Physics of Fluids 14 (6), pp. 1851–1861. External Links: Document Cited by: §I.
  • [23] B. Hamzi, R. Maulik, and H. Owhadi (2021) Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 477 (2252), pp. 20210326. External Links: Document Cited by: §I.
  • [24] C. R. Heckman, M. A. Hsieh, and I. B. Schwartz (2016) Controlling basin breakout for robots operating in uncertain flow environments. In Springer Tracts in Advanced Robotics, External Links: Document, ISSN 1610742X Cited by: §I.
  • [25] T. Hofmann, B. Schölkopf, and A. J. Smola (2008) Kernel methods in machine learning. Annals of Statistics 36 (3), pp. 1171–1220. External Links: Document, ISSN 00905364, 0701907v3 Cited by: §II-B, §II-B.
  • [26] M. A. Hsieh, K. Mallory, E. Forgoston, and I. B. Schwartz (2014) Distributed allocation of mobile sensing agents in geophysical flows. In Proceedings of the American Control Conference, External Links: ISBN 9781479932726, Document, ISSN 07431619 Cited by: §I.
  • [27] T. Inanc, S. C. Shadden, and J. E. Marsden (2005) Optimal Trajectory Generation in ocean flows. Proceedings of the American Control Conference 1, pp. 674–679. External Links: Document Cited by: §I.
  • [28] Y. Kawahara (2016) Dynamic Mode Decomposition with Reproducing Kernels for Koopman Spectral Analysis. Nips (1), pp. 911–919. Cited by: §II-A.
  • [29] D. H. Kelley, M. R. Allshouse, and N. T. Ouellette (2013) Lagrangian coherent structures separate dynamically distinct regions in fluid flows. Physical Review E - Statistical, Nonlinear, and Soft Matter Physics 88 (1). External Links: Document, ISSN 15393755 Cited by: §I.
  • [30] S. Klus, B. E. Husic, and M. Mollenhauer (2019) Kernel canonical correlation analysis approximates operators for the detection of coherent structures in dynamical data. arXiv preprint arXiv:1904.07752. Cited by: §I, §I, §II-D, §II-E, §III-B.
  • [31] S. Klus, F. Nüske, and B. Hamzi (2020) Kernel-based approximation of the koopman generator and Schrödinger operator. Entropy 22 (7). External Links: Document, ISSN 10994300 Cited by: §I.
  • [32] S. Klus, I. Schuster, and K. Muandet (2020) Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spaces. Journal of Nonlinear Science 30 (1), pp. 283–315. External Links: Document, ISSN 14321467 Cited by: §I, §I, §II-A, §II-B, §II-C, §II-C.
  • [33] I. Mezić (2005) Spectral properties of dynamical systems, model reduction and decompositions. Nonlinear Dynamics, pp. 309–325. External Links: ISBN 0924-090X, Document, ISSN 0924090X Cited by: §II-A.
  • [34] P. J. Nolan, H. Foroutan, and S. D. Ross (2020) Pollution transport patterns obtained through generalized lagrangian coherent structures. Atmosphere 11 (2), pp. 14. External Links: Document, ISSN 20734433 Cited by: §I.
  • [35] S. Pellegrini, A. Ess, K. Schindler, and L. Van Gool (2009) You’ll never walk alone: Modeling social behavior for multi-target tracking. In 2009 IEEE 12th International Conference on Computer Vision, pp. 261–268. Cited by: §IV-A.
  • [36] A. G. Ramos, V. J. García-Garrido, A. M. Mancho, S. Wiggins, J. Coca, S. Glenn, O. Schofield, J. Kohut, D. Aragon, J. Kerfoot, T. Haskins, T. Miles, C. Haldeman, N. Strandskov, B. Allsup, C. Jones, and J. Shapiro (2018-03) Lagrangian coherent structure assisted path planning for transoceanic autonomous underwater vehicle missions. Scientific Reports 8 (1), pp. 1–9. External Links: Document, ISSN 20452322 Cited by: §I.
  • [37] S. C. Shadden and C. A. Taylor (2008) Characterization of coherent structures in the cardiovascular system. Annals of Biomedical Engineering 36 (7), pp. 1152–1162. External Links: ISBN 1043900895023, Document, ISSN 00906964 Cited by: §I.
  • [38] N. Takeishi (2019) Kernel Learning for Data-Driven Spectral Analysis of Koopman Operators. Proceedings of Machine Learning Research 101. Cited by: §I.
  • [39] P. Tallapragada and S. D. Ross (2013) A set oriented definition of finite-time Lyapunov exponents and coherent sets. Communications in Nonlinear Science and Numerical Simulation. External Links: Document, ISSN 10075704 Cited by: §I.
  • [40] W. Tang, P. W. Chan, and G. Haller (2011) Lagrangian Coherent Structure Analysis of Terminal Winds Detected by Lidar. Part II: Structure Evolution and Comparison with Flight Data. Journal of Applied Meteorology and Climatology 50 (10), pp. 2167–2183. External Links: Document Cited by: §I.
  • [41] C. Wei, H. G. Tanner, X. Yu, and M. A. Hsieh (2019) Low-range interaction periodic rendezvous along lagrangian coherent structures. In Proceedings of the American Control Conference, External Links: ISBN 9781538679265, ISSN 07431619 Cited by: §I.
  • [42] M. O. Williams, C. W. Rowley, and I. G. Kevrekidis (2015) A kernel-based method for data-driven Koopman spectral analysis. Journal of Computational Dynamics 2 (2). External Links: Document, ISSN 21582505 Cited by: §I.