Data-driven reduced-order models via regularized operator inference for a single-injector combustion process

by   Shane A. McQuarrie, et al.

This paper derives predictive reduced-order models for rocket engine combustion dynamics via a scientific machine learning approach, based on Operator Inference, that blends data-driven learning with physics-based modeling. The non-intrusive nature of the approach enables variable transformations that expose system structure. The specific contribution of this paper is to advance the formulation robustness and algorithmic scalability of the approach. Regularization is introduced to the formulation to avoid over-fitting. The task of determining an optimal regularization is posed as an optimization problem that balances training error and stability of long-time integration dynamics. A scalable algorithm and open-source implementation are presented, then demonstrated for a single-injector rocket combustion example. This example exhibits rich dynamics that are difficult to capture with state-of-the-art reduced models. With appropriate regularization and an informed selection of learning variables, the reduced-order models exhibit high accuracy in re-predicting the training regime and acceptable accuracy in predicting future dynamics, while achieving close to a million times speedup in computational cost. When compared to a state-of-the-art model reduction method, the Operator Inference models provide the same or better accuracy at approximately one thousandth of the computational cost.



There are no comments yet.


page 12

page 13


Operator Inference and Physics-Informed Learning of Low-Dimensional Models for Incompressible Flows

Reduced-order modeling has a long tradition in computational fluid dynam...

Gradient-free optimization of chaotic acoustics with reservoir computing

We develop a versatile optimization method, which finds the design param...

A Differentiable Solver Approach to Operator Inference

Model Order Reduction is a key technology for industrial applications in...

Bayesian operator inference for data-driven reduced-order modeling

This work proposes a Bayesian inference method for the reduced-order mod...

Machine Learning Regression for Operator Dynamics

Determining the dynamics of the expectation values for operators acting ...

Learning stable reduced-order models for hybrid twins

The concept of Hybrid Twin (HT) has recently received a growing interest...

Reduced Order and Surrogate Models for Gravitational Waves

We present an introduction to some of the state of the art in reduced or...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The emerging field of scientific machine learning brings together the perspectives of physics-based modeling and data-driven learning. In the field of fluid dynamics, physics-based modeling and simulation have played a critical role in advancing scientific discovery and driving engineering innovation in domains as diverse as biomedical engineering (Yin et al. 2010; Nordsletten et al. 2011), geothermal modeling (O’Sullivan et al. 2001; Cui et al. 2011), and aerospace (Spalart and Venkatakrishnan 2016). These advances are based on decades of mathematical and algorithmic developments in computational fluid dynamics (CFD). Scientific machine learning builds upon these rigorous physics-based foundations while seeking to exploit the flexibility and expressive modeling capabilities of machine learning (Baker et al. 2019). This paper presents a scientific machine learning approach that blends data-driven learning with the theoretical foundations of physics-based model reduction. This creates the capability to learn predictive reduced-order models (ROMs) that provide approximate predictions of complex physical phenomena while exhibiting several orders of magnitude computational speedup over CFD.

Projection-based model reduction considers the class of problems for which the governing equations are known and for which we have a high-fidelity (e.g., CFD) model (Antoulas 2005; Benner et al. 2015). The goal is to derive a ROM that has lower complexity and yields accurate solutions with reduced computation time. Projection-based approaches define a low-dimensional manifold on which the dominant dynamics evolve. This manifold may be defined as a function of the operators of the high-fidelity model, as in interpolatory methods that employ a Krylov subspace (Bai 2002; Freund 2003), or it may be determined empirically from representative high-fidelity simulation data, as in the proper orthogonal decomposition (POD) (Lumley 1967; Sirovich 1987; Berkooz et al. 1993). The POD has been particularly successful in fluid dynamics, dating back to the early applications in unsteady flows and turbulence modeling (Sirovich 1987; Deane et al. 1991; Gatski and Glauser 1992), and in unsteady fluid-structure interaction (Dowell and Hall 2001).

Model reduction methods have advanced to include error estimation

(Veroy et al. 2003; Veroy and Patera 2005; Grepl and Patera 2005; Rozza et al. 2008) and to address parametric and nonlinear problems (Barrault et al. 2004; Astrid et al. 2008; Chaturantabut and Sorensen 2010; Carlberg et al. 2013), yet the intrusive nature of the methods has limited their impact in practical applications. When legacy or commercial codes are used, as is often the case for CFD applications, it can be difficult or impossible to implement classical projection-based model reduction. Black-box surrogate modeling instead derives the ROM by fitting to simulation data; such methods include response surfaces and Gaussian process models, long used in engineering, as well as machine learning surrogate models. These methods are powerful and often yield good results, but since the approximations are based on generic data-fit representations, they are not equipped with the guarantees (e.g., stability guarantees, error estimators) that accompany projection-based ROMs. Nonlinear system identification techniques seek to illuminate the black box by discovering the underlying physics of a system from data (Brunton et al. 2016). However, when the governing dynamics are known and simulation data are available, reduced models may be directly tailored to the specific dynamics without access to the details of the large-scale CFD code.

Building on the work in Swischuk et al. (2020), this paper presents a non-intrusive alternative to black-box surrogate modeling. We use the Operator Inference method of Peherstorfer and Willcox (2016)

to learn a ROM from simulation data; the structure of the ROM is defined by the known governing equations combined with the theory of projection-based model reduction. Our approach may be termed ‘glass-box modeling’, as the targeted dynamics are known via the governing partial differential equations that define the problem of interest but the inner workings of the CFD code are not accessed. This paper extends our prior work by formally introducing regularization to the approach, which is critical to avoid overfitting for problems with complex dynamics, as is the case for the combustion example considered here. A second contribution of this paper is a scalable implementation of the approach, which is available via an open-source implementation. Section 

2 presents the methodology and regularization approach and describes the scalable implementation. Section 3 presents numerical results for a single-injector combustion problem and Section 4 concludes the paper.

2 Methodology

This section begins with an overview the Operator Inference approach in Section 2.1. Section 2.2 augments Operator Inference with a new regularization formulation, posed as an optimization problem, and presents a complete algorithm for regularization selection and model learning. In Section 2.3, we discuss a scalable implementation of the algorithm, which can then be applied to CFD problems of high dimension.

2.1 Operator Inference

We target problems governed by systems of nonlinear partial differential equations. Consider the governing equations of the system of interest written, after spatial discretization, in semi-discrete form



is the state vector discretized over

points in space at time , denotes the inputs at time , typically related to boundary conditions or forcing terms, and are respectively the initial and final time, and is the given initial condition. We refer to Eq. (1) as the full-order model (FOM) and note that it has been written to have a polynomial structure: are constant terms; are the terms that are linear in the state , with the discretized operator ; are the terms that are quadratic in , with ; and are the terms that are linear in the input , with . This polynomial structure arises in three ways: (1) it may be an attribute of the governing equations; (2) it may be exposed via variable transformations; or (3) it may be derived by introducing auxiliary variables through the process of lifting. As examples of each: (1) the incompressible Navier-Stokes equations have a quadratic form; (2) the Euler equations can be transformed to quadratic form by using pressure, velocity, and specific volume as state variables; (3) a nonlinear tubular reactor model with Arrhenius reaction terms can be written in quadratic form via the lifting transformation shown in Kramer and Willcox (2019) that introduces six auxiliary variables.

A projection-based reduced-order model (ROM) of Eq. (1) preserves the polynomial structure (Benner et al. 2015). Approximating the high-dimensional state in a low-dimensional basis , with , we write , where is the reduced state. Using a Galerkin projection, this yields the intrusive ROM of Eq. (1):

where , , , and are the ROM operators corresponding to the FOM operators , , , and , respectively. The ROM is intrusive because computing these ROM operators requires access to the discretized FOM operators, which typically entails intrusive queries and/or access to source code.

The non-intrusive Operator Inference (OpInf) approach proposed by Peherstorfer and Willcox (2016) parallels the intrusive projection-based ROM setting, but learns ROMs from simulation data without direct access to the FOM operators. Recognizing that the intrusive ROM has the same polynomial form as Eq. (1), OpInf uses a data-driven regression approach to derive a ROM of Eq. (1) as


where , , , and are determined by solving a data-driven regression problem, and is the state of the OpInf ROM.111From here on we use to indicate a compact Kronecker product with only the unique quadratic terms (, , , ); for matrices, the product is applied column-wise.

OpInf solves a regression problem to find reduced operators that yield the ROM that best matches projected snapshot data in a minimum-residual sense. Mathematically, OpInf solves the least-squares problem


where is the dataset used to drive the learning with denoting a reduced-state snapshot at timestep , are the associated time derivatives, and is the collection of inputs corresponding to the data with . To generate the dataset, we employ the following steps: (1) Collect a set of high-fidelity state snapshots by solving the original high-fidelity model at times with inputs . (2) Apply a variable transformation to obtain snapshots of the transformed variables. Here is the map representing an invertible transformation (e.g., from density to specific volume) or a lifting transformation (Qian et al. 2020). (3) Compute the proper orthogonal decomposition (POD) basis of the transformed snapshots.222Or any other low-dimensional basis as desired. (4) Project the transformed snapshots onto the POD subspace as . (5) Estimate projected time derivative information . The training period for which we have data is a subset of the full time domain of interest ; results from the ROM over will be entirely predictive.

Eq. (3) can also be written in matrix form as



(unknown operators)
(known data)
(time derivatives)

and where and is the length- column vector with all entries set to unity. The OpInf least-squares problem Eq. (3) is therefore linear in the coefficients of the unknown ROM operators , , , and .

Note that the OpInf approach permits us to compute the ROM operators , , , and without explicit access to the original high-dimensional operators , , , and . This point is key since we apply variable transformations only to the snapshot data, not to the operators or the underlying model. Thus, even in a setting where deriving a classical intrusive ROM might be possible, the OpInf approach enables us to work with variables other than those used for the original high-fidelity discretization. In Section 3 we will see the importance of this for a reacting flow application. We also note that under some conditions, OpInf recovers the intrusive POD ROM (Peherstorfer and Willcox 2016; Peherstorfer 2019).

2.2 Regularization

Eq. (4) decouples into independent linear least-squares problems, one for each of the rows of (Peherstorfer and Willcox 2016). Each problem is generally overdetermined, but is also noisy due to error in the numerically estimated time derivatives , model mis-specification (e.g., if the system is not truly quadratic), and truncated POD modes that leave some system dynamics unresolved. The ROMs resulting from Eq. (4) can thus suffer from overfitting the operators to the data and therefore exhibit poor predictive performance over the time domain of interest .

To avoid overfitting, we introduce a Tikhonov regularization to the sub-problems in Eq. (4), which then become


where is the th row of (the th column of ), is the th row of , and is a full-rank regularizer. Each regularized sub-problem in Eq. (5) admits a unique solution since, by construction, the augmented data matrix is taller than it is wide and has full column rank.

An regularizer , and

the identity matrix, penalizes each entry of the inferred ROM operators

, , , and , thereby driving the ROM toward the globally stable system

. An appropriate value for the scalar hyperparameter

, which balances the minimization between the data fit and the regularization, must be chosen with care. The ideal produces a ROM that minimizes some error metric over the full time domain ; however, since data are only available for the smaller training domain , we choose so that the resulting ROM minimizes error over while maintaining a bound on the integrated POD coefficients over . That is, we require that any produced by the ROM trained with parameter satisfy for some . This in turn ensures a bound on the magnitude of the entries of the high-dimensional state :

where is the th element of the th POD basis vector. Note that could be chosen with the intent of imposing a particular bound on the since the sums can be precomputed.

Algorithm 1 details our regularized OpInf procedure, in which we choose as a multiple of the maximum absolute entry of the projected training data . This particular strategy for selecting could be replaced with a cross-validation or resampling grid search technique, but our experiments in this vein did not produce robust results. The training error in step 14 may compare and directly in the reduced space (e.g., with a matrix norm or an norm), or it may be replaced with a more targeted comparison of some quantity of interest.

1:procedure RegOpInf(snapshots , inputs , final time , invertible map , reduced dimension , regularization parameter interval , bound margin )
2:      Map native variables to learning variables (columnwise).
3:     pod() Compute a rank- POD basis from transformed snapshots.
4:      Project snapshots to the -dimensional POD subspace.
5:      Estimate time derivatives of the projected snapshots.
6:      The initial condition for Eq. (2) is the first projected snapshot.
7:      Select a bound to require for integrated POD modes.
8:     procedure TrainError()
9:          solve Eq. (5) with data and regularizer
10:          integrate Eq. (2) with from over
11:         if  then
12:              return Disqualify ROMs that violate the bound.
13:         else
14:              return Compute ROM error over .               
15:     TrainError Find the with least training error.
16:      solve Eq. (5) with data and optimal regularizer
17:     return
Algorithm 1 Regularized Operator Inference with regularization selection

2.3 Scalable Implementation

The steps of Algorithm 1 are highly modular and amenable to large problems. The variable transformation of step 2 consists of elementary computations on the original snapshot matrix . To compute the rank- POD basis in step 3

, we use a randomized singular-value decomposition (SVD) algorithm requiring

operations (Halko et al. 2011); since , the leading order behavior is . The projection in step 4 costs operations. Note that is small compared to , as typically and . The time derivatives in step 5 may be provided by the full-order solver or estimated, e.g., with finite differences of the columns of . In the latter case, the cost is . The computational cost of steps 26 is therefore .

Solving the regularized problems Eq. (5) in steps 9 and 16 requires computing the SVD of the augmented data matrix — equivalently, the eigendecomposition of — which costs operations (Demmel 1997). This SVD is then applied to each of the augmented vectors to produce the unique solution. The ROM integration in step 10 can be carried out with any time-stepping scheme; for explicit methods, evaluating the ROM at a single point, i.e., computing the right-hand side of Eq. (2), costs operations. The total cost of evaluating the subroutine TrainError is therefore which, importantly, is independent of . The minimization in step 15 is carried out with a bisection-type search method, which enables fewer total evaluations of the subroutine than a blind grid search. However, an initial coarse grid search is useful for identifying an appropriate search interval .

3 Results

This section applies regularized OpInf to a single-injector combustion problem, studied previously by Swischuk et al. (2020), on the two-dimensional computational domain shown in Figure 1. Section 3.1 describes the governing dynamics, a set of high-fidelity data obtained from a CFD code, and the variable transformations used to produce training data for learning reduced models with Algorithm 1. The resulting OpInf ROM performance is analyzed in Section 3.2 and compared to a state-of-the-art intrusive model reduction method in Section 3.3.

3.1 Problem setup

Figure 1: The computational domain for the single-injector combustion problem. On the left, monitor locations for numerical results. On the right, a typical temperature field demonstrating the complexity and nonlinear nature of the problem.

The combustion dynamics for this problem are governed by conservation laws


where are the conservative variables, are the inviscid flux terms, are the viscous flux terms, and are the source terms. Here is the density , and are the and velocity , is the total energy , and is the th species mass fraction with . The chemical species are CH, O, HO, and CO, which follow a global one-step chemical reaction (Westbrook and Dryer 1981). See Harvazinski et al. (2015) for more details on the governing equations.

At the downstream end of the combustor, we impose a non-reflecting boundary condition while maintaining the chamber pressure via


where  Pa and  Hz. The top and bottom wall boundary conditions are no-slip conditions, and for the upstream boundary we impose a constant mass flow at the inlets.

Swischuk et al. (2020) show that if the governing equations (6) are transformed to be written in the specific volume variables, many of the terms take a quadratic form. Following that idea, we choose as learning variables the transformed and augmented state where is the pressure , is the temperature , is the specific volume , and are the species molar concentrations given by with the molar mass of the th species . As shown in Swischuk et al. (2020), the equations for , , and are exactly quadratic in , while the remaining equations are quadratic with some non-polynomial terms in . Note that, differently from Swischuk et al. (2020), here is chosen to contain specific volume, pressure, and temperature, even though only two of the three quantities are needed to fully define the high-fidelity model (and the equation of state then defines the third). We augment the learning variables in this way because doing so exposes the quadratic form while also directly targeting the variables that are of primary interest for assessing ROM performance. In particular, the resulting ROMs provide more accurate predictions of temperature when temperature is included explicitly as a learning variable. We can do this since the transformations are applied only to the snapshot data, not to the CFD model itself. This flexibility is a major advantage of the non-intrusive OpInf approach in comparison to traditional intrusive projection-based model reduction methods.

To generate high-fidelity training data, we use the finite-volume based General Equation and Mesh Solver (GEMS) (Harvazinski et al. 2015) to solve for the variables over cells, resulting in snapshots with entries each. The snapshots are computed for 60,000 time steps beyond the initial condition with a temporal discretization of  s, from  s to  s. The computational cost of computing this dataset is approximately 1,200 CPU hours on two computing nodes, each of which contains two Haswell CPUs at  GHz and cores per node.

Scaling is an essential aspect of successful model reduction and is particularly critical for this problem due to the wide range of scales across variables. After transforming the GEMS snapshot data to the learning variables , the species molar concentrations are scaled to , and all other variables are scaled to . This scaling ensures that null velocities and null molar concentrations are preserved. For example, some upstream regions of the injector have zero methane concentration at all times. By construction, the POD basis vectors and thus the ROM predictions will preserve those zero concentration values.

We implement Algorithm 1 via the rom_operator_inference Python package,333See which is built on NumPy, SciPy, and scikit-learn (Walt et al. 2011; Virtanen et al. 2020; Buitinck et al. 2013). The time derivatives in step 5 of Algorithm 1 are estimated with fourth-order finite differences, and the least-squares problems in step 9 are solved by the divide-and-conquer LAPACK routine DGELSD. The learned ROMs are integrated in step 10 with the explicit, adaptive, fourth-order Runge-Kutta scheme RK45, and the error evaluation of step 14 uses the norm in the reduced space. To minimize the scalar function TrainError in step 15, we use Brent’s method (Brent 2002) with enforced bounds . The code and details are publicly available at

3.2 Sensitivity to Training Data

We study the sensitivity of our approach to the training data by varying the number of snapshots used to compute the POD basis and learn the OpInf ROM. Specifically, we consider the three cases where we use the first , , and snapshots from GEMS as training data sets. In each case we compute the POD basis, and to select an appropriate reduced dimension , we compute the cumulative energy based on the POD singular values: , where are the singular values of the learning variable snapshot matrix (see Figure 2). Specifically, we choose the minimal such that is greater than a fixed threshold energy. Table 3.2 shows that is increasing linearly with the number of snapshots, indicating that the basis is not being saturated as additional information is incorporated. This is an indication of the challenging nature of this application, due to the rich and complex dynamics. It is also an indication of the importance of having sufficient training data.

Figure 2: The POD singular values for varying size of the snapshot training set.

The basis size required to reach a given cumulative energy level increases linearly with the number of snapshots in the training set, .

Cumulative energy

Figure 3 plots the GEMS and OpInf ROM results for pressure, temperature and -velocity predictions over time at the monitor locations in Figure 1.444See for additional results. While it can be misleading to assess accuracy based on predictions at a single spatial point, these plots reveal several representative insights. First, we see the importance of the training data—as the amount of training data increases, the ROM predictions change significantly and generally (but not always) improve. This is yet another indication of the complexity of the dynamics we are aiming to approximate. Second, we see that good ROM performance over the training region is not sufficient to produce good predictive behavior. In particular, the ROM with only training snapshots can accurately re-predict the training dynamics, but shows large errors as it predicts further beyond the training horizon. Third, we see that pressure and -velocity are better approximated by the ROM than is temperature. The effects of the 5,000 Hz downstream pressure forcing are clearly visible in the pressure and -velocity traces. The temperature dynamics, on the other hand, are more irregular, and exhibit short-term swings exceeding 1,000 K. This is because the temperature profile is influenced by both the flow dynamics and the local chemical reactions, which in combination lead to a highly nonlinear and multiscale behavior. The ROM cannot accurately predict the detailed variations, but with training snapshots, the ROM does an adequate job of predicting the general trends of temperature evolution beyond the training horizon.

Figure 3: Traces of , , and through time at monitor locations , , and of Figure 1, respectively, for (top row), (middle row), and (bottom row) training snapshots. The vertical black lines separate the training and prediction periods. For each choice of the error over the training domain is low, but increasing has a significant impact on the error in the testing domain. Here is chosen in each case so that .

Figure 4 plots integrated quantities as a function of time: the temperature averaged over the spatial domain, and the CH and O concentrations integrated over the spatial domain. These measures give a more global sense of the ROM predictive accuracy. In each case, the ROMs are able to accurately re-predict the training data and capture much of the overall system behavior in the prediction phase.

Figure 4: Spatially averaged temperature and molar concentration integrals for CH and O, computed over the spatial domain for each point in time, for (top row), (middle row), and (bottom row) training snapshots. These statistical features further highlight the importance of increasing . As in Figure 3, is chosen so that .

3.3 Comparison to POD-DEIM

We now compare regularized OpInf to a state-of-the-art nonlinear model reduction method that uses a least-squares Petrov-Galerkin POD projection coupled with the discrete empirical interpolation method (DEIM)

(Chaturantabut and Sorensen 2010), as implemented for the same combustion problem in Huang et al. (2019, 2018). This POD-DEIM method is intrusive—it requires nonlinear residual evaluations of the GEMS code at sparse discrete interpolation points. This also increases the computational cost of solving the POD-DEIM ROM in comparison to the OpInf ROM: integrating a POD-DEIM ROM with for 6,000 time steps of size  s takes approximately minutes on two nodes, each with two Haswell CPUs processors at  GHz and cores per node; for OpInf, using Python and a single CPU on an AMD EPYC 7,702 64-core processor at  GHz with  TB RAM, we solve Eq. (5) with training snapshots and POD modes in approximately  s and integrate the resulting OpInf ROM for 60,000 time steps of size  s in approximately  s. While these measurements are made on different hardware, and though the execution time for POD-DEIM can be improved with optimal load balancing, the difference in execution times (30 minutes versus 0.5 seconds) is representative and illustrates one of the advantages over POD-DEIM of the polynomial form employed in the OpInf approach.

Figure 5: Pressure trace at monitor location of Figure 1 and the spatially averaged temperature, computed by GEMS, a POD-DEIM ROM, and an OpInf ROM. Both ROMs use training snapshots with chosen so that .

Figure 5 compares select GEMS outputs to POD-DEIM and OpInf ROM outputs, with each ROM trained on training snapshots and the ROM dimension chosen to achieve a cumulative energy of . Both approaches reconstruct the training data well and maintain appropriate pressure oscillation frequencies, but they struggle to predict the erratic temperature dynamics beyond the training horizon. The figure shows that, for this monitor location, the temperature signal over the training region is not particularly representative of the dynamics observed at later times.

Figures 6, 7 and 8 show, respectively, full-domain results for the pressure, temperature, and molar concentration of CH. The figures show the solution at time instants within the training regime, at the end of the training regime, and into the prediction regime. As with the point traces shown earlier, we see that the ROMs have impressive accuracy over the training region, but lose accuracy as they attempt to predict dynamics beyond the training horizon. Pressure is again well predicted, but the temperature and CH concentration fields predicted at  s are not pointwise accurate. However, many of the coherent features are reasonably predicted, especially the recirculation zone dynamics near the dump plane ( in Figure 1) shown in the temperature fields.

(a)  s.
(b)  s.
(c)  s.
Figure 6: Pressure produced by GEMS (top row), POD-DEIM (middle row), and OpInf (bottom row), at times within the training regime (left column), at the end of the training regime (middle column), and into the prediction regime (right column), with training snapshots and chosen so that .
(a)  s.
(b)  s.
(c)  s.
Figure 7: Temperature produced by GEMS (top row), POD-DEIM (middle row), and OpInf (bottom row), at times within the training regime (left column), at the end of the training regime (middle column), and into the prediction regime (right column), with training snapshots and chosen so that .
(a)  s.
(b)  s.
(c)  s.
Figure 8: Molar concentrations of CH produced by GEMS (top row), POD-DEIM (middle row), and OpInf (bottom row), at times within the training regime (left column), at the end of the training regime (middle column), and into the prediction regime (right column), with training snapshots and chosen so that .

4 Conclusions

The presented scientific machine learning approach is broadly applicable to problems where the governing equations are known but access to the high-fidelity simulation code is limited. The approach is computationally as accessible as black-box surrogate modeling while achieving the accuracy of intrusive projection-based model reduction. While the conclusions drawn from the numerical studies apply to the single-injector combustion example, they are relevant and likely apply to other problems. First, the quality and quantity of the training data are critical to the success of the method. Second, regularization is essential to avoid overfitting. Third, achieving a low error over the training regime is not necessarily indicative of a reduced model with good predictive capability. This emphasizes the importance of the training data. Fourth, physical quantities that exhibit large-scale coherent structures (e.g., pressure) are more accurately predicted by a reduced-order model than quantities that exhibit multiscale behavior (e.g., temperature, species concentrations). Fifth, a significant advantage of the data-driven learning aspects of the approach is that the reduced model may be derived in any variables. This includes the possibility to include redundancy in the learning variables (e.g., to include both pressure and temperature). Overall, this paper illustrates the power and effectiveness of learning from data through the lens of physics-based models as a physics-grounded alternative to black-box machine learning.


This work has been supported in part by the Air Force Center of Excellence on Multi-Fidelity Modeling of Rocket Combustor Dynamics under award FA9550-17-1-0195, and the US Department of Energy AEOLUS MMICC center under award DE-SC0019303.


  • Antoulas (2005) Antoulas AC. 2005. Approximation of large-scale dynamical systems. Philadelphia, PA: SIAM.
  • Astrid et al. (2008) Astrid P, Weiland S, Willcox K, Backx T. 2008. Missing point estimation in models described by proper orthogonal decomposition. IEEE Transactions on Automatic Control. 53(10):2237–2251.
  • Bai (2002) Bai Z. 2002. Krylov subspace techniques for reduced-order modeling of large-scale dynamical systems. Applied Numerical Mathematics. 43(1–2):9–44.
  • Baker et al. (2019)

    Baker N, Alexander F, Bremer T, Hagberg A, Kevrekidis Y, Najm H, Parashar M, Patra A, Sethian J, Wild S, et al. 2019. Workshop report on basic research needs for scientific machine learning: Core technologies for artificial intelligence. USDOE Office of Science (SC), Washington, DC (United States). Report No:.
  • Barrault et al. (2004) Barrault M, Maday Y, Nguyen NC, Patera AT. 2004. An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations. Comptes Rendus Mathematique. 339(9):667–672.
  • Benner et al. (2015) Benner P, Gugercin S, Willcox K. 2015. A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Review. 57(4):483–531.
  • Berkooz et al. (1993) Berkooz G, Holmes P, Lumley J. 1993. The proper orthogonal decomposition in the analysis of turbulent flows. Annual Review of Fluid Mechanics. 25:539–575.
  • Brent (2002) Brent RP. 2002. Algorithms for minimization without derivatives. New York, NY: Dover Publications.
  • Brunton et al. (2016) Brunton SL, Proctor JL, Kutz JN. 2016. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences. 113(15):3932–3937.
  • Buitinck et al. (2013) Buitinck L, Louppe G, Blondel M, Pedregosa F, Mueller A, Grisel O, Niculae V, Prettenhofer P, Gramfort A, Grobler J, et al. 2013. API design for machine learning software: Experiences from the scikit-learn project. In: ECML PKDD Workshop: Languages for Data Mining and Machine Learning. p. 108–122.
  • Carlberg et al. (2013) Carlberg K, Farhat C, Cortial J, Amsallem D. 2013. The GNAT method for nonlinear model reduction: Effective implementation and application to computational fluid dynamics and turbulent flows. Journal of Computational Physics. 242:623–647.
  • Chaturantabut and Sorensen (2010) Chaturantabut S, Sorensen DC. 2010. Nonlinear model reduction via discrete empirical interpolation. SIAM Journal on Scientific Computing. 32(5):2737–2764.
  • Cui et al. (2011) Cui T, Fox C, O’Sullivan MJ. 2011. Bayesian calibration of a large-scale geothermal reservoir model by a new adaptive delayed acceptance Metropolis Hastings algorithm. Water Resources Research. 47(10).
  • Deane et al. (1991) Deane A, Kevrekidis I, Karniadakis GE, Orszag S. 1991. Low-dimensional models for complex geometry flows: Application to grooved channels and circular cylinders. Physics of Fluids A: Fluid Dynamics. 3(10):2337–2354.
  • Demmel (1997) Demmel JW. 1997. Applied numerical linear algebra. Philadelphia, PA: SIAM.
  • Dowell and Hall (2001) Dowell EH, Hall KC. 2001. Modeling of fluid-structure interaction. Annual Review of Fluid Mechanics. 33(1):445–490.
  • Freund (2003) Freund R. 2003. Model reduction methods based on Krylov subspaces. Acta Numerica. 12:267–319.
  • Gatski and Glauser (1992) Gatski T, Glauser M. 1992. Proper orthogonal decomposition based turbulence modeling. In: Instability, transition, and turbulence. Springer; p. 498–510.
  • Grepl and Patera (2005) Grepl M, Patera A. 2005. A posteriori error bounds for reduced-basis approximations of parametrized parabolic partial differential equations. ESAIM-Mathematical Modelling and Numerical Analysis (M2AN). 39(1):157–181.
  • Halko et al. (2011) Halko N, Martinsson PG, Tropp JA. 2011. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review. 53(2):217–288.
  • Harvazinski et al. (2015) Harvazinski ME, Huang C, Sankaran V, Feldman TW, Anderson WE, Merkle CL, Talley DG. 2015. Coupling between hydrodynamics, acoustics, and heat release in a self-excited unstable combustor. Physics of Fluids. 27(4):045102.
  • Huang et al. (2019) Huang C, Duraisamy K, Merkle CL. 2019. Investigations and improvement of robustness of reduced-order models of reacting flow. AIAA Journal. 57(12):5377–5389.
  • Huang et al. (2018) Huang C, Xu J, Duraisamy K, Merkle C. 2018. Exploration of reduced-order models for rocket combustion applications. In: 2018 AIAA Aerospace Sciences Meeting; Orlando, FL. Paper AIAA-2018-1183.
  • Kramer and Willcox (2019) Kramer B, Willcox K. 2019. Nonlinear model order reduction via lifting transformations and proper orthogonal decomposition. AIAA Journal. 57(6):2297–2307.
  • Lumley (1967) Lumley J. 1967. The structures of inhomogeneous turbulent flow. Atmospheric Turbulence and Radio Wave Propagation:166–178.
  • Nordsletten et al. (2011) Nordsletten D, McCormick M, Kilner P, Hunter P, Kay D, Smith N. 2011. Fluid–solid coupling for the investigation of diastolic and systolic human left ventricular function. International Journal for Numerical Methods in Biomedical Engineering. 27(7):1017–1039.
  • O’Sullivan et al. (2001) O’Sullivan M, Pruess K, Lippmann M. 2001. State of the art of geothermal reservoir simulation. Geothermics. 30:395–429.
  • Peherstorfer (2019) Peherstorfer B. 2019. Sampling low-dimensional Markovian dynamics for pre-asymptotically recovering reduced models from data with operator inference. arXiv:190811233.
  • Peherstorfer and Willcox (2016) Peherstorfer B, Willcox K. 2016. Data-driven operator inference for nonintrusive projection-based model reduction. Computer Methods in Applied Mechanics and Engineering. 306:196–215.
  • Qian et al. (2020) Qian E, Kramer B, Peherstorfer B, Willcox K. 2020. Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems. Physica D: Nonlinear Phenomena. 406:132401.
  • Rozza et al. (2008) Rozza G, Huynh D, Patera A. 2008. Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations: application to transport and continuum mechanics. Archives of Computational Methods in Engineering. 15(3):229–275. Available from:
  • Sirovich (1987) Sirovich L. 1987. Turbulence and the dynamics of coherent structures. I. Coherent structures. Quarterly of Applied Mathematics. 45(3):561–571.
  • Spalart and Venkatakrishnan (2016) Spalart P, Venkatakrishnan V. 2016. On the role and challenges of cfd in the aerospace industry. The Aeronautical Journal. 120(1223):209.
  • Swischuk et al. (2020) Swischuk R, Kramer B, Huang C, Willcox K. 2020. Learning physics-based reduced-order models for a single-injector combustion process. AIAA Journal. 58(6):2658–2672.
  • Veroy and Patera (2005) Veroy K, Patera A. 2005. Certified real-time solution of the parametrized steady incompressible Navier-Stokes equations: Rigorous reduced-basis a posteriori error bounds. International Journal for Numerical Methods in Fluids. 47:773–788.
  • Veroy et al. (2003) Veroy K, Prud’homme C, Rovas D, Patera A. 2003. A posteriori error bounds for reduced-basis approximation of parametrized noncoercive and nonlinear elliptic partial differential equations. In: Proceedings of the 16th AIAA Computational Fluid Dynamics Conference; Orlando, FL. Paper AIAA-2003-3847.
  • Virtanen et al. (2020) Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, Burovski E, Peterson P, Weckesser W, Bright J, et al. 2020. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods. 17:261–272.
  • Walt et al. (2011) Walt Svd, Colbert SC, Varoquaux G. 2011. The NumPy array: A structure for efficient numerical computation. Computing in Science & Engineering. 13(2):22–30.
  • Westbrook and Dryer (1981) Westbrook CK, Dryer FL. 1981. Simplified reaction mechanisms for the oxidation of hydrocarbon fuels in flames. Combustion Science and Technology. 27(1-2):31–43.
  • Yin et al. (2010) Yin Y, Choi J, Hoffman EA, Tawhai MH, Lin CL. 2010. Simulation of pulmonary air flow with a subject-specific boundary condition. Journal of Biomechanics. 43(11):2159–2163.