Fluid-structure interactions (FSI) are omnipresent in engineering applications MP_book_1998; MP_book_2004, e.g. in long pipes carrying fluids, in heat exchangers, in wind turbines, in gas turbines, in oil platforms and long risers for deep sea drilling. Vortex induced vibrations (VIV), in particular, are a special class of fluid-structure interactions (FSI), which involve a resonance condition. They are caused in external flows past bluff bodies when the frequency of the shed vortices from the body is close to the natural frequency of the structure Williamson_2004. A prototypical example is flow past a circular cylinder that involves the so-called von Kármán shedding with a non-dimensional frequency (Strouhal number) of about . If the cylinder is elastically mounted, its resulting motion is caused by the lift force and the drag force in the crossflow and streamwise directions, respectively, and can reach about and in amplitude, where is the cylinder diameter. Clearly, for large structures like a long riser in deep sea drilling, this is a very large periodic motion that will lead to fatigue and hence a short life time for the structure.
Using traditional computational fluid dynamics (CFD) methods we can predict accurately VIV (Evangelinos_VIV), both the flow field and the structure’s motion. However, CFD simulations are limited to relatively low Reynolds numbers and simple geometric configurations and involve the generation of elaborate and moving grids that may need to be updated frequently. Moreover, some of the structural characteristics, e.g. damping, are not readily available and hence separate experiments are required to obtain such quantities involved in CFD modeling of FSI. Solving inverse coupled CFD problems, however, is typically computationally prohibitive and often requires the solution of ill-posed problems. For the vibrating cylinder problem, in particular, we may have available data for the motion of the cylinder or some limited noisy measurements of the velocity field in the wake or some flow visualizations obtained by injecting dye upstream for liquid flows or smoke for air flows. Of interest is to determine the forces on the body that will determine the dynamic motion and possibly deformation of the body, and ultimately its fatigue life for safety evaluations.
In this work, we take a different approach building on our previous work on physics informed deep learning raissi2017physics_I; raissi2017physics_II
and extending this concept to coupled multi-physics problems. Instead of solving the fluid mechanics equations and the dynamic equation for the motion of the structure using numerical discretization, we learn the velocity and pressure fields and the structure’s motion using coupled deep neural networks with scattered data in space-time as input. The governing equations are employed as part of the loss function and play the role of regularization mechanisms. Hence, experimental input that may be noisy and at scattered spatio-temporal locations can be readily utilized in this new framework. Moreover, as we have shown in previous work,physics informed neural networks are particularly effective in solving inverse and data assimilation problems in addition to leveraging and discovering the hidden physics of coupled multi-physics problems raissi2018deep.
This work aims to demonstrate feasibility and accuracy of a new approach that involves Navier-Stokes informed deep neural networks inference, and is part of our ongoing development of physics-informed learning machines raissi2017physics_I; raissi2017physics_II. The first glimpses of promise for exploiting structured prior information to construct data-efficient and physics-informed learning machines have already been showcased in the recent studies of raissi2017inferring; raissi2017machine; owhadi2015bayesian. There, the authors employed Gaussian process regression Rasmussen06gaussianprocesses
to devise functional representations that are tailored to a given linear operator, and were able to accurately infer solutions and provide uncertainty estimates for several prototype problems in mathematical physics. Extensions to nonlinear problems were proposed in subsequent studies by Raissiet. al. raissi2018numerical; raissi2018hidden in the context of both inference and systems identification. Despite the flexibility and mathematical elegance of Gaussian processes in encoding prior information, the treatment of nonlinear problems introduces two important limitations. First, in raissi2018numerical; raissi2018hidden a local linearization in time of the nonlinear terms in time is involved, thus limiting the applicability of the proposed methods to discrete-time domains and possibly compromising the accuracy of their predictions in strongly nonlinear regimes. Secondly, the Bayesian nature of Gaussian process regression requires certain prior assumptions that may limit the representation capacity of the model and give rise to robustness/brittleness issues, especially for nonlinear problems owhadi2015brittleness, although this may be overcome by hybrid neural networks/Gaussian process algorithms pang2018neural.
Physics informed deep learning raissi2017physics_I; raissi2017physics_II; raissi2018deep takes a different approach by employing deep neural networks and leveraging their well known capability as universal function approximators hornik1989multilayer. In this setting, one can directly tackle nonlinear problems without the need for committing to any prior assumptions, linearization, or local time-stepping. Physics informed neural networks exploit recent developments in automatic differentiation baydin2015automatic – one of the most useful but perhaps under-utilized techniques in scientific computing – to differentiate neural networks with respect to their input coordinates and model parameters to leverage the underlying physics of the problem. Such neural networks are constrained to respect any symmetries, invariances, or conservation principles originating from the physical laws that govern the observed data, as modeled by general time-dependent and nonlinear partial differential equations. This simple yet powerful construction allows us to tackle a wide range of problems in computational science and introduces a potentially transformative technology leading to the development of new data-efficient and physics-informed learning machines, new classes of numerical solvers for partial differential equations, as well as new data-driven approaches for model inversion and systems identification.
Here, we should underline an important distinction between this line of work and existing approaches (see e.g., beidokhti2009solving) in the literature that elaborate on the use of machine learning in computational physics. The term physics-informed machine learning has been also recently used by Wang et. al. wang2017comprehensive in the context of turbulence modeling. Other examples of machine learning approaches for predictive modeling of physical systems include zhu2018bayesian; hagge2017solving; tripathy2018deep; vlachas2018data; parish2016paradigm; duraisamy2015new; ling2016reynolds; zhang2015machine; milano2002neural; perdikaris2016multifidelity; rico1994continuous; ling2015evaluationblack-box tools. As described above, our goal here is to open the black-box, understand the mechanisms inside it, and utilize them to develop new methods and tools that could potentially lead to new machine learning models, novel regularization procedures, and efficient and robust inference techniques. To this end, the proposed work draws inspiration from the early contributions of Shekari Beidokhti and Malek beidokhti2009solving, Psichogios and Ungar psichogios1992hybrid, Lagaris et. al. lagaris1998artificial, as well as the contemporary works of Kondor kondor2018n; kondor2018generalization, Hirn hirn2017wavelet, and Mallat mallat2016understanding.
The focus of this paper is fluid-structure interactions (FSI) in general and vortex induced vibrations (VIV) in particular. Specifically, we are interested in the prediction of the fluid’s lift and drag forces on the structure given some limited and scattered information on the velocity field or simply snapshots of dye visualization. This is a data assimilation problem that is notoriously difficult to solve using standard computational fluid dynamics (CFD) methods, especially since no initial conditions are specified for the algorithm, the training domain is small leading to the well-known numerical artifacts, and no information is provided for the pressure. An even greater challenge is to infer the lift and drag forces given some dye or smoke visualizations of the flow field. Here we employ Navier-Stokes informed deep neural networks that are extended to encode the structure’s dynamic motion equation. The paper is organized as follows. In the next section, we give an overview of the proposed algorithm, set up the problem and describe the synthetic data that we generate to test the performance of the method. In section 3, we present our results for three benchmark cases; (1) We start with a pedagogical example and assume that we know the forces acting on the body while seeking to obtain the structure’s motion in addition to its properties without explicitly solving the equation for displacement. (2) We then consider a case where we know the velocity field and the structural motion at some scattered data points in space-time. In this case, we try to infer the lift and drag forces while learning the pressure field as well as the entire velocity field in addition to the structure’s dynamic motion. (3) We then consider an even more interesting case where we only assume the availability of concentration data in space-time. From such information we obtain all components of the flow fields and structure’s motion as well as lift and drag forces. We conclude with a short summary.
2 Problem Setup and Solution Methodology
We begin by considering the prototype VIV problem of flow past a circular cylinder. The fluid motion is governed by the incompressible Navier-Stokes equations while the dynamics of the structure is described in a general form involving displacement, velocity, and acceleration terms. In particular, let us consider the two-dimensional version of flow over a flexible cable, i.e., an elastically mounted cylinder (Bourguet_2011_JFM). The two-dimensional problem contains most of the salient features of the three-dimensional case and consequently it is relatively straightforward to generalize the proposed framework to the flexible cylinder/cable problem. In two dimensions, the physical model of the cable reduces to a mass-spring-damper system. There are two directions of motion for the cylinder: the streamwise (i.e., ) direction and the crossflow (i.e., ) direction. In this work, we assume that the cylinder can only move in the crossflow (i.e., ) direction; we concentrate on crossflow vibrations since this is the primary VIV direction. However, it is a simple extension to study cases where the cylinder is free to move in both streamwise and crossflow directions.
2.1 A Pedagogical Example
The cylinder displacement is defined by the variable corresponding to the crossflow motion. The equation of motion for the cylinder is then given by
where , , and are the mass, damping, and stiffness parameters, respectively. The fluid lift force on the structure is denoted by . The mass of the cylinder is usually a known quantity; however, the damping and the stiffness parameters are often unknown in practice. In the current work, we put forth a deep learning approach for estimating these parameters from measurements. We start by assuming that we have access to the input-output data and on the displacement and the lift force functions, respectively. Having access to direct measurements of the forces exerted by the fluid on the structure is obviously a strong assumption. However, we start with this simpler but pedagogical case and we will relax this assumption later in this section.
Inspired by recent developments in physics informed deep learning (raissi2017physics_I; raissi2017physics_II) and deep hidden physics models (raissi2018deep), we propose to approximate the unknown function by a deep neural network. This choice is motivated by modern techniques for solving forward and inverse problems involving partial differential equations, where the unknown solution is approximated either by a neural network (raissi2017physics_I; raissi2017physics_II; raissi2018deep; raissi2018forwardbackward; raissi2018multistep) or a Gaussian process (raissi2018numerical; raissi2018hidden; raissi2017inferring; raissi2017machine; raissi2017parametric; perdikaris2017nonlinear; raissi2016deep). Moreover, placing a prior on the solution is fully justified by similar approaches pursued in the past centuries by classical methods of solving partial differential equations such as finite elements, finite differences, or spectral methods, where one would expand the unknown solution in terms of an appropriate set of basis functions. Approximating the unknown function by a deep neural network and using equation (1) allow us to obtain the following physics-informed neural network (see figure 1)
It is worth noting that the damping and the stiffness parameters turn into parameters of the resulting physics informed neural network . We obtain the required derivatives to compute the residual network
by applying the chain rule for differentiating compositions of functions using automatic differentiation(baydin2015automatic). Automatic differentiation is different from, and in several respects superior to, numerical or symbolic differentiation – two commonly encountered techniques of computing derivatives. In its most basic description (baydin2015automatic)
, automatic differentiation relies on the fact that all numerical computations are ultimately compositions of a finite set of elementary operations for which derivatives are known. Combining the derivatives of the constituent operations through the chain rule gives the derivative of the overall composition. This allows accurate evaluation of derivatives at machine precision with ideal asymptotic efficiency and only a small constant factor of overhead. In particular, to compute the required derivatives we rely on Tensorflow(abadi2016tensorflow), which is a popular and relatively well documented open source software library for automatic differentiation and deep learning computations.
The shared parameters of the neural networks and , in addition to the damping and the stiffness parameters, can be learned by minimizing the following sum of squared errors loss function
The first summation in this loss function corresponds to the training data on the displacement while the second summation enforces the dynamics imposed by equation (1).
2.2 Inferring Lift and Drag Forces from Scattered Velocity Measurements
So far, we have been operating under the assumption that we have access to direct measurements of the lift force . In the following, we are going to relax this assumption by recalling that the fluid motion is governed by the incompressible Navier-Stokes equations given explicitly by
Here, and are the streamwise and crossflow components of the velocity field, respectively, while denotes the pressure, and Re is the Reynolds number based on the cylinder diameter and the free stream velocity. We consider the incompressible Navier-Stokes equations in the coordinate system attached to the cylinder, so that the cylinder appears stationary in time. This explains the appearance of the extra term in the second momentum equation (4).
Problem 1 (Viv-I)
Given scattered and potentially noisy measurements of the velocity field111Take for example the case of reconstructing a flow field from scattered measurements obtained from Particle Image Velocimetry (PIV) or Particle Tracking Velocimetry (PTV). in addition to the data on the displacement and knowing the governing equations of the flow (4), we are interested in reconstructing the entire velocity field as well as the pressure field in space-time. Such measurements are usually collected only in a small sub-domain, which may not be appropriate for classical CFD computations due to the presence of numerical artifacts. Typically, the data points are scattered in both space and time and are usually of the order of a few thousands or less in space.
To solve the aforementioned problem, we proceed by approximating the latent functions , , , and by a single neural network outputting four variables while taking as input , and . This prior assumption along with the incompressible Navier-Stokes equations (4) result in the following Navier-Stokes informed neural networks (see figure 2)
We use automatic differentiation (baydin2015automatic) to obtain the required derivatives to compute the residual networks , , and . The shared parameters of the neural networks , , , , , , and can be learned by minimizing the sum of squared errors loss function
Here, the first two summations correspond to the training data on the fluid velocity and the structure displacement while the last summation enforces the dynamics imposed by equation (4).
The fluid forces on the cylinder are functions of the pressure and the velocity gradients. Consequently, having trained the neural networks, we can use
to obtain the lift and drag forces exerted by the fluid on the cylinder. Here, is the outward normal on the cylinder and is the arc length on the surface of the cylinder. We use the trapezoidal rule to approximately compute these integrals, and we use equation (8) to obtain the required data on the lift force. These data are then used to estimate the structural parameters and by minimizing the loss function (3).
2.3 Inferring Lift and Drag Forces from Flow Visualizations
We now consider the second VIV learning problem by taking one step further and circumvent the need for having access to measurements of the velocity field by leveraging the following equation
governing the evolution of the concentration of a passive scalar injected into the fluid flow dynamics described by the incompressible Navier-Stokes equations (4). Here, Pe denotes the Péclet number, defined based on the cylinder diameter, the free-stream velocity and the diffusivity of the concentration species.
Problem 2 (Viv-Ii)
Given scattered and potentially noisy measurements of the concentration of the passive scalar in space-time, we are interested in inferring the latent (hidden) quantities , , and while leveraging the governing equations of the flow (4) as well as equation (9) describing the evolution of the passive scalar. Typically, the data points are of the order of a few thousands or less in space. Moreover, equations (7) and (8) enable us to consequently compute the drag and lift forces, respectively, as functions of the inferred pressure and velocity gradients. Unlike the first VIV problem, here we assume that we do not have access to direct observations of the velocity field.
To solve the second VIV problem, in addition to approximating , , , and by deep neural networks as before, we represent by yet another output of the network taking and as inputs. This prior assumption along with equation (9) results in the following additional component of the Navier-Stokes informed neural network (see figure 3)
The residual networks , , and are defined as before according to equation (5). We use automatic differentiation (baydin2015automatic) to obtain the required derivatives to compute the additional residual network . The shared parameters of the neural networks , , , , , , , , and can be learned by minimizing the sum of squared errors loss function
Here, the first summation corresponds to the training data on the concentration of the passive scalar and the structure’s displacement, the second summation corresponds to the Dirichlet boundary data on the velocity field, and the last summation enforces the dynamics imposed by equations (4) and (9). Upon training, we use equation (8) to obtain the required data on the lift force. Such data are then used to estimate the structural parameters and by minimizing the loss function (3).
3 Results222All data and codes used in this manuscript will be publicly available on GitHub at https://github.com/maziarraissi/DeepVIV.
To generate a high-resolution dataset for the VIV problem we have performed direct numerical simulations (DNS) employing the high-order spectral-element method (GK_book), together with the coordinate transformation method to take account of the boundary deformation (NEWMAN_viv). The computational domain is , consisting of 1,872 quadrilateral elements. The cylinder center was placed at . On the inflow, located at , we prescribe . On the outflow, where , zero-pressure boundary condition is imposed. On both top and bottom boundaries where , a periodic boundary condition is used. The Reynolds number is , , and . For the case with dye, we assumed the Péclet number . First, the simulation is carried out until when the system is in steady periodic state. Then, an additional simulation for is performed to collect the data that are saved in 280 field snapshots. The time interval between two consecutive snapshots is . Note here is the diameter of the cylinder and is the inflow velocity. We use the DNS results to compute the lift and drag forces exerted by the fluid on the cylinder. All data and codes used in this manuscript will be publicly available on GitHub at https://github.com/maziarraissi/DeepVIV.
3.1 A Pedagogical Example
To illustrate the effectiveness of our approach, let us start with the two time series depicted in figure 4 consisting of observations of the displacement and the lift force. These data correspond to damping and stiffness parameters with exact values and , respectively. Here, the cylinder is assumed to have a mass of . This data-set is then used to train a 10-layer deep neural network with 32 neurons per hidden layers (see figure 1) by minimizing the sum of squared errors loss function (3) using the Adam optimizer (kingma2014adam). Upon training, the network is used to predict the entire solution functions and , as well as the unknown structural parameters and . In addition to almost perfect reconstructions of the two time series for displacement and lift force, the proposed framework is capable of identifying the correct values for the structural parameters and with remarkable accuracy. The learned values for the damping and stiffness parameters are and . This corresponds to around and relative errors in the estimated values for and , respectively.
As for the activation functions, we use . In general, the choice of a neural network’s architecture (e.g., number of layers/neurons and form of activation functions) is crucial and in many cases still remains an art that relies on one’s ability to balance the trade off between expressivity and trainability of the neural network (raghu2016expressive). Our empirical findings so far indicate that deeper and wider networks are usually more expressive (i.e., they can capture a larger class of functions) but are often more costly to train (i.e., a feed-forward evaluation of the neural network takes more time and the optimizer requires more iterations to converge). Moreover, the sinusoid (i.e., ) activation function seems to be numerically more stable than , at least while computing the residual neural networks , and , (see figures 1, 2, and 3). In this work, we have tried to choose the neural networks’ architectures in a consistent fashion throughout the manuscript by setting the number of layers to 10 and the number of neurons to 32. Consequently, there might exist other architectures that improve some of the results reported in the current work.
3.2 Inferring Lift and Drag Forces from Scattered Velocity Measurements
Let us now consider the case where we do not have access to direct measurements of the lift force . In this case, we can use measurements of the velocity field, obtained for instance via Particle Image Velocimetry (PIV) or Particle Tracking Velocimetry (PTV), to reconstruct the velocity field, the pressure, and consequently the drag and lift forces. A representative snapshot of the data on the velocity field is depicted in the top left and top middle panels of figure 5. The neural network architectures used here consist of 10 layers with 32 neurons in each hidden layer. A summary of our results is presented in figure 5. The proposed framework is capable of accurately (of the order of ) reconstructing the velocity field; however, a more intriguing result stems from the network’s ability to provide an accurate prediction of the entire pressure field in the absence of any training data on the pressure itself (see figure 6). A visual comparison against the exact pressure is presented in figure 5
for a representative snapshot of the pressure. It is worth noticing that the difference in magnitude between the exact and the predicted pressure is justified by the very nature of incompressible Navier-Stokes equations, since the pressure field is only identifiable up to a constant. This result of inferring a continuous quantity of interest from auxiliary measurements by leveraging the underlying physics is a great example of the enhanced capabilities that our approach has to offer, and highlights its potential in solving high-dimensional data assimilation and inverse problems.
The trained neural networks representing the velocity field and the pressure can be used to compute the drag and lift forces by employing equations (7) and (8), respectively. The resulting drag and lift forces are compared to the exact ones in figure 7. In the following, we are going to use the computed lift force to generate the required training data on and estimate the structural parameters and by minimizing the loss function (3). Upon training, the proposed framework is capable of identifying the correct values for the structural parameters and with remarkable accuracy. The learned values for the damping and stiffness parameters are and . This corresponds to around and relative errors in the estimated values for and , respectively.
3.3 Inferring Lift and Drag Forces from Flow Visualizations
Let us continue with the case where we do not have access to direct observations of the lift force . This time rather than using data on the velocity field, we use measurements of the concentration of a passive scalar (e.g., dye or smoke) injected into the system. In the following, we are going to employ such data to reconstruct the velocity field, the pressure, and consequently the drag and lift forces. A representative snapshot of the data on the concentration of the passive scalar is depicted in the top left panel of figure 8. The neural networks’ architectures used here consist of 10 layers with 32 neurons per each hidden layer. A summary of our results is presented in figure 8. The proposed framework is capable of accurately (of the order of ) reconstructing the concentration. However, a truly intriguing result stems from the network’s ability to provide accurate predictions of the entire velocity vector field as well as the pressure, in the absence of sufficient training data on the pressure and the velocity field themselves (see figure 9). A visual comparison against the exact quantities is presented in figure 8 for a representative snapshot of the velocity field and the pressure. This result of inferring multiple hidden quantities of interest from auxiliary measurements by leveraging the underlying physics is a great example of the enhanced capabilities that physics-informed deep learning has to offer, and highlights its potential in solving high-dimensional data-assimilation and inverse problems (see e.g., raissi2018hiddenfluid).
Following the same procedure as in the previous example, the trained neural networks representing the velocity field and the pressure can be used to compute the drag and lift forces by employing equations (7) and (8), respectively. The resulting drag and lift forces are compared to the exact ones in figure 10. In the following, we are going to use the computed lift force to generate the required training data on and estimate the structural parameters and by minimizing the loss function (3). Upon training, the proposed framework is capable of identifying the correct values for the structural parameters and with surprising accuracy. The learned values for the damping and stiffness parameters are and . This corresponds to around and relative errors in the estimated values for and , respectively.
4 Discussion and Concluding Remarks
We have considered the classical coupled problem of a freely vibrating cylinder due to lift forces and demonstrated how physics informed deep learning can be used to infer quantities of interest from scattered data in space-time. In the first VIV learning problem, we inferred the pressure field and structural parameters, and hence the lift and drag on the vibrating cylinder using velocity and displacement data in time-space. In the second VIV learning problem, we inferred the velocity and pressure fields as well as the structural parameters given data on a passive scalar in space-time. The framework we propose here represents a paradigm shift in fluid mechanics simulation as it uses the governing equations as regularization mechanisms in the loss function of the corresponding minimization problem. It is particularly effective for multi-physics problems as the coupling between fields can be readily accomplished by sharing parameters among the multiple neural networks – here a neural network outputting 4 variables for the first problem and 5 variables for the second one – and for more general coupled problems by also including coupled terms in the loss function. There are many questions that this new type of modeling raises, both theoretical and practical, e.g. efficiency, solution uniqueness, accuracy, etc. We have considered such questions here in the present context as well as in our previous work in the context of physics-informed learning machines but admittedly at the present time it is not possible to rigorously answer such questions. We hope, however, that our present work will ignite interest in physics-informed deep learning that can be used effectively for many different fields of multi-physics fluid mechanics.
The results presented in the current work are among those cases where a pure machine learning algorithm or a mere scientific computing approach simply cannot reproduce. A pure machine learning strategy has no sense of the physics of the problem to begin with, and a mere scientific computing approach relies heavily on careful specification of the geometry as well as initial and boundary conditions. Assuming the geometry to be known, to arrive at similar results as the ones presented in the current work, one needs to solve significantly more expensive optimization problems, using conventional computational methods (e.g., finite differences, finite elements, finite volumes, spectral methods, and etc.). The corresponding optimization problems involve some form of “parametrized” initial and boundary conditions, appropriate loss functions, and multiple runs of the conventional computational solvers. In this setting, one could easily end up with very high-dimensional optimization problems that require either backpropagating through the computational solverschen2018neural or “Bayesian” optimization techniques shahriari2016taking for the surrogate models (e.g., Gaussian processes). If the geometry is further assumed to be unknown (as is the case in this work), then its parametrization requires grid regeneration, which makes the approach almost impractical.
However, it must be mentioned that we are avoiding the regimes where the Navier-Stokes equations become chaotic and turbulent (e.g., as the Reynolds number increases). In fact, it should not be difficult for a plain vanilla neural network to approximate the types of complicated functions that naturally appear in turbulence. However, as we compute the derivatives required in the computation of the physics informed neural networks (see figures 1, 2, and 3), minimizing the loss functions might become a challenge raissi2018deep, where the optimizer may fail to converge to the right values for the parameters of the neural networks. It might be the case that the resulting optimization problem inherits the complicated nature of the turbulent Navier-Stokes equations. Hence, inference of turbulent velocity and pressure fields should be considered in future extensions of this line of research. Moreover, in this work we have been operating under the assumption of Newtonian and incompressible fluid flow governed by the Navier-Stokes equations. However, the proposed algorithm can also be used when the underlying physics is non-Newtonian, compressible, or partially known. This, in fact, is one of the advantages of our algorithm in which other unknown parameters such as the Reynolds and Péclet numbers can be inferred in addition to the velocity and pressure fields.
This work received support by the DARPA EQUiPS grant N66001-15-2-4055 and the AFOSR grant FA9550-17-1-0013. All data and codes used in this manuscript will be publicly available on GitHub at https://github.com/maziarraiss/DeepVIV.