1 Introduction
The recent years have witnessed great advances for applications of deep learning to fluid mechanics [25, 26, 9, 12, 16, 20]
. Owing to the significance in both fundamental research and industrial practice, deep learning methods to infer solutions of NavierStokes equations have attracted considerable interest. For many approaches, convolutional neural networks (CNNs) provide an important building block, e.g., for lightweight approximations of steady flows
[14]. Aiming for the goal of accurate predictions of flow solutions, stateoftheart deep learning methods and architectures have been developed for the inference of Reynoldsaveraged NavierStokes solutions as well as steady laminar flow fields [39, 3, 10]. More recently, trained deep neural network models that infer flow fields have also been used as surrogate models to optimise shapes [4].Although progress has been made, there are notable challenges for deep learning to achieve the high accuracies that realworld engineering applications demand. Most importantly, such applications require flexible domain discretizations that adapt to the embedded geometries, and are able to resolve fine features in flow fields, such as thin boundary layers at high Reynolds numbers and shock waves in compressible flows. Specifically, in most of the previous studies, geometric information is encoded with signed distance function [1, 4] or binary masks [39, 22]
, and flow field data has to be interpolated onto an auxiliary Cartesian grid for the neural networks. As these grids are uniformly spaced, the resolution of the auxiliary grid has to be redundantly fine to prevent undesirable blurring of important flow structures. Other recent works resort to fully connected networks to resolve airfoil geometries of transonic flows
[38].A potential solution is the use of graph neural networks (GNNs) which directly encode and employ information about the discretization, i.e. mesh points and edges [7, 31]. However, three reasons lead us to seek other ways. First, a GNN is essentially unstructured. As we are particularly interested in aerodynamics, we realise that structured meshes are widely used not only in industry, e.g. aerospace engineering, but also in fundamental research areas such as the direct numerical simulation based on highorder finitedifference method [36, 17]. Second, many current deep learning techniques are specifically designed for structuredgridbased convolutional neural networks [16, 41]. And third, GNNs still face many open challenges such as questions regarding the theory of graph representations, demanding hardware requirements, nontrivial parallelism and distributed solving [24]
. Taking these considerations into account, we instead develop an inference method that uses information about the discretization of partial differential equations (PDEs) for a given structured curvilinear mesh that adapts to geometries.
With a finitevolume or finitedifference method based on a smooth, structured grid, it is possible to apply a local univalent transformation to the physical space coordinates (x, y) to obtain the curvilinear coordinates (, ) with curved coordinate lines, i.e. structured bodyfitted grids [13, 21, 2, 42, 36, 17]
. The transformation matrix uniquely maps the physical space to the computational space. As the latter represents a Cartesian coordinate system, we use it as a canonical representation of the desired solutions with different discretizations. Due to its Cartesian nature, this space makes the use of efficient, regular convolutional architectures for the neural networks possible. Early theoretical work has addressed the equivalence of image representations in networks
[23] and showed promising applications with warped images in polar and spherical coordinate systems [11, 6]. However, studies on coordinate transformations with arbitrary grids and the associated deep neural network architectures are relatively scarce. In this regard, the present work can be seen as a step towards a generalized handling of geometric encodings for neural networks that deal with arbitrary warped images.We recall that the combination of structuredgrid solvers and deep learning has naturally led to architectures related to curvilinear coordinates [28, 40]. However, in these studies the coordinate transformation is not fully incorporated into the neural network, and the resulting deep neural networks are designed only for providing an approximate initial guess to accelerate the main computational fluid dynamics (CFD) solver. By contrast, in the present work, the coordinate transformation matrix is considered as an important representation of geometric information that is used for the inference of solutions on different discretizations.
Unlike other deep learning frame works that rely on the coupling with CFD solvers [7, 28, 40], the present method is a pure convolutional neural network architecture trained in a fully supervised manner. The network directly makes highfidelity predictions about flow fields, and hence works as a flexible surrogate model that produces a full flow field [4]. Once trained, it can be used in different optimisation tasks with multiple objective and can be seamless integrated with deep learning algorithms, such as differentiable physicsbased methods [16].
Moreover, compared to others [39, 1], the present deep neural network model accurately predicts complex flow behaviour at high Reynolds numbers and transonic speeds, such as shock wave/boundary layer interaction, and quantitative distributions like pressure coefficient, skin friction coefficient as well as wake total pressure profiles downstream of aerofoils. This is of particular importance in practical engineering designs for aerofoils and compressor/turbine blades [27]. To the best of our knowledge, no previous studies exist that achieve a comparable prediction accuracy using a pure neural network model.
The purpose of the present work is to demonstrate the capabilities of deep learning techniques for accurate predictions of compressible turbulent flow, and for achieving an improved understanding of the fundamental phenomena involved in such flows. The methods developed in the present study open up an avenue for the use of deep learning techniques in the inference of precise CFD solutions using structured bodyfitted grids. This paper is organised as follows. The mathematical formulation and numerical method are briefly presented in §2. The detailed experiments and results are then given in §3, the performance of deep neural network models is reviewed in §4 and concluding remarks are in §5.
2 Methodology
2.1 Inference task for deep neural networks
The goal of the present work is to study the inference of Reynoldsaveraged NavierStokes (RANS) solutions for the compressible flow over aerofoils in two dimensions with a deep neural network (DNN). The converged solution of a RANS simulation is physically an ensemble average, and it represents a deterministic solution. Thus, we define the inference task as , where the inputs and denote the given freestream conditions and the geometric information, respectively. The latter encodes information about the discretization of flow domain and the embedded airfoil. The output denotes the solution of a RANS simulation, which is expressed by flow field variables in either primitive or conservative forms (e.g. density, momentum and total energy). We approximate the output of the true function as closely as possible with a deep neural network representation
based on the degrees of freedom
such that .In most existing studies, a constant, uniformly distributed auxiliary grid is used to infer RANS solutions with deep learning methods
[39, 1]. The auxiliary grid itself takes the form of a Cartesian grid, and hence the geometric input takes the form of a binary mask or a signed distance function to encode the embedded aerofoil. However, an inherent drawback is that the discretization of the physical domain for inference can not change, and hence the representation of is constant at training time. As a consequence the discretization has to be redundantly fine to avoid the undesirable blurring of important flow structures.We propose an alternative approach where the domain discretization varies and adapts to each input . Information about this discretization is encoded in . We design the geometric encoding based on the following considerations: First, we leverage the existing domain expertise in CFD to obtain a suitable bodyfitted structured mesh. These offer an optimal discretization to resolve important features in the flow field. Second, assuming that the structured mesh is smooth, we map the variables from the physical space to a canonical space, often also called “computational space”, in which the neural networks operate. Hence, no matter how the domain discretization in the form of the CFD mesh changes, the canonical space for the DNN remains the same. However, we provide the DNN with information about the discretization in physical space in terms of a geometric encoding, such that the neural network can infer the solution at the correct location. Since we investigate a varying aerofoil profile , the corresponding discretization of flow domain (i.e. mesh) for a given can be expressed as . Here, function can be seen as the practice of mesh generation subjected to a given aerofoil . To find a good representation of for geometric encoding in the canonical space, we introduce three methods in §2.2. We discuss details of the deep neural network architecture in §2.5.
2.2 Encoding geometric information
For a given structured mesh, we perform a univalent transformation from the coordinates (x, y) in the physical space along indices i and j to the curvilinear coordinates (, ) in the computational space [13], as follows:
(1) 
and
(2) 
Here, the definition of and is and with given integers from 1 to and from 1 to , which can be extended by analytic continuation into a realnumber domain . Then, we define the transformation matrix to be:
(3) 
This approach is fully in line with the discretization using structured grids, among which cgrid and ogrid topology meshes are most commonly used in numerical simulations of flows over aerofoils as shown in figure 1. Through the coordinate transformation, variables in the physical space can be uniquely mapped into the computational space and back. Hence, we define a unit cube of curvilinear coordinates as the canonical space in which the neural networks operate. Figure 2(a) shows the xcomponent velocity field over the aerofoil “fx84w097” in the physical space. In terms of the curvilinear coordinates (, ), which align with the array indices (i, j), the field becomes the image shown in figure 2(b). Similarly, the x and y coordinates of the mesh in the physical space can be visualised in the curvilinear coordinate system (, ) as shown in figures 3(a) and 3(b). The encoding method of directly using images of mesh coordinates and as geometric information in figure 3 is denoted as “Method A”, i.e.
Considering the fact that the coordinate transformation matrix is univalent for a given mesh, the four components of transformation matrix as well as the reference points can be used for geometric encoding, i.e. , , , , , . Taking the mesh of aerofoil “fx84w097” as an example, the full encoded geometric information is shown in figure 4. We denote this one as “Method B”, where
Here, the reference coordinates (, ) at or contain the full coordinates of the corresponding aerofoil (the shape of the and arrays is ). To match the network architecture, we extend the two variables to a size of to match the other inputs, which means the content of the resulting arrays is constant along the direction.
It is beneficial to consider the nondimensional form of transformation matrix in order to find more meaningful pattern, so we define
(4) 
Therefore, the geometric information can be determined by eight variables: , , , , , , , , , where is the transformation Jacobian defined to be
As we are dealing with a twodimensional space, the geometrical interpretation of those variables are: the area of the grid cell, x component of unit normal to i edge, y component of unit normal to i edge, the length of i edge, x component of unit normal to j edge, y component of unit normal to j edge, the length of j edge, and reference coordinates on the aerofoil surface, respectively. In fact, the use of nondimensional metrics is a common practice in the implementation of finite volume approach. For more details on the geometrical interpretation, we refer to Krist et al. [21, p. 263], Blazek [2, p. 401] and Wang [42, p. 25]. The eight variables in curvilinear coordinate system are shown in figure 5 taking the mesh of aerofoil “fx84w097” for example. We denote this method as “Method C”, where
The three encoding methods are summarised in table 1. For all the three methods, the encoded free stream conditions are Mach number, Reynolds number and the angle of attack (AoA), i.e.
Note that we extend each variable to a size of to match the other inputs in the neural network architecture, which means that each of the three channels in the tensor is constant.
Method  Geometric information  Inflow condition  No. of channels 

A  ,  , ,  
B  , , , , ,  , ,  
C  , , , , , , , ,  , , 
2.3 Computational details and data generation
The open source structuredgrid code CFL3D version 6.7 is used to obtain the ground truth data
[34, 35]. The twodimensional Reynoldsaveraged compressible NavierStokes equations (RANS) in generalized coordinates are numerically solved by the finitevolume method. The convective terms are discretized with thirdorder upwindbiased scheme and viscous terms with secondorder differencing [32]. For lowspeed steady cases, lowMach number preconditioning is employed [43]. SpalartAllmaras oneequation model is used for turbulence closure [37].In the present work, we particularly focus on aerofoils with sharp trailing edges, so we choose 990 aerofoil shapes from the UIUC database. The open source code Construct2D version 2.1.4 is used to generate a 2D structured curvilinear grid. The leading edge of the aerofoil is taken as the origin of the coordinate system and the chord length is 1.0. We develop a template with the same grid topology to mesh the domain around aerofoils. For a cgrid topology mesh, the resolution is kept the same, i.e. 128 grid cells in the normalwise direction, and 25 grid points in the wake, and 79 grid points around the aerofoil surface. In the case of an ogrid topology, there are 128 grid cells in the normalwise direction and 128 cells in the circumferential direction around the aerofoil. Typical cgrid and ogrid meshes with cells are given in figures 1(a) and 1(b). From the viewpoint of practical engineering, the mesh resolution used in the present study is meaningful at the preliminary design stage. Note that our main focus here is to showcase the trained DNN model is capable of providing fast and accurate prediction by comparing with the ground truth results, and the proposed methodology can be easily extended to highresolution cases.
We generate groundtruth flow fields for training, validation and test sets in the following four cases which are summarised in table 2. The first case is compressible flow over aerofoils, i.e. 990 different aerofoils, at a fixed free stream condition , . The second case is low Mach number (i.e. ) viscous flow over aerofoils with the variation of angles of attack and Reynolds numbers. The third case is compressible inviscid flow over the aerofoil “naca0012”. The fourth one is transonic viscous flow over aerofoils with the variation of angles of attack, Mach numbers and Reynolds numbers.
The typical data generation routine is as follows:
(1) Freestream conditions and the aerofoil shape are randomly selected from the range of interest as shown in table 2.
(2) is technically realised by mesh generation. In cases 1, 2 and 4, once an aerofoil is chosen, a cgrid topology mesh is automatically generated based on the previously mentioned template. Note that in the current procedure, one aerofoil only has one unique mesh discretization, that means, we eventually have 990 cgrid topology meshes for 990 aerofoils. In case 3, as we only consider the aerofoil “naca0012”, an ogrid topology mesh is generated.
(3) CFL3D code is run with the generated mesh. Once converged, the flow field (i.e. density, velocity components in x and y directions and the speed of sound), the corresponding geometric information and free stream conditions are saved as a data file for training, validation and test. All of the saved variables are in the canonical space or computational space (, ). Note also that the aerofoil shape information has been included or fully represented by the grid line or .
As shown in figure 1(a), when using a cgrid topology mesh, an inflow/outflow boundary based on onedimensional Riemann invariant is imposed at about 20c away from the aerofoil in the (x, y) plane.
The grid stretching is employed to provide higher resolution near the surface and in the wake region and the minimal wallnormal grid spacing is to ensure for the Reynolds numbers within the range from to . In viscous cases, a noslip adiabatic wall boundary condition is applied on the aerofoil surface. The ogrid topology mesh in figure 1(b) is only used in the inviscid cases, in which the slip wall boundary condition is imposed on the surface. The same inflow/outflow boundary is imposed at about 45c away from the surface.
Unless noted otherwise, we use a 80% to 20% split for training and validation datasets. We found validation sets of several hundred samples to yield stable estimates, and hence use an upper limit of 400 as the maximal size of the validation dataset. The validation set allows for an unbiased evaluation of the quality of the trained model during training, for example, to detect overfitting. In addition, as learning rate decay is used, the variance of the learning iterations gradually decreases, which lets the training process finetune the final state of the model. To later on evaluate the capabilities of the trained models with respect to generalization, we use an additional set of 20 airfoil shapes that were not used for training, to generate a test dataset with 20 samples (using the same range of Mach, Reynolds numbers and angles of attack as described in table
2).Case  Training/vali./test  Freestream () 
1  780/190/20  , , 
2  9300/400/20  , , 
3  168/0/63  , 
,  
4  9300/400/20  , , 
2.4 Preprocessing
As neural networks inherently rely on specific ranges of input values, preprocessing of the data is crucial for obtaining a high inference accuracy. Thus, all input channels and target flow field data in the training dataset are processed in the curvilinear coordinate system with uniform grid spacing as mentioned above, and are normalised to the range from 0 to 1 in order to minimise the errors from limited precision in the training phase. To do so, we first find the maximum and minimum values for each variable in the entire training dataset, e.g. and , and , and . Then we get the final normalised form:
where, is a small positive value, e.g. . As most of the variations of computational grid occur in vicinity of aerofoils, we manually set and .
2.5 Neural network architecture
The deep neural network model is based on a Unet architecture [33], a convolutional network originally used for the fast and precise segmentation of images, and later used for the inference of flow fields and the development of DNN models [39, 4].
As inputs for the learning task , we consider the inflow boundary conditions, i.e. , , , and the geometric information in the curvilinear coordinate system for . This is illustrated in figure 6. Hence, the input becomes , where N is the number of the input channels (see table 1). In the encoding part, 7 convolutional blocks are used to transform the input (i.e. ) into a single data point with 512 features. The decoder part of the network is designed symmetrically with another 7 layers in order to reconstruct the outputs with the desired dimension, i.e. , corresponding to the flow field variables (i.e. density, velocity components in x and y directions and the speed of sound) on the
grid. Leaky ReLU activation functions with a slope of 0.2 is used in the encoding layers, and regular ReLU activations in the decoding layers.
In order to assess the performance of the DNN models, we have tested three different models with varying weight counts in appendix A.1, and found the network with a weight count of to yield a good performance across all our tests.
2.6 Training and test accuracy
The neural network is trained with the Adam optimiser in PyTorch
[19, 30]. An difference is used for the loss calculation, where denotes the ground truth solutions from the training data set andthe DNN output. For most of the cases, the training runs converge after 1200 epochs with a learning rate
and a batch size of 5 (unless otherwise mentioned). The learning rate was set to decrease to 10% of its initial value over the course of the second half of the training iterations, which helps to finetune the final state of the model and decrease the variance in the performance.3 Results and discussion
3.1 A comparison of the three methods
Table 3 summarises the training, validation and test losses for the models trained with methods A, B, and C in the four cases. The smallest validation and test losses are highlighted in bold. In case 1, which is the dataset with a fixed freestream condition, methods B and C show very close accuracy in terms of the validation and test losses. In case 2 which at the quasiincompressible regime, method C outperforms others with the smallest validation and test losses. Case 4 is at transonic viscous regime, the prediction by the model using method C is the most accurate. On a separate note, the dataset for case 3 is generated following the settings of the “interpolation” case proposed by [7]. Looking at the loss of the predictions, the three methods are very close to each other.
It should be mentioned that we also accessed the effects on performance with different scales of networks and amounts of flow samples in the dataset (see appendices A.1 and A.2), and results show the trends are consistent with table 3. Therefore, we will focus on the largescale neural network models (i.e. weights) trained with method C for the detailed discussion in §3.
Case  Method  Training ()  Vali. ()  Test () 

A  
1  B  
C  
A  
2  B  
C  
A  
3  B  
C  
A  
4  B  
C 
3.2 Case 1: a fixed freestream condition
3.2.1 Overview
As shown in table 1, for case 1 there are 990 flow fields that are obtained with cgrid topology grids at a fixed flow condition, i.e. , , and . Each solution uses a different discretization of the physical space. In the training phase, 780 flow fields are used as training set and 190 for validation set. After this, additional 20 aerofoil shapes that did not appear for training are used to generate a test dataset with 20 samples at the same freestream condition. Case 1 is designed to isolate and evaluate the effect of changing geometric information. In most instances, the boundary layers on both aerofoil surfaces remain attached at the current freestream condition. The average loss for the test set using method C is , indicating a high accuracy of the trained DNN model.
3.2.2 Flow field prediction
One of the advantages of the proposed method is that the DNN model predicts a full flow field. Taking the aerofoil “ah94156” as an example, figure 7 shows the density, x and y components of velocity as well as speed of sound predicted by DNN (top row) and the CFD solver (middle row). The relative errors at each individual pixel are shown in the bottom row, which are , , and . There is no visible difference in predictions by DNN and CFL3D.
We choose four representative cases from 20 test results to investigate details: “ah94156”, “e473”, “e59” and “goe97k”, corresponding to thick, symmetrical, thin, flatbottom aerofoils, respectively.
Looking at figure 8, the distributions of pressure coefficient obtained from DNN predictions agree well with that from CFL3D, reflecting a typical behaviour of attached boundary layers. In particular, in figures 8(c) and 8(d), the mildmannered pressure distribution on the lower surface (or pressure side) is due to the flat design of aerofoils. As “e59” is a thin aerofoil with a sharper leading edge, the suction peak occurs in very close proximity to the leading edge as shown in figure 8(c).
The comparisons of skin friction coefficient distributions in figure 9 are also encouraging. We define the positive shear in the clockwise direction. The distribution follows the behaviour of a fully turbulent flow over an aerofoil, starting from a high shear stress at the leading edge then decaying gradually. Some slight volatility in the vicinity of the leading edge on the suction side observed in figures 9(c) and 9(d) are caused by the changes in pressure gradient.
Figure 10 shows the total pressure profiles are measured at 20% of chord length downstream of trailing edges. As the flows are mostly attached, the wake profiles are relatively homogeneous in the outer region but exhibit sharp deficit peaks.
These results highlight that the DNN model captures fine details of the flow fields. It also confirms that the geometric information has been encoded such that the DNN can employ it to infer the desired solutions. Hence, this approach provides a good basis for further studies with more complex freestream conditions.
3.3 Case 2: low Mach number viscous flow
3.3.1 Overview
Next we target a benchmark case for the inference of incompressible RANS results [39]: RANS solutions are randomly distributed in a range of Reynolds numbers Re million, and angles of attack in the range of [22.5, 22.5]. In the present setup, we run a simulation by randomly choosing one of the 970 aerofoils with domain discretizations in the training data set, and randomly choosing the freestream condition from the above described range. We use an additional set of 20 aerofoil shapes that were not used for training, to generate a test dataset with 20 samples using the same range of Reynolds numbers and angles of attack. To make a reasonable comparison, we set the freestream Mach number to 0.1 in the current compressible solver, and employ a lowMach number preconditioning [43] to improve the convergence. The mean test loss () for the test set is . The mean relative error for the test set is defined to be , where T is the number of samples in the test set (T=20 here), M is the number of pixels of images (M= here), f the function under consideration (here, density, velocities and pressure), the approximation of the DNN. The best prediction by the trained model in [39] achieves relative errors of 2.15% for the x velocity, 2.6% for the y velocity and 14.76% for pressure values. In the present work, the corresponding relative errors are 0.0438% for density, 0.137% for the x velocity and 0.0712% for the y velocity, 0.0933% for the speed of sound and 0.178% for pressure. Hence, improvements of more than an order of magnitude compared to the results by Thuerey et al. [39] have been achieved, with almost two orders of magnitude for the pressure fields.
3.3.2 Flow field prediction
We select four representative cases from the test dataset for detailed discussions as shown in table 4. Aerofoil “e342” experiences a high positive angle of attack. In the case of aerofoil “goe07k”, the angle of attack is slightly negative, and aerofoil “mue139” are at a relatively high Reynolds number of the described range. In all of the three cases, the predictions exhibit small losses, i.e. the order of magnitude of . Note that even in the worst case in terms of loss, i.e. aerofoil “goe398” at a high negative angle of attack of , the DNN predicts an accurate flow field with . Figure 11 shows the density, x and y components of velocity as well as speed of sound predicted by DNN (top row) and the CFD solver (middle row). The relative errors at each individual pixel are shown in the bottom row, which are , , and . The predictions by DNN and CFD solver are not apparent, and the only visible difference resides in u in the separated region, which is shown in the relative error contour.
As the pressure coefficient and skin friction coefficient on the aerofoil surface are critical for engineering designs, we evaluate our DNNs with respect to these quantities. Figure 12 shows pressure coefficients as well as the pressure field around the aerofoil predicted by the DNN. To keep consistency with the upper and lower surfaces of the aerofoils, the yaxis is inverted for positive angles of attack. The distributions predicted by DNN agree well with the reference results. Especially, the strong suction peaks are well captured in the DNN’s predictions as shown in figures 12(a) and 12(c).
Skin friction coefficients along the aerofoils of the four test cases are shown in figure 13 and a reasonable agreement has been achieved with the results predicted by the solver CFL3D. We define the positive shear in the clockwise direction. In figure 13(a), the skin friction coefficient on the upper surface of aerofoil “e342” at shows a peak in at the leading edge and slowly declined to a level of zero indicate at due to the occurrence of separation as evidenced in the accompanying figure xcomponent velocity field. In the cases of aerofoils “goe07k” and “mue139” in figures 13(b) and 13(d), the skin friction peaks occur near the leading edges and the distributions don’t change signs, indicating boundary layers on both upper and lower surfaces remain attached, which is consistent with the corresponding xcomponent velocity fields. Figure 13(c) shows a typical skin friction distribution with a largescale separation at leading edge, which is reflected in the abrupt change at from a negative peak to a positive value.
Figure 14 shows the total pressure profiles measured at 20% of the chord length downstream of the trailing edges. The profiles are affected by the wake mixing as well as the separated flows. As shown in figure 14(a), due to the separation at a high angle of attack, the wake deficit is bigger than those with attached flows shown in figures 14(b) and 14(d). In the case of a very high negative angle of attack case for “goe398” with , the wake profile is clearly divided into a high and a low speed regions. These selected cases illustrate that the essential behaviour of low Mach number pressure fields are successfully captured by the DNN model.
Aerofoil  Test ()  

e342  0.1  
goe07k  0.1  
goe398  0.1  
mue139  0.1 
3.4 Case 3: compressible inviscid flow
3.4.1 Overview
In this section, we consider compressible inviscid flows over a given aerofoil, i.e. “naca0012”, at a set of freestream conditions. This case is originally proposed by de Avila BelbutePeres et al. [7] as a showcase of their deep learning method based on the coupling of an adjoint CFD solver and the GNN.
To make reasonable comparisons, we generate the flow field data with similar mesh resolution. In particular, the ogrid topology mesh for the aerofoil naca0012 is considered as shown in figure 1(b). The freestream flow conditions are given in table 1. As can be seen from the table, there are much less samples in the training and test sets, which poses additional challenges on the learning and inference task. Thus, it represents a good case to show that the proposed method is capable of providing high fidelity results even when the amount of training data is relatively small.
We train the models for more epochs with a batch size of 7. A learning rate is used for 8000 epochs, then it gradually decays to for 4000 epochs, and exponentially to for another 2000 epochs. Although we train the DNN model with an loss of the flow variables, i.e. density, xvelocity, yvelocity and speed of sound, we now calculate a root mean square error (RMSE) to assess predictions following de Avila BelbutePeres et al. [7]:
In previous work [7], the GNN achieves a RMSE of and the hybrid CFDGNN method gives . By contrast, the present study which is purely based on the DNN predicts flow field accurately with a very small RMSE, i.e. with method C and with method B.
3.4.2 Flow field prediction
The high accuracy of the predictions of the DNN is reaffirmed by the distribution pressure coefficients as shown in figures 15(ac). The worst prediction in terms of loss among the present test cases happens at and , as shown in figure 15(d). Despite the strong gradient region caused by the shock wave being suboptimally resolved, the overall distribution of pressure coefficient is still predicted reasonably well, especially with respect to the shock location on the suction side. In addition, the stagnation point agrees well with the one computed by the CFD solver CFL3D.
3.5 Case 4: transonic viscous flow
Aerofoil  Test ()  

e221  0.71  
e473  0.55  
fx84w097  0.78  
mue139 
3.5.1 Overview
In this section, we consider transonic turbulent flows over aerofoils. The inference of RANS solutions randomly distribute in a range of Mach numbers M , Reynolds numbers Re million, and angles of attack in the range of [, ]. The range of freestream conditions described above is chosen according to the early study of shock wave/boundary layer interaction on aerofoils by Holder et al. [15].
As for case 2, we randomly choose aerofoil shape and freestream condition from the training range, and use 20 additional aerofoils as test set. Note that the flow field is intrinsically unsteady when reaching transonic “buffet” boundary [8, 5]. In order to obtain meaningful ensemble averaged flow field, we run 24000 iterations with Courant number 1.0 for each case and we average the intermediate results over the last 8000 iterations. For a more fundamental discussion on the nonuniqueness at transonic regimes, we refer to [18] and [29].
In the present work, the relative errors are 0.255% for density, 0.289% for the x velocity and 0.0996% for the y velocity, 0.0621% for the speed of sound and 0.269% for pressure. The test loss () is .
3.5.2 Flow field prediction
We select four representative cases from the test dataset for detailed discussions as shown in table 5. The “worst” case in terms of loss among the test results is “e221” at , and . “e473” is a thick aerofoil and at the given condition it turns out to be a shockwave free case. In the case of “fx84w097” with the listed freestream condition, the shock wave is very strong. On the other hand, “mue139” is chosen because of its negative angle of attack.
The flow field of aerofoil “e221” at the test condition is characterised by a strong standing shock wave and shockinduced separation. Figure 16 shows the density, x and y components of velocity as well as speed of sound predicted by DNN (top row) and the CFD solver (middle row). The shock wave is well reproduced by the DNN model and the flow fields in smooth regions are visually indistinguishable between the DNN and CFD predictions. Despite that, noticeable relative errors are found in the vicinity of shock waves and separated regions induced by the strong adverse pressure gradient, the DNN model still provides reliable and accurate predictions in the smooth flow region.
Figure 17 shows that the DNN model’s predictions of pressure coefficients on the aerofoils compare favourably with the ground truth data. Key features are well captured by the DNN model, such as strong adverse pressure gradient due to the shocks on the upper surfaces (see figures 17(a), 17(c) and 17(d)), and the “crossover” of due to the negative angle of attack and the occurrence of a small shock wave near the leading edge shown in figure 17(d). The shockfree attached flow case also agrees well with the CFD result as shown in figure 17(b).
Figure 18 gives a comparison of skin friction coefficients obtained by DNN and ground truth data. The distributions in figures 18(a) and 18(c) are typically characterised by leading edge peaks subjected to transonic fully turbulent inflow and abrupt drops to negative values at around due to the shock induced separation. Figure 18(b) exhibits smoother distribution as the flow field is shock free under a moderate Mach number. In figure 18(d), there are two sudden changes: the one near the leading edge is due to the small shock wave on the lower surface and the other occurs at around is caused by the strong standing shock wave one the upper surface. It is clear from this comparison that the DNN predicts the characteristics of the boundary layer accurately.
In the wake region, total pressure profiles are measured vertically at given in figure 19. Some fine details caused by the complex flow behaviour such as shockinduced separation on the upper surfaces are well reproduced, such as in figure 19(a) and in figure 19(c). As aerofoil “e473” at the test condition is a shockfree and separationfree case, the wake profile is characterised by a smaller deficit and homogeneous distributions outside the wake. Figures 19(a), 19(b) and 19(d) indicate that the upper parts of the wake profiles exhibit in both DNN and CFD solutions likely due to the existence of shock waves, which lead to an increment in entropy and hence a total pressure loss. This phenomenon is well captured by the DNN model, which further demonstrates the high reliability of the DNN model. Therefore, the DNN model is clearly capable of reproducing physical details in the flow field, which lays the foundation for future applications such as lossbased optimisation for turbine blades subjected to complex inflows [27].
4 Performance
Solver  Wall time  Platform 

CFL3D 8000 iterations  296.96 s  CPU only, 1 core 
CFL3D 24000 iterations  886.18 s  CPU only, 1 core 
MethodA batch 1  ms  CPU, 1 core & GPU 
MethodB batch 1  ms  CPU, 1 core & GPU 
MethodC batch 1  ms  CPU, 1 core & GPU 
MethodC batch 5  2.610 ms  CPU, 1 core & GPU 
MethodC batch 20  2.870 ms  CPU, 1 core & GPU 
MethodC batch 40  3.240 ms  CPU, 1 core & GPU 
.
The performance of trained DNN models is one of the central factors motivating their use. We evaluate our models using a regular workstation with 12 cores, i.e. Intel^{®} Xeon^{®} W2133 CPU @ 3.60GHz, with an NVidia GeForce RTX 2060 GPU. Due to the strongly differing implementations, we compare the different solvers in terms of elapsed wall clock time, which is a pure computation time and is averaged over multiple runs without taking the startup and initialisation overheads into account.
As listed in table 6, a typical RANS simulation with CFL3D solver at the current grid resolution at requires 8000 iterations with a wall clock time of 296.96 seconds. As previously mentioned in §3, some cases require an averaging of intermediate results to improve the convergence which takes 24000 iterations and 886.18 seconds, respectively. Likewise, we evaluate the elapsed time for the DNN prediction, taking about 2.29 ms with any of three methods. It worth mentioning that the runtime per solution can be reduced significantly when evaluating multiple solutions at once. As an example with method C, for a batch size of 5, the evaluation time rises only slightly to 2.610 ms, and then to 3.240 ms for a batch size of 40. Therefore, relative to the CFD solver CFL3D, a speedup factor between 130000 and 388000 is achieved. Even when considering a factor of ca. 10 in terms of GPU advantage due to an improved onchip memory bandwidth, these measurements indicate the significant reductions in terms of run time that can potentially be achieved by employing trained deep neural networks. The time to train the DNN models varies with neural network size and the amount of training data. Taking Dataset3880 (see appendix A.2) as an example leads to training times of ca. 1036 minutes for the largescale ranged model. Given the potentially large onetime cost for training a model, learned approaches bear particular promise in settings where similar optimisation problems are solved repeatedly.
5 Concluding remarks
In this paper, we presented a method to train a deep neural network (DNN) model that learns the Reynoldsaveraged NavierStokes solutions based on widelyused structured grids for various angles of attack, Reynolds numbers and Mach numbers. Our approach yields networks that learn to generate precise flow fields for varying bodyfitted structured grids by providing them with an encoding of the corresponding mapping to a canonical space for the solutions. In the benchmark case of incompressible flow at randomly given angles of attack and Reynolds numbers the DNN model achieves an improvement of more than an order of magnitude compared to previous work. For transonic flow cases, the DNN model accurately predicts complex flow behaviour at high Reynolds numbers, such as shock wave/boundary layer interaction, and quantitative distributions like pressure coefficient, skin friction coefficient as well as wake total pressure profiles downstream of aerofoils. Furthermore the DNN model, being orders of magnitude faster than the conventional CFD solver, yields very significant computational speedups.
The proposed deep learning method offers several advantages. First, the method doesn’t rely on auxiliary fields with information about the embedded aerofoil. Instead, it employs geometric information from structured grids that can be easily combined with common CFD applications to infer solutions with changing domain discretizations. Second, the model provides flexibility as it predicts a full flow field. Once trained, it can, for example, be used in different optimisation tasks. Third, as the model is differentiable, it can be seamlessly integrated into gradientbased optimisation algorithms. Moreover, the neural network architecture is generic, able to deal with changing discretizations, and applicable to boundary value problems of PDEs discretized with structured bodyfitted grids.
On the other hand, a limitation of the proposed method is that the DNN architecture assumes that the different discretizations share the same topology, e.g. a cgrid topology mesh. Nonetheless, the computational meshes of many of practical applications, such as designspace parameter studies, are usually generated with the same template based on best practices and existing knowledge and hence amenable to the approach. In order to cope with different grid topologies, we believe it will be a interesting future direction to add varying boundary conditions and introduce physical inductive biases into the learning process.
Acknowledgements
This work was supported by the ERC Consolidator Grant SpaTe (CoG2019863850).
Appendix A
a.1 Network size
To investigate the impact of the neural network size on the accuracy, several training runs are conducted with four different network sizes for case 1. Figures 20(a) and 20(b) show the training, validation as well as test losses vary with the network size in terms of the number of trainable parameters, i.e. , , and , respectively. Although volatility is still noticeable, the basic tendency is towards high accuracy smaller losses with the increment of trainable parameters. There is no drastic change in loss curves when the number of trainable parameters is higher than .
It worth noting that the DNN model with parameters and using method A cannot give a reasonable prediction due to the high validation and test losses, i.e. 0.02688 and 0.01808, respectively. Note also that the largest scale DNN model with method B shows a high validation loss as shown in figure 20(a), far from improving the prediction accuracy as expected, so we consider that method C outperforms the other two and will use the results obtained by DNN models with method C for detailed discussions.
a.2 Dataset size
To assess the effects of the number of samples in the datasets on the training and validation losses, training runs for cases 2 and 4 are conducted with various amounts of data. The results are listed in table 7
. Note that in this case any smaller dataset is a subset of a larger dataset and all follow the same probability distribution. We found validation sets of several hundred samples to yield stable estimates, and hence use an upper limit of 400 as the maximal size of the validation dataset. The typical number of epochs for training is 1290.
Figures 21(a) and 21(b) show the values of training, validation and test losses for case 2. It can be observed that the models with small amounts of data exhibit larger losses. The behavior stabilizes with larger amounts of data being available for training, especially when the number of samples is greater than 6790.
From figures 22(a) and 22(b) for case 4, it can be also seen that with an increased number of samples, the training, validation and test losses follow an overall declining trend. Due to the intrinsic complex nature of transonic flow in case 4, the DNN model using method A gives a high test loss when the number of samples is 1940, i.e. 0.02853, which can be considered as a failure.
To summarise, the above results in cases 2 and 4 imply that further increasing the amount of data does not yield significant improvements in terms of inference accuracy. Therefore, the DNN model trained with “Dataset9700” is chosen for the detailed study for cases 2 and 4. In most runs, the performance of methods B and C is very close. However, the validation loss curves of method B in figures 21(a) and 22(a) exhibit higher losses when the number of samples increase to 9700. Thus, method C is still considered to outperform the other two.
Name  No. of flowfields  training  validation  test 

Dataset1940  1940  1555  385  20 
Dataset3880  3880  3480  400  20 
Dataset6790  6790  6390  400  20 
Dataset9700  9700  9300  400  20 
References
 [1] (201906) Prediction of aerodynamic flow fields using convolutional neural networks. Computational Mechanics 64 (2), pp. 525–545. External Links: ISSN 14320924, Link, Document Cited by: §1, §1, §2.1.
 [2] (2001) Computational fluid dynamics: principles and applications. Elsevier Science. External Links: ISBN 9780080430096, LCCN 2001023790, Link Cited by: §1, §2.2.
 [3] (2019) Unet architectures for fast prediction of incompressible laminar flows. External Links: 1910.13532 Cited by: §1.
 [4] (2021) Numerical investigation of minimum drag profiles in laminar flow using deep learning surrogates. Journal of Fluid Mechanics 919, pp. A34. External Links: Document Cited by: §1, §1, §1, §2.5.
 [5] (2010) Numerical investigation of the compressible flow past an aerofoil. Journal of Fluid Mechanics 643, pp. 97–126. External Links: Document Cited by: §3.5.1.
 [6] (2018) Spherical CNNs. External Links: 1801.10130 Cited by: §1.

[7]
(202013–18 Jul)
Combining differentiable PDE solvers and graph neural networks for fluid flow prediction.
In
Proceedings of the 37th International Conference on Machine Learning
, H. D. III and A. Singh (Eds.), Proceedings of Machine Learning Research, Vol. 119, pp. 2402–2411. External Links: Link Cited by: §1, §1, §3.1, §3.4.1, §3.4.1, §3.4.1.  [8] (2005) Numerical simulation of transonic buffet over a supercritical airfoil. AIAA Journal 43 (7), pp. 1556–1566. External Links: Document Cited by: §3.5.1.
 [9] (2019) Turbulence modeling in the age of data. Annu. Rev. Fluid Mech. 51 (1), pp. 357–377. External Links: Document, Link, https://doi.org/10.1146/annurevfluid010518040547 Cited by: §1.
 [10] (2021) Stationary flow predictions using convolutional neural networks. In Numerical Mathematics and Advanced Applications ENUMATH 2019, F. J. Vermolen and C. Vuik (Eds.), Cham, pp. 541–549. External Links: ISBN 9783030558741 Cited by: §1.

[11]
(2018)
Polar transformer networks
. External Links: 1709.01889 Cited by: §1.  [12] (2019) Superresolution reconstruction of turbulent flows with machine learning. Journal of Fluid Mechanics 870, pp. 106–120. External Links: Document Cited by: §1.
 [13] (1973) Construction of curvilinear coordinate systems and applications to mesh generation. International Journal for Numerical Methods in Engineering 7 (4), pp. 461–477. External Links: Document, Link, https://onlinelibrary.wiley.com/doi/pdf/10.1002/nme.1620070405 Cited by: §1, §2.2.
 [14] (2016) Convolutional neural networks for steady flow approximation. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Cited by: §1.
 [15] (1955) The interaction between shock waves and boundary layers. Aeronautical Research Council Technical Report C. P. No. 180. Cited by: §3.5.1.
 [16] (2020) Learning to control PDEs with differentiable physics. In International Conference on Learning Representations, External Links: Link Cited by: §1, §1, §1.
 [17] (2017) OpenSBLI: a framework for the automated derivation and parallel execution of finite difference solvers on a range of computer architectures. Journal of Computational Science 18, pp. 12–23. External Links: ISSN 18777503, Document, Link Cited by: §1, §1.
 [18] (1991) Airfoils admitting nonunique solutions of the Euler equations. In AIAA 22nd Fluid Dynamics, Plasmadynamics & Lasers Conference, Cited by: §3.5.1.
 [19] (2014) Adam: a method for stochastic optimization. External Links: 1412.6980 Cited by: §2.6.
 [20] (2021) Machine learning–accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences 118 (21). External Links: Document, ISSN 00278424, Link, https://www.pnas.org/content/118/21/e2101784118.full.pdf Cited by: §1.
 [21] (1998) CFL3D user’s manual (version 5.0). NASA TM–1998208444. Cited by: §1, §2.2.
 [22] (2021) Fast flow field estimation for various applications with a universally applicable machine learning concept. Flow Turbul. Combust. 107, pp. 175–200. External Links: Document Cited by: §1.
 [23] (2015) Understanding image representations by measuring their equivariance and equivalence. External Links: 1411.5908 Cited by: §1.
 [24] (2021) GraphTheta: a distributed graph neural network learning system with flexible training strategy. External Links: 2104.10569 Cited by: §1.
 [25] (2016) Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. Journal of Fluid Mechanics 807, pp. 155–166. External Links: Document Cited by: §1.
 [26] (2018) Deep learning for universal linear embeddings of nonlinear dynamics. Nat. Commun. 9, pp. 4950. Cited by: §1.
 [27] (2015) Compressible direct numerical simulation of lowpressure turbinesPart II: effect of inflow disturbances. ASME. J. Turbomach. 137, pp. 071005. Cited by: §1, §3.5.2.
 [28] (202006) CFDNet: a deep learningbased accelerator for fluid simulations. Proceedings of the 34th ACM International Conference on Supercomputing. External Links: ISBN 9781450379830, Link, Document Cited by: §1, §1.
 [29] (2014) Airfoils supporting nonunique transonic solutions for unsteady viscous flows. In 7th AIAA Theoretical Fluid Mechanics Conference, Cited by: §3.5.1.
 [30] (2019) PyTorch: an imperative style, highperformance deep learning library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlchéBuc, E. Fox, and R. Garnett (Eds.), pp. 8024–8035. External Links: Link Cited by: §2.6.
 [31] (2021) Learning meshbased simulation with graph networks. External Links: 2010.03409 Cited by: §1.

[32]
(1981)
Approximate riemann solvers, parameter vectors, and difference schemes
. Journal of Computational Physics 43 (2), pp. 357–372. External Links: ISSN 00219991, Document, Link Cited by: §2.3.  [33] (2015) Unet: convolutional networks for biomedical image segmentation. In Medical Image Computing and ComputerAssisted Intervention – MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi (Eds.), Cham, pp. 234–241. External Links: ISBN 9783319245744 Cited by: §2.5.
 [34] (1997) CFL3D: its history and some recent applications. NASA TM–112861. Cited by: §2.3.
 [35] (2010) CFL3D contribution to the AIAA supersonic shock boundary layer interaction workshop. NASA TM–2010216858. Cited by: §2.3.
 [36] (201505) Compressible Direct Numerical Simulation of LowPressure Turbines—Part I: Methodology. Journal of Turbomachinery 137 (5). Note: 051011 External Links: ISSN 0889504X, Document, Link, https://asmedigitalcollection.asme.org/turbomachinery/articlepdf/137/5/051011/6302098/turbo_137_05_051011.pdf Cited by: §1, §1.
 [37] (1992) A oneequation turbulence model for aerodynamic flows. In 30th Aerospace Sciences Meeting and Exhibit, Cited by: §2.3.
 [38] (2021) A deep learning based prediction approach for the supercritical airfoil at transonic speeds. Phys. Fluids 33 (8), pp. 086109. Cited by: §1.
 [39] (2018) Deep learning methods for Reynoldsaveraged NavierStokes simulations of airfoil flows. External Links: 1810.08217 Cited by: §1, §1, §1, §2.1, §2.5, §3.3.1.
 [40] (2021) Accuracy improvement technique of dnn for accelerating cfd simulator. External Links: 2106.01775 Cited by: §1, §1.
 [41] (2020) Solverintheloop: learning from differentiable physics to interact with iterative PDESolvers. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33, pp. 6111–6122. External Links: Link Cited by: §1.
 [42] (2002) Vortical dynamics and acoustic response in gasturbine swirlstabilized injectors. Ph.D. Thesis, The Pennsylvania State University, . Cited by: §1, §2.2.
 [43] (1995) Preconditioning applied to variable and constant density flows. AIAA Journal 33 (11), pp. 2050–2057. External Links: Document, Link, https://doi.org/10.2514/3.12946 Cited by: §2.3, §3.3.1.
Comments
There are no comments yet.