An efficient optimization based microstructure reconstruction approach with multiple loss functions

02/04/2021 ∙ by Anindya Bhaduri, et al. ∙ Johns Hopkins University 10

Stochastic microstructure reconstruction involves digital generation of microstructures that match key statistics and characteristics of a (set of) target microstructure(s). This process enables computational analyses on ensembles of microstructures without having to perform exhaustive and costly experimental characterizations. Statistical functions-based and deep learning-based methods are among the stochastic microstructure reconstruction approaches applicable to a wide range of material systems. In this paper, we integrate statistical descriptors as well as feature maps from a pre-trained deep neural network into an overall loss function for an optimization based reconstruction procedure. This helps us to achieve significant computational efficiency in reconstructing microstructures that retain the critically important physical properties of the target microstructure. A numerical example for the microstructure reconstruction of bi-phase random porous ceramic material demonstrates the efficiency of the proposed methodology. We further perform a detailed finite element analysis (FEA) of the reconstructed microstructures to calculate effective elastic modulus, effective thermal conductivity and effective hydraulic conductivity, in order to analyse the algorithm's capacity to capture the variability of these material properties with respect to those of the target microstructure. This method provides an economic, efficient and easy-to-use approach for reconstructing random multiphase materials in 2D which has the potential to be extended to 3D structures.



There are no comments yet.


page 5

page 6

page 13

page 20

page 21

page 23

page 25

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The discovery and development of new materials with targeted properties has always been a central theme for computational material science research, which requires a fundamental understanding of materials behavior and properties across different scales. Stochastic microstructure reconstruction is an important piece of that challenging puzzle. It enables computational generation of microstructures that match the statistically equivalent characteristic(s) of a (set of) target microstructure(s), avoiding the need for exhaustive and costly physical microstructure characterization of every sample to be evaluated.
There exists a number of different types of microstructure reconstruction approaches in the current literature. Reconstruction via statistical functions Hazlett (1997); Rintoul and Torquato (1997); Graham-Brady and Xu (2008) is a popular technique applicable to a wide range of materials such as, particulate structures Rintoul and Torquato (1997); Collins et al. (2010), chalk Talukdar et al. (2002), soil Gerke et al. (2012), sandstone Jiao et al. (2008); Tang et al. (2009).The reconstruction is usually done via a stochastic optimization method, known as Yeong and Torquato (YT) method Yeong and Torquato (1998); Torquato and Yeong (1998) and tries to ensure that the chosen statistical functions of the target microstructure match closely with that of the reconstructed microstructure. However, if the target image is anisotropic or multiphase or large in size, the convergence of the YT method is especially slow which makes the procedure computationally expensive. Physical descriptor-based approaches Breneman et al. (2013); Xu et al. (2014b, a), on the other hand, involve a relatively cheaper optimization procedure which tries to match the physically meaningful descriptors of the target microstructure to that of the reconstructed one. However, their application is limited to the microstructure reconstruction of crystalline and particulate structures (regular geometries) and not to material systems with irregular geometries. Spectral density function (SDF)-based approaches aim to reconstruct statistically equivalent microstructure by generating SDF representations of the target microstructure and matching it to the reconstructed one. The methods Quiblier (1984); Levitz (1998); Cahn (1965); Yu et al. (2017) are mostly analytical in nature and hence much faster than the optimization-based reconstruction approaches but their application is restricted to isotropic binary materials. Another class of microstructure reconstruction methods involves using deep learning Schmidhuber (2015); LeCun et al. (2015)

, a machine learning approach that can be used amongst other tasks for surrogate modeling

Cristianini et al. (2000); Williams (1998); Bhaduri and Graham-Brady (2018); Bhaduri et al. (2018, 2020b, 2020a, 2020c)

and thus has been implemented successfully for a wide range of classification and regression tasks. Deep learning approaches, in particular convolutional neural networks, are particularly well-suited to handle image data, and have thus received attention lately from the materials research community to process microstructures image data for a variety of tasks

DeCost et al. (2017); Lubbers et al. (2017). Deep learning approaches for reconstructions are of two different types: material-system-dependent and material-system-independent approaches. Material-system dependent approaches include work by Cang et al. Cang et al. (2017)

which uses a convolutional deep belief network

Lee et al. (2009) and work by Li et al. Li et al. (2018a)

where a Generative Adversarial Network (GAN)

Goodfellow et al. (2014)

model is employed. These approaches train the weights of the network with images specific to a material system and thus need to be retrained for a new material system. Alternatively, transfer learning approaches are material-system-independent and does not require training weights with a set of materials data. Instead, deep learning models, pre-trained using benchmark datasets in the field of computer vision, are used to generate microstructure reconstructions. However, it is noted that, given the fixed pretrained weights, there still may be a need for hyperparameter tuning to get the most optimal results for a given material data set. The work by Li et al.

Li et al. (2018b) is one such approach where a deep convolutional network VGG-19 Simonyan and Zisserman (2014)

pretrained on ImageNet

Deng et al. (2009) dataset has been used for the reconstruction based on a single given target microstructure. A feature-matching optimization has been performed using a Gram-matrix loss function to generate statistically equivalent microstructures. The reconstruction approach presented in this paper is derived from the work by Li et al Li et al. (2018b).
In this paper, we propose a modified version of the existing transfer learning approach Li et al. (2018b). The modifications include: a) different combinations of Gram matrix layers, b) a weighted combination of the gram matrix (GM) loss components, c) elimination of the potentially costly simulated-annealing based volume fraction matching process, d) inclusion of microstructural descriptor metrics in the overall loss function in the main optimization step. The modifications aim to enhance the efficiency, accuracy as well as interpretability of the proposed modified approach compared to that of the existing approach. It is noted here that the performance of the proposed approach is assessed via a statistical analysis of both the physics-based descriptors of the microstructure as well as the Finite Element Analysis (FEA) simulated averaged material properties. The analysis is demonstrated using a true microstructure dataset consisting of images of size pixels pixels, corresponding to successive slices of a porous ceramic microstructure measured by x-ray tomography similar to those found in Nickerson et al. (2019); Kočí et al. (2019). The target microstructures are described in section . Section discusses the denoising of the original target microstructures and compares the statistical properties between the original and the denoised versions. Section explains the six microstructural descriptors considered in this study to assess the reconstruction quality. The microstructure reconstruction procedure with different loss function combination and different slices (original and denoised) as the target image is discussed in details in section . In section , FEA simulated material properties are compared between the original and the reconstructed images. Section provides a discussion of the additional advantages of the proposed approach. Section provides the conclusions.

2 Target microstructures

The target microstructure is that of a two-phase porous ceramic material and there are images of successive slices of size pixels pixels. Thus a binary indicator function : represents the microstructure and is defined as

where is the material domain, is the black solid phase domain and is the white porous phase domain such that and . This is discretized in a computational setting by an array of (0,1) values. The slice microstructure is shown in figure 1.

Figure 1: Target microstructure (slice 1)

3 Denoising of target microstructure

(a) slice 1 (original)
(b) slice 1 (denoised)
(c) slice 90 (original)
(d) slice 90 (denoised)
(e) slice 185 (original)
(f) slice 185 (denoised)
Figure 2: Comparison of original and denoised microstructures

The need for denoising of the original target microstructure arises from the fact that spurious inclusions of only one or a few pixels appear in the image. These spurious pixels significantly complicate finite element (FE) analyses used to calculate effective properties. While denoising enables coarser discretization of the material domain and more efficient FE analysis, too much denoising can lead to significant changes in the local microstructure. Keeping this in mind, all cluster sizes for the black solid phase less than pixels are converted to the white porous phase. Figure 2 shows a comparison of the original and the corresponding denoised microstructures for slices , and .

4 Microstructural descriptors

Microstructural descriptors are considered key in determining the effective physical properties of random heterogeneous materials Torquato and Haslach Jr (2002). Thus, it is important to match the microstructural descriptors of the reconstructed microstructure with that of the original microstructure. In this study, six such descriptors are considered: porosity, correlation length, specific boundary length, mean pore size, -percentile pore size and -percentile pore size. Each of these descriptors are described below and some are shown in figure 3.
Porosity is the fraction of the porous phase in the microstructure given by:


where is any pixel location in the microstructural domain , while and are the number of pixels along the horizontal and the vertical directions respectively. Figure 4(a) shows the porosity of each of the 185 slices of the material. Denoising slightly increases porosity since small regions of the black solid phase are replaced by the white porous phase.
While the volume fraction provides information about the microstructural phase at individual points, it does not provide information about the spatial patterns of the microstructure that can be explained to some extent by the two-point probability function. The two-point probability function calculates the probability that any two points and

with a separation vector

fall in the white porous phase and is given by:

Figure 3: A schematic showing different microstructural descriptors in a binary-phase (black solid phase and white porous phase) microstructure. The two-point correlation function measure for the white phase is represented by a blue solid line. The red solid lines denote the phase boundaries, the total length of which gives the boundary length. The pore size distribution function is denoted by where is the pore radius variable.

It thus provides a measure of how the two end points of a vector (denoted by a blue solid line in figure 3) in a phase (in this case, the white porous phase) are correlated. It is noted that the above description of the two-point probability function similarly applies for the solid phase. In the above definition, it is assumed that the material is spatially stationary (or statistically homogeneous), which may not be valid for materials in which the microstructural statistics vary spatially (e.g., in a functionally graded material). Correlation length is one scalar measure associated with the two-point probability function, which is calculated by integrating the normalized two-point probability function over the entire domain Vanmarcke (1983) and then taking its square root:


Figure 4(b) shows the correlation length of each of the 185 slices of the material. It is noted that denoising slightly increases correlation length because it removes small correlation length inclusions.

(a) Porosity
(b) Correlation length
(c) Specific boundary length
(d) Mean pore size
(e) -percentile pore size
(f) -percentile pore size
Figure 4: Variation of statistical properties across original and denoised slices

Specific boundary length is the cumulative length of the boundary of separation between the black solid phase and the white porous phase, normalized by the total microstructural area. The cumulative boundary length denoted by red solid lines in figure 3 is obtained by generating the corresponding morphological gradient of the microstructure by taking the difference of its dilation and erosion. The morphological gradient of the microstructure is a matrix of the same size as the microstructural image with non-zero valued pixels along the boundaries of the two phases and zero-valued pixels elsewhere. The cumulative boundary length is thus obtained by counting all the non-zero pixels in the morphological gradient image. Figure 4(c) shows the specific boundary length of each of the 185 slices of the material for the original and denoised cases. It is seen that denoising slightly decreases the specific boundary length because the replacement of small clusters of black solid phase by white porous pixels leads to a decrease in the cumulative boundary length.

The pore-size probability density function

is used to characterize the void or “pore” space in porous media Prager (1963). The function for isotropic media is defined such that is the probability that a randomly chosen point in the porous phase domain lies at a distance between and from the nearest point on the pore-solid interface. Thus, . The mean pore size is defined as:


The associated complementary cumulative distribution function

is the fraction of pore space that has a pore radius larger than and is given by:


-percentile pore size is defined as the pore size such that of the pore space has a pore radius less than or equal to that pore size, i.e.,. -percentile pore size, similarly, is defined as the pore size such that of the pore space has a pore radius less than or equal to that pore size, i.e.,. Figures 4(d), 4(e) and 4(f) show the variation of the mean pore size, the -percentile pore size and the -percentile pore size, respectively, across all 185 slices for the original and their corresponding denoised microstructures. It is seen that denoising has very little effect on these three statistical properties of the microstructure.

5 Microstructure reconstruction

In this section, the reconstruction procedure and the corresponding results are discussed. The algorithm is an optimization based procedure and different loss function combinations are implemented with an optimal set of hyperparameters for the reconstruction of original slice which is discussed in sections 5.1, 5.2 and 5.3. Section 5.4 uses the same set of hyperparameters and compares the reconstruction results of original slice with that of original slices and . In section 5.5, reconstruction is performed on all the original slices and their denoised versions with different initializations of the reconstructed image, and the corresponding results are provided.

It is noted here that the initial reconstructed image is a white noise color (

-channels) image generated by random sampling. Thus the random number generator seed can be changed to generate different initial noisy images that eventually lead to statistically equivalent but distinct reconstructed microstructures. The reconstruction algorithm involves conversion of the -channels representation of the reconstructed color image to the single channel 2-phase representation. This is achieved by a binary classification of the 3D color image pixel coordinates based on their euclidean distances from the 3D black pixel coordinate and the 3D white pixel coordinate .

5.1 Reconstruction with Gram matrix loss

Figure 5: VGG-19 network architecture Simonyan and Zisserman (2014)

In this section, a transfer learning based pattern-matching optimization is performed using a pre-trained VGG-19 network shown in figure

5. VGG-19 is a deep convolutional neural network and in total has convolutional layers and max pooling layers, divided into blocks. The original network, in addition, has fully connected layers at the end which are excluded here. It is noted that -channels representation is a requirement for the input image to this network Li et al. (2018b). Thus, the -phase single channel representation of the original microstructure is converted to a -channels representation by stacking duplicates of the single channel image. Any intermediate output from layer operation in a convolutional neural network is called a feature map. Convolutional layers Aghdam and Heravi (2017); Venkatesan and Li (2017)

consists of one or more kernels (matrix of trainable weights) that convolve the input (dot product between the kernel-sized patch of the input and kernel) and pass it on to the next layer. Each convolution layer is followed by a nonlinear activation ReLU (rectified linear unit)

Krizhevsky et al. (2017); Romanuke (2017) layer which induces non-linearity into the network model. Max pooling Ciregan et al. (2012); Yamaguchi et al. (1990) uses a max filter for down sampling by selecting the maximum value of the non-intersecting kernel-sized patches of the feature map. Max pooling, in essence, extracts the most prominent features of the previous feature map.

A study has been performed to select the optimal number and combination of VGG-19 network layers for the most accurate reconstruction with respect to the Gram matrix (GM) loss calculation. Gram matrix Gatys et al. (2015) is a measure of texture in an image and the gram matrix loss here is a measure of the difference between the textures of different intermediate image outputs from the VGG-19 network for the original and the reconstructed microstructure respectively. Following the same notations as in Li et al. (2018b), let and denote the feature maps of the filter at position in layer for the original and the reconstructed microstructures, respectively. The Gram matrix of the original microstructure and of the reconstructed microstructure, in layer , is thus defined by the following inner product:


The Gram matrix loss for the layer is given by:


where is the number of filters and is the size of the vectorized feature maps in layer . The total Gram matrix loss across the selected layers is thus given by:


where the weight of each layer is given by .

Figure 6: Optimization framework with gram matrix loss

The first convolutional layers of each of the 5 blocks of the VGG-19 network as shown in figure 5 is found to be most optimal. The overall optimization framework is shown in figure 6. The reconstruction algorithm was run for different initializations of the reconstructed microstructure.

(a) Two-point probability function
for the porous phase
(b) Pore size complementary cdf
Figure 7: Comparison of statistical properties between the original slice and the corresponding reconstructed microstructure using the gram matrix loss function

Figure 7 compares the two-point probability function and the pore size cumulative density function (CDF) for the original slice microstructure and the reconstructed image corresponding to one of the initializations.

Figure 8: Reconstructed microstructure (checkerboard patterns) using gram matrix loss function based optimization

Figure 8 shows the reconstructed image corresponding to the original slice

image. Although the statistical property estimates of the reconstructed image are close to that of the target slice

, on visual inspection it is found that spurious checkerboard patterns are present in the reconstructions as seen in figure 8.

5.2 Reconstruction with GM and TV loss

Figure 9: Optimization framework with a combination of gram matrix loss and total variation loss
Figure 10: Reconstructed microstructure (no checkerboard patterns) using a combination of gram matrix loss and total variation loss for optimization

To address the checkerboard patterns, total variation (TV) loss between the target image and the reconstructed image is included in the optimization framework as shown in figure 9. The total variation for an image is defined as the sum of the absolute differences in the pixel values between neighboring pixels in that image. It is a measure of noise in an image that is sensitive to features like checkerboard patterns. The total variation loss is the anisotropic version of the total variation norm Rudin et al. (1992) and is given by:


where is the -th pixel value of the 3D representation of the reconstructed image. Figure 10 shows the reconstructed image corresponding to one of the initializations, and it is evident that the inclusion of the total variation loss eliminates the checkerboard patterns found in the reconstructed images in section 5.1. It is also noted that similar observations are found in all the other initialization cases. The total loss in this case is given by:


where and e are the weight combinations that seem to produce the best results using ‘trial and error’ approach. Table 1 shows the mean error in the statistical properties of interest calculated over different initialization cases for the reconstruction with the combined “GM+TV” loss and is compared to the corresponding results with only the GM loss discussed in section 5.1. It is seen that the mean error for almost all the statistical properties improves for the “GM+TV” loss case compared to the GM loss case. Although the error in the porosity is very low in both cases, the porosity matches slightly better on an average for the GM loss case.

GM loss
GM+TV loss
Mean error in porosity
0.0027 0.0048
Mean error in correlation length
0.2923 0.0580
Mean error in specific boundary length
0.0144 0.0050
Mean error in mean pore size
0.1225 0.0228
Mean error in 90-th percentile pore size
0.3846 0.0947
Mean error in 97-th percentile pore size
0.8641 0.1953
Table 1: Error in reconstruction with GM loss and “GM+TV” loss

5.3 Reconstruction with GM and TV and 2PP loss

Figure 11: Optimization framework with a combination of gram matrix loss, total variation loss and two-point correlation function loss

In this section, we try to improve the reconstruction quality further by including another loss function in the optimization, the two-point probability (2PP) loss . This loss is defined as the mean squared error between the 2-point probability functions [refer to eq (4)] of the original and the reconstructed microstructures. The two-point probability is sensitive to the spatial distribution of a given phase. The overall optimization framework is shown in figure 11. The total loss in this case is given by:


where , e and e are the weight combinations that seem to produce the best results using ‘trial and error’ approach. It is noted that a loss function corresponding to any other physical property of interest can be added to the optimization framework in a similar fashion. Figure 12 shows the box plot of the errors in the statistical properties of interest corresponding to each of the loss function cases: “GM”, “GM+TV” and “GM+TV+2PP” for the reconstruction of original slice . The errors are calculated over different initializations of the reconstructed image.

(a) Porosity
(b) Correlation length
(c) Specific boundary length
(d) Mean pore size
(e) -percentile pore size
(f) -percentile pore size
Figure 12: Comparison of error in statistical functions for the three different loss function cases

It is clear from the box plots that, with the exception of the very small porosity errors, all the other statistical properties match more accurately for the “GM+TV” and “GM+TV+2PP” loss cases compared to the “GM” loss case. However, the performance of the “GM+TV” and “GM+TV+2PP” loss cases are very similar to each other and the addition of the two-point probability loss function to the optimization framework does not improve the reconstruction results significantly, which is evident from the mean and coefficient of variation (CV) values of the absolute error in the properties shown in table 2.

GM+TV loss
GM+TV+2PP loss
Mean CV Mean CV
Error in porosity
0.0048 0.6402 0.0046 0.6169
Error in correlation length
0.0580 0.6698 0.0615 0.6437
Error in specific boundary length
0.0050 0.1587 0.0050 0.1867
Error in mean pore size
0.0228 0.7957 0.0247 0.7321
Error in 90-th percentile pore size
0.0947 0.6825 0.0992 0.6295
Error in 97-th percentile pore size
0.1953 0.6432 0.1873 0.6288
Table 2: Error in reconstruction with “GM+TV” loss and “GM+TV+2PP” loss

5.4 Performance comparison with other slices

(a) Porosity
(b) Correlation length
(c) Specific boundary length
(d) Mean pore size
(e) -percentile pore size
(f) -percentile pore size
Figure 13: Errors in the statistical properties across three different slices (1, 90, 185) for GM+TV and GM+TV+2PP loss

In the previous sections, reconstructions were performed with original slice as the target microstructure. In this section, with the same hyperparameter settings for the “GM+TV” and “GM+TV+2PP” loss cases as for slice -based reconstruction, the reconstruction algorithms are run with original slices and as the target microstructure over initializations. These runs are performed to verify whether the optimal hyperparameter settings for the slice -based reconstruction works well for the reconstruction on other slices too. Slices and are chosen as representative slices for that study. Figure 13 shows the performance comparison in the statistical properties for the “GM+TV” and “GM+TV+2PP” loss cases when the target microstructures are slices , and respectively. It is seen that the relative absolute errors in the statistical properties are in similar orders of magnitude across the three different slices for either of the loss cases. For each of the three target slice-based reconstructions, the reconstruction quality for both the loss cases are very similar to each other.

5.5 Reconstruction on all slices

(a) Porosity
(b) Correlation length
(c) Specific boundary length
(d) Mean pore size
(e) -percentile pore size
(f) -percentile pore size
Figure 14: Errors in the statistical properties across all 185 slices

In this section, reconstruction is performed by considering all original microstructure slices and their corresponding denoised versions individually as the target images. Three different initializations of the reconstructed image (leading to three distinct statistically equivalent images) are chosen for each of target image. Therefore, overall there are 555 reconstructions cases and their results are shown in figure 14. Figure 14 shows the comparison of the error in the statistical properties after reconstructions with the “GM+TV” and “GM+TV+2PP” loss cases, once with the original slices as the target microstructures and in the other instance, with the denoised slices as the target images. It is found that the comparison of almost all statistical properties are very similar to each other for all four reconstruction scenarios. The only exception is for the specific boundary length values where the overall error for the reconstruction cases with the denoised target slices are lower than that with the original target slices.

6 FE analysis based material properties

In this section, quality of the reconstructions are assessed in terms of a few averaged material properties of the porous ceramic material. The material properties of interest are the effective Young’s modulus (), effective thermal conductivity () and effective hydraulic conductivity (). After an image is reconstructed based on a target image, the properties of the reconstructed microstructure are simulated using the commercial FE software ABAQUS Hibbett et al. (1998). elements (each pixel assigned an element) are used to model each microstructure in 2D.

(a) Finite element model
(b) Von Mises stress map
Figure 15: Linear elastic analysis. (a) Finite element model: displacement is applied at the top face () with the bottom face held fixed (), both along the vertical direction; the left face is also fixed along the horizontal direction (); the in-plane Young’s modulus for the solid phase is given by MPa, and that of the porous phase is given by MPa. (b) Contour map of Von Mises stress obtained from the linear elastic analysis is shown here.

A static linear elastic analysis is performed to calculate the effective Young’s modulus by subjecting the finite element model of the microstructure to a tensile load as shown in figure 15. The macroscopic stress in the vertical direction (direction of applied displacement) is calculated Dalaq et al. (2016) by summing all the nodal reaction forces on the top face along the vertical direction and dividing it by the cross sectional area of the face:


where is the reaction force on node along the vertical direction, is the number of nodes on the surface where displacement is applied. The effective Young’s modulus is then obtained using:


where macroscopic strain and is the edge length along the vertical direction.

The effective thermal conductivity is obtained by applying a constant temperature difference between the top and bottom faces of the finite element model of the microstructure Ramani and Vaidyanathan (1995) as shown in figure 16. The macroscopic heat flux density is calculated by summing the nodal heat fluxes on the top face. The effective thermal conductivity is then calculated using Fourier’s law of heat conduction Bergman et al. (2011):


where is the temperature difference between the top and bottom face, and is the length of the edge along the vertical direction.

(a) Finite element model
(b) Heat flux density map
Figure 16: Heat conduction analysis. (a) Finite element model: Temperature difference is applied between the top face () and the bottom face (); the thermal conductivities for the solid phase and the porous phase are given by W/m/K and W/m/K respectively. (b) Contour map of the heat flux density obtained from the heat conduction analysis is shown here.
(a) Finite element model
(b) Specific discharge map
Figure 17: Fluid flow analysis. (a) Finite element model: Hydraulic head is applied between the top face () and the bottom face (); the hydraulic conductivities for the porous phase and the solid phase are given by m/s and m/s respectively. (b) Contour map of the specific discharge obtained from the fluid flow analysis is shown here.

The effective hydraulic conductivity for a laminar fluid flow through the porous medium of the material is simulated by specifying the hydraulic head between the top and bottom faces of the finite element model of the microstructure and then calculated from Darcy’s equation as shown in figure 17. The macroscopic specific discharge is calculated by summing up the nodal specific discharge values on the top face. The effective hydraulic conductivity is then calculated using Darcy’s law of fluid flow Whitaker (1986):


where is the hydraulic head between the top and bottom face, and is the length of the edge along the vertical direction.

(a) Effective modulus
(b) Effective thermal
(c) Effective hydraulic
Figure 18: Errors in the material properties across all 185 slices

Figure 18 shows the boxplot of the relative absolute errors in the effective modulus, conductivity and hydraulic conductivity of the reconstructed microstructures with respect to the corresponding target microstructures. It is seen in figure 18(a) that the effective Young’s modulus error values for all the cases are in an acceptable range. The reconstruction quality is similar between the two loss function cases when the original images are used as target slices. However, when the denoised images are used, the “GM+TV” loss case performs better than the “GM+TV+2PP” case. The error values corresponding to the effective thermal conductivity in figure 18(b) and the effective hydraulic conductivity in figure 18(c) are significantly higher than that of the elastic modulus. This suggests that the reconstruction algorithm is not able to capture these properties efficiently. This can be attributed to the fact that the effective modulus is a function of the overall pattern of the microstructure, i.e., the relative arrangement of the black solid and the white porous phase, which is well matched in the original and the reconstructed microstructure. However, the effective thermal conductivity and hydraulic conductivity depend significantly on localizations in the microstructures that enable heat flow and fluid flow respectively. The goal of the reconstruction algorithm is to achieve statistical equivalence by matching the pattern and statistics of the original and reconstructed microstructure across the entire image domain. Thus, even with a statistically equivalent reconstructed image, differences in small localized regions can lead to significant differences in the heat and fluid flow channels, resulting in very different effective thermal and hydraulic conductivity. Overall, if the error values are compared among all the four reconstruction cases for each material property, it is seen that the “GM+TV” loss case with the denoised slices as the target images yields the most accurate reconstructions.

(a) Original slice 1 ( )
(b) Reconstructed image ( )
Figure 19: Comparison of the original and a larger reconstructed microstructure

7 Discussion

Results in the previous sections have shown the performance of the proposed reconstruction approach to be reasonably accurate and efficient. One further advantage of the proposed approach is that the reconstructed image is not constrained to be of the same size as the original image. Obtaining a high resolution image of a microstructure can be expensive, which limits the size and/or resolution of the obtained image. It is often of interest to analyze a microstructural image of dimensions larger than that of the original image. One of the useful features of this approach is the ability to reconstruct microstructure of spatial dimensions different from that of the original microstructural image. Here, as a demonstration, reconstruction of an image of size pixels pixels is considered using the “GM+TV+2PP” loss. The larger reconstructed image is shown in figure 19 along with the original slice microstructure for a visual comparison. To demonstrate the statistical equivalence of the larger reconstruction microstructure with its target image, the two-point probability function plot for the porous phase and the pore size complementary cdf plot for the two images are compared and shown in figure 20.

(a) Two-point probability function
for the porous phase
(b) Pore size complementary cdf
Figure 20: Comparison of statistical properties between the original slice and the corresponding 700x1000 reconstructed microstructure using the “GM+TV+2PP” loss function

Another advantage of the proposed approach is that it is more efficient and accurate than other existing approaches based on statistical features of the microstructure. The approach is thus compared here with a modified version of the YT method Yeong and Torquato (1998); Torquato and Yeong (1998); Graham-Brady and Xu (2008) in terms of efficiency and accuracy. The YT method initiates by generating an image having the same porosity measure as the target microstructure. It is an iterative method where in each iteration, a randomly chosen black solid phase pixel and a white porous phase pixel are interchanged in the image. If the pixel swapping leads to an improvement in the statistical measure (equivalent to a reduction in loss function measure) of the new image compared to the previous one, the new image is updated to be the current reconstructed image. Otherwise, the previous image remains the current reconstructed image at the end of that iteration. The pixel swapping is also allowed if a random number generator, generating random numbers uniformly between and , returns a value greater than or equal to . Here is defined as the pixel interchange probability. The iterative convergence of the reconstructed image to the target image is measured by the root of sum of squares error (RSSE) between the two-point correlation function of the reconstructed image and the target image. With increase in the number of iterations, RSSE has a decreasing trend and the reconstructed image converges closer to the target microstructure with respect to the two-point correlation function measure.

(a) Original slice
(b) Multi-loss based reconstruction
(c) YT method based reconstruction
Figure 21: Comparison of reconstructed images using the mutli-loss optimization based recontruction algorithm and a modified version of the YT method.

Figure 21 shows a visual comparison of the reconstructed images formed by using the proposed reconstruction approach and the YT method. Slice , shown in figure (a)a, is used as the target image in this case. It is clearly seen that the quality of the reconstructed image obtained by the proposed approach is better than that of the YT method. In this case, the YT method is run for million iterations and it takes around minutes to obtain the image in figure (c)c. On the other hand, the proposed algorithm takes , iterations and only around minutes to produce the image in figure (b)b. Thus, the proposed approach is found to be much more efficient and accurate than the YT method in the reconstruction of the porous ceramic material. It is noted that the algorithms are run on Google Colaboratory Pro Bisong (2019), a cloud-based Jupyter notebook environment.

8 Conclusions

In this paper, a modified version of an existing transfer learning reconstruction approach Li et al. (2018b) is presented by introducing additional loss functions in the optimization framework. It is then used to reconstruct a binary porous ceramic material. A thorough statistical study was done to check the robustness of the algorithm over different initializations and over different microstructure slices (original and denoised) of the same material. Furthermore, the reconstruction quality is assessed not only based on the statistical properties of the microstructure but also the FEA simulated effective material properties. This algorithm has the advantage of being easily extended to 3D microstructure reconstruction Sundararaghavan and Zabaras (2005); Bochenek and Pyrz (2004); Bhandari et al. (2007) if a suitable pretrained 3D convolutional network is available. It also has the flexibility of generating microstructures of size different from that of the target microstructure. A potential future work can be incorporating the simulated effective material property information within the reconstruction algorithm framework in an efficient manner, in order to generate microstructures with effective material properties closer to that of the target microstructure.


Research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-12-2-0023 and W911NF-12-2-0022. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.


  • H. H. Aghdam and E. J. Heravi (2017) Guide to convolutional neural networks. New York, NY: Springer 10, pp. 978–973. Cited by: §5.1.
  • T. L. Bergman, F. P. Incropera, D. P. DeWitt, and A. S. Lavine (2011) Fundamentals of heat and mass transfer. John Wiley & Sons. Cited by: §6.
  • A. Bhaduri, D. Brandyberry, M. D. Shields, P. Geubelle, and L. Graham-Brady (2020a) On the usefulness of gradient information in surrogate modeling: application to uncertainty propagation in composite material models. Probabilistic Engineering Mechanics 60, pp. 103024. Cited by: §1.
  • A. Bhaduri, J. Gardner, C. F. Abrams, and L. Graham-Brady (2020b) Free energy calculation using space filled design and weighted reconstruction: a modified single sweep approach. Molecular Simulation 46 (3), pp. 193–206. Cited by: §1.
  • A. Bhaduri and L. Graham-Brady (2018) An efficient adaptive sparse grid collocation method through derivative estimation. Probabilistic Engineering Mechanics 51, pp. 11–22. Cited by: §1.
  • A. Bhaduri, Y. He, M. D. Shields, L. Graham-Brady, and R. M. Kirby (2018) Stochastic collocation approach with adaptive mesh refinement for parametric uncertainty analysis. Journal of Computational Physics 371, pp. 732–750. Cited by: §1.
  • A. Bhaduri, C. S. Meyer, J. W. Gillespie Jr, B. Z. Haque, M. D. Shields, and L. Graham-Brady (2020c) Probabilistic modeling of discrete structural response with application to composite plate penetration models. arXiv preprint arXiv:2011.11780. Cited by: §1.
  • Y. Bhandari, S. Sarkar, M. Groeber, M. Uchic, D. Dimiduk, and S. Ghosh (2007) 3D polycrystalline microstructure reconstruction from fib generated serial sections for fe analysis. Computational Materials Science 41 (2), pp. 222–235. Cited by: §8.
  • E. Bisong (2019) Google Colaboratory. In Building Machine Learning and Deep Learning Models on Google Cloud Platform, pp. 59–64. Cited by: §7.
  • B. Bochenek and R. Pyrz (2004) Reconstruction of random microstructures—-a stochastic optimization problem. Computational Materials Science 31 (1-2), pp. 93–112. Cited by: §8.
  • C. M. Breneman, L. C. Brinson, L. S. Schadler, B. Natarajan, M. Krein, K. Wu, L. Morkowchuk, Y. Li, H. Deng, and H. Xu (2013) Stalking the materials genome: a data-driven approach to the virtual design of nanostructured p olymers. Advanced Functional Materials 23 (46), pp. 5746–5752. Cited by: §1.
  • J. W. Cahn (1965) Phase separation by spinodal decomposition in isotropic systems. The Journal of Chemical Physics 42 (1), pp. 93–99. Cited by: §1.
  • R. Cang, Y. Xu, S. Chen, Y. Liu, Y. Jiao, and M. Yi Ren (2017) Microstructure representation and reconstruction of heterogeneous materials via deep belief network for computational material design. Journal of Mechanical Design 139 (7). Cited by: §1.
  • D. Ciregan, U. Meier, and J. Schmidhuber (2012) Multi-column deep neural networks for image classification. In

    2012 IEEE Conference on Computer Vision and Pattern Recognition

    pp. 3642–3649. Cited by: §5.1.
  • B. Collins, K. Matous, and D. Rypl (2010) Three-dimensional reconstruction of statistically optimal unit cells of multimodal particulate composites. International Journal for Multiscale Computational Engineering 8 (5). Cited by: §1.
  • N. Cristianini, J. Shawe-Taylor, et al. (2000)

    An introduction to support vector machines and other kernel-based learning methods

    Cambridge University Press. Cited by: §1.
  • A. S. Dalaq, D. W. Abueidda, R. K. A. Al-Rub, and I. M. Jasiuk (2016) Finite element prediction of effective elastic properties of interpenetrating phase composites with architectured 3D sheet reinforcements. International Journal of Solids and Structures 83, pp. 169–182. Cited by: §6.
  • B. L. DeCost, T. Francis, and E. A. Holm (2017) Exploring the microstructure manifold: image texture representations applied to ultrahigh carbon steel microstructures. Acta Materialia 133, pp. 30–40. Cited by: §1.
  • J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei (2009) Imagenet: a large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. Cited by: §1.
  • L. Gatys, A. S. Ecker, and M. Bethge (2015) Texture synthesis using convolutional neural networks. In Advances in Neural Information Processing Systems, pp. 262–270. Cited by: §5.1.
  • K. Gerke, M. Karsanina, and E. Skvortsova (2012) Description and reconstruction of the soil pore space using correlation functions. Eurasian Soil Science 45 (9), pp. 861–872. Cited by: §1.
  • I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio (2014) Generative adversarial nets. In Advances in Neural Information Processing Systems, pp. 2672–2680. Cited by: §1.
  • L. Graham-Brady and X. F. Xu (2008) Stochastic morphological modeling of random multiphase materials. Journal of Applied Mechanics 75 (6). Cited by: §1, §7.
  • R. Hazlett (1997) Statistical characterization and stochastic modeling of pore networks in relation to fluid flow. Mathematical Geology 29 (6), pp. 801–822. Cited by: §1.
  • Hibbett, Karlsson, and Sorensen (1998) ABAQUS/Standard: user’s manual. Vol. 1, Hibbitt, Karlsson & Sorensen. Cited by: §6.
  • Y. Jiao, F. Stillinger, and S. Torquato (2008) Modeling heterogeneous materials via two-point correlation functions. II. Algorithmic details and applications. Physical Review E 77 (3), pp. 031135. Cited by: §1.
  • P. Kočí, M. Isoz, M. Plachá, A. Arvajová, M. Václavík, M. Svoboda, E. Price, V. Novák, and D. Thompsett (2019) 3D reconstruction and pore-scale modeling of coated catalytic filters for automotive exhaust gas aftertreatment. Catalysis Today 320, pp. 165–174. Cited by: §1.
  • A. Krizhevsky, I. Sutskever, and G. E. Hinton (2017) Imagenet classification with deep convolutional neural networks. Communications of the ACM 60 (6), pp. 84–90. Cited by: §5.1.
  • Y. LeCun, Y. Bengio, and G. Hinton (2015) Deep learning. Nature 521 (7553), pp. 436–444. Cited by: §1.
  • H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng (2009)

    Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations

    In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 609–616. Cited by: §1.
  • P. Levitz (1998) Off-lattice reconstruction of porous media: critical evaluation, geometrical confinement and molecular transport. Advances in Colloid and Interface Science 76, pp. 71–106. Cited by: §1.
  • X. Li, Z. Yang, L. C. Brinson, A. Choudhary, A. Agrawal, and W. Chen (2018a) A deep adversarial learning methodology for designing microstructural material systems. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Vol. 51760, pp. V02BT03A008. Cited by: §1.
  • X. Li, Y. Zhang, H. Zhao, C. Burkhart, L. C. Brinson, and W. Chen (2018b) A transfer learning approach for microstructure reconstruction and structure-property predictions. Scientific Reports 8 (1), pp. 1–13. Cited by: §1, §5.1, §8.
  • N. Lubbers, T. Lookman, and K. Barros (2017) Inferring low-dimensional microstructure representations using convolutional neural networks. Physical Review E 96 (5), pp. 052111. Cited by: §1.
  • S. Nickerson, Y. Shu, D. Zhong, C. Könke, and A. Tandia (2019) Permeability of porous ceramics by X-ray CT image analysis. Acta Materialia 172, pp. 121–130. Cited by: §1.
  • S. Prager (1963) Interphase transfer in stationary two-phase media. Chemical Engineering Science 18 (4), pp. 227–231. Cited by: §4.
  • J. A. Quiblier (1984) A new three-dimensional modeling technique for studying porous media. Journal of Colloid and Interface Science 98 (1), pp. 84–102. Cited by: §1.
  • K. Ramani and A. Vaidyanathan (1995) Finite element analysis of effective thermal conductivity of filled polymeric composites. Journal of Composite Materials 29 (13), pp. 1725–1740. Cited by: §6.
  • M. D. Rintoul and S. Torquato (1997) Reconstruction of the structure of dispersions. Journal of Colloid and Interface Science 186 (2), pp. 467–476. Cited by: §1.
  • V. Romanuke (2017) Appropriate number and allocation of relus in convolutional neural networks. Scientific News of the National Technical University of Ukraine, Kyiv Polytechnic Institute (1), pp. 69–78. Cited by: §5.1.
  • L. I. Rudin, S. Osher, and E. Fatemi (1992) Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena 60 (1-4), pp. 259–268. Cited by: §5.2.
  • J. Schmidhuber (2015) Deep learning in neural networks: an overview. Neural Networks 61, pp. 85–117. Cited by: §1.
  • K. Simonyan and A. Zisserman (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. Cited by: §1, Figure 5.
  • V. Sundararaghavan and N. Zabaras (2005) Classification and reconstruction of three-dimensional microstructures using support vector machines. Computational Materials Science 32 (2), pp. 223–239. Cited by: §8.
  • M. Talukdar, O. Torsaeter, M. Ioannidis, and J. Howard (2002) Stochastic reconstruction of chalk from 2d images. Transport in Porous Media 48 (1), pp. 101–123. Cited by: §1.
  • T. Tang, Q. Teng, X. He, and D. Luo (2009) A pixel selection rule based on the number of different-phase neighbours for the simulated annealing reconstruction of sandstone microstructure. Journal of Microscopy 234 (3), pp. 262–268. Cited by: §1.
  • S. Torquato and C. Yeong (1998) Reconstructing random media. II. Three-dimensional media from two-dimensional cuts. Physical Review E 58 (1), pp. 224–233. Cited by: §1, §7.
  • S. Torquato and H. Haslach Jr (2002) Random heterogeneous materials: microstructure and macroscopic properties. Applied Mechanics Reviews 55 (4), pp. B62–B63. Cited by: §4.
  • E. Vanmarcke (1983) Random fields. Cited by: §4.
  • R. Venkatesan and B. Li (2017) Convolutional neural networks in visual computing: a concise guide. CRC Press. Cited by: §5.1.
  • S. Whitaker (1986) Flow in porous media I: A theoretical derivation of Darcy’s law. Transport in Porous Media 1 (1), pp. 3–25. Cited by: §6.
  • C. K. Williams (1998)

    Prediction with gaussian processes: from linear regression to linear prediction and beyond

    In Learning in Graphical Models, pp. 599–621. Cited by: §1.
  • H. Xu, D. A. Dikin, C. Burkhart, and W. Chen (2014a) Descriptor-based methodology for statistical characterization and 3D reconstruction of microstructural materials. Computational Materials Science 85, pp. 206–216. Cited by: §1.
  • H. Xu, Y. Li, C. Brinson, and W. Chen (2014b) A descriptor-based design methodology for developing heterogeneous microstructural materials system. Journal of Mechanical Design 136 (5). Cited by: §1.
  • K. Yamaguchi, K. Sakamoto, T. Akabane, and Y. Fujimoto (1990) A neural network for speaker-independent isolated word recognition. In First International Conference on Spoken Language Processing, Cited by: §5.1.
  • C. Yeong and S. Torquato (1998) Reconstructing random media. Physical Review E 57 (1), pp. 495. Cited by: §1, §7.
  • S. Yu, Y. Zhang, C. Wang, W. Lee, B. Dong, T. W. Odom, C. Sun, and W. Chen (2017) Characterization and design of functional quasi-random nanostructured materials using spectral density function. Journal of Mechanical Design 139 (7). Cited by: §1.