An active learning high-throughput microstructure calibration framework for solving inverse structure-process problems in materials informatics

04/24/2020 ∙ by Anh Tran, et al. ∙ 0

Determining a process-structure-property relationship is the holy grail of materials science, where both computational prediction in the forward direction and materials design in the inverse direction are essential. Problems in materials design are often considered in the context of process-property linkage by bypassing the materials structure, or in the context of structure-property linkage as in microstructure-sensitive design problems. However, there is a lack of research effort in studying materials design problems in the context of process-structure linkage, which has a great implication in reverse engineering. In this work, given a target microstructure, we propose an active learning high-throughput microstructure calibration framework to derive a set of processing parameters, which can produce an optimal microstructure that is statistically equivalent to the target microstructure. The proposed framework is formulated as a noisy multi-objective optimization problem, where each objective function measures a deterministic or statistical difference of the same microstructure descriptor between a candidate microstructure and a target microstructure. Furthermore, to significantly reduce the physical waiting wall-time, we enable the high-throughput feature of the microstructure calibration framework by adopting an asynchronously parallel Bayesian optimization by exploiting high-performance computing resources. Case studies in additive manufacturing and grain growth are used to demonstrate the applicability of the proposed framework, where kinetic Monte Carlo (kMC) simulation is used as a forward predictive model, such that for a given target microstructure, the target processing parameters that produced this microstructure are successfully recovered.



There are no comments yet.


page 13

page 15

page 16

page 21

page 23

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Identifying the process-structure-property relationship is the holy grail of materials science, where recent advances in data science and informatics have offered many scalable solutions to accelerate the materials discovery and development

kalidindi2016vision , which aims to reduce the materials development time by reducing the reliance on physical experimentation national2011materials ; backman2006icme . While substantial efforts have been made in the predictive forward direction, including model calibration for integrated materials computational engineering (ICME) models chaparro2008material , high-throughput experiments salzbrenner2017high , and high-throughput simulations ong2019accelerating , materials design is instead equivalent to solving an inverse problem sinnott2013material ; arroyave2019systems ; olson2013genomic . In the forward problem, one asks the predictive question of materials properties and performance characteristics, given a set of materials descriptors related to materials synthesis and processing. In the inverse problem, typically, one searches for the optimal and feasible materials descriptors related to materials synthesis and processing, to optimize a set of materials properties and performance characteristics. Notably, while most of the inductive materials design research have been cast directly into the process-structure-property relationship by considering materials properties as objective functions, there is a lack of research involving microstructures where microstructures are considered as either input or output in this relationship, particularly with image representation. However, the approach of bypassing materials structure and seeking direct process-property correlation is not broadly applicable, because it requires a complete description of materials synthesis and processing history, which is often incomplete or not available kalidindi2016vision . We argue that the materials design problem in inverting process-microstructure relationship is also useful, particularly in reverse materials engineering of a specimen, where only microstructure is available. The main challenge in this problem lies in the materials characterization as an effort to obtain a set of useful microstructure descriptors that capture the heterogeneous, anisotropic, and stochastic behaviors of the microstructure.

Deterministic inverse problems, which include inverse problems in materials science contexts, are typically solved using optimization methods. In the field of manufacturing, a direct process-property relationship is typically considered, as the goal is to search for the optimal processing parameters which produces single or multiple materials properties. In microstructure-sensitive design, one requires an efficient mathematical representation of microstructures and homogenization techniques to establish the structure-property linkage, and searches for the optimal microstructures in the microstructure space that correspond with optimal materials performance adams2012microstructure ; fullwood2010microstructure . We provide a brief literature review on forward prediction of process-structure linkage, inverse problems in process-structure-property relationship, and microstructure-sensitive design problems in Section 2.

In our approach, the inverse process-structure problem is formulated as a noisy multi-objective optimization problem, where each objective function is associated with a microstructure descriptor. In particular, the objective function measures the deterministic or statistical difference of the same microstructure descriptor applied on two different microstructure images. First, a set of microstructure descriptors, which can be deterministic or statistical microstructure descriptors, are used to describe the heterogeneous and anisotropic microstructure. Second, some statistical metrics are applied on the probability density functions of these microstructure descriptors to measure the difference between two microstructures. This approach results in a set of objective functions, where each objective function corresponds to a microstructure descriptor. Multiple objective functions are then

scalarized into a single-objective function, where an asynchronously parallel Bayesian optimization framework is deployed to solve efficiently on a high-performance computing platform. To the best of the authors’ knowledge, it is the first attempt to solve for the inverse problem in process-structure linkage, where the microstructures are represented as images, which is the most common representation for microstructure.

In the rest of this paper, Section 2 discusses the background on the predictive forward problem in process-structure linkage (Section 2.1), as well as a brief literature review on inverse problems in process-structure-property linkages (Section 2.2), and microstructure-sensitive design (Section 2.3). Section 3 describes the microstructure calibration methodology, which is then formulated as a noisy multi-objective optimization problem. To that end, a parallel Bayesian optimization framework is utilized to efficiently minimize the difference between the target and candidate microstructures, resulting in the optimal microstructure and a set of processing parameters that can yield a similar and statistically equivalent microstructure. Section 4 presents the first case study of kinetic Monte Carlo (kMC) in welding, in which the set of target processing parameters are recovered. Section 5 presents the second case study of kMC in grain growth, where the numerical temperature parameter input is recovered. In both sections 4 and 5, high-dimensional microstructures are considered as outputs. Section 6 discusses the results and Section 7 concludes the paper, respectively.

2 Related works

In this section, we provide a brief literature review about related works. Section 2.1 discusses related work on modeling and simulation of microstructure evolution for a manufacturing process. Section 2.2 discusses inverse problems for the process-structure linkage in computational materials science, which many are related to manufacturing. Section 2.3 reviews previous works in microstructure-sensitive design problems.

2.1 Process-structure linkage

Numerous ICME modeling and simulation methods have been developed to predict microstructures, including kMC, finite element, cellular automata, phase field, lattice Boltzmann, and even hybrid methods. Rodgers et al. rodgers2016predicting ; rodgers2017monte implemented a kMC model in SPPARKS to simulate microstructures for a welding process, to which Li and Soshi li2019modeling performed a model analysis on grain morphology with different processing parameters. Zhang et al. zhang2016coupled proposed a hybrid framework of cellular automata and finite element to predict thermal history and grain morphology in Ti-6Al-4V for direct metal deposition. Rai et al. rai2017simulation combined cellular automata and lattice Boltzmann to predict microstructure evolution for powder bed based additive manufacturing. Archaya et al. acharya2017prediction , Chen et al. ji2018understanding ; yu2018phase employed a phase-field model to study the microstructure prediction and microstructure evolution in laser bed fusion and welding processes. Wang et al. tran2019quantifying ; cao2019multi ; liu2019mesoscale combined lattice Boltzmann method with phase-field model to study the dendritic growth in solidification process. For a data-driven approach, Kalidindi et al. yabansu2017extraction ; brough2017microstructure ; popova2017process

developed a two-point statistics approach, called the Materials Knowledge System framework, to establish the relationship between process and structure linkage by principal component analysis, which is a linear-dimensionality reduction framework.

2.2 Inverse problems in process-structure-property

Inverse problems in materials science are typically cast directly into process-property linkage, where materials properties are considered as outputs and manufacturing processes are considered as inputs. The linkage is often considered as an unknown black-box function, where optimization methods are utilized to find process inputs which minimize or maximize materials properties. This approach is very common, particularly in the field of manufacturing. For example, Jiang et al. jiang2016optimization optimized the depth of penetration and bead width in laser welding process without considering microstructure. Liang et al. park2011multi ; zhang2015study ; wang2018simultaneous

utilized a genetic algorithm and a particle swarm optimization method to various manufacturing problems, where a computational model is considered as objective functional evaluators. Pfeifer et al.

pfeifer2018process utilized the sequential Bayesian optimization framework to search for optimal processing conditions for the fabrication of organic photovoltaic thin films. Fernandez-Zelaia and Melkote fernandez2019process also relied on multi-output Gaussian process model to build a surrogate model between process and property. Forsmark et al. forsmark2015using bypassed the traditional approach and established directly the process-property linkage for thin-walled high-pressure die castings of the magnesium alloy AM60. Nath et al. nath2019uncertainty applied Gaussian process model and global sensitivity analysis to quantify uncertainty in microstructure grain morphology based on the cellular automata simulations of laser direct metal deposition. Ling et al. ling2017high

proposed a sequential learning approach, where the surrogate model is Random Forests but materials properties are considered as outputs. The only work that we have found which targets microstructure as inputs is Paul et al.

paul2019microstructure . In this work, they also employed Random Forests algorithm, as in Ling et al. ling2017high to optimize materials properties with respect to microstructure orientation distribution functions. Johnson and Arróyave johnson2016inverse proposed an inverse design framework for process-structure linkage and identify the heat treatment in Ni-rich NiTi shape memory alloys with a desired size distribution of NiTi precipitates. To the best of the authors’ knowledge, we are unaware of any prior work considering microstructure images as outputs in the process-structure linkage, which is the focus of this study.

2.3 Microstructure-sensitive design

Microstructure-sensitive design problems aim to tailor the microstructure in such a way that materials properties and performance characteristics are optimized adams2012microstructure ; fullwood2010microstructure . First, given a microstructure, a mathematical representation, such as two-point statistics, can be devised to describe the microstructure in a low-dimensional space kalidindi2016vision ; arroyave2019systems to improve the efficiency of the optimization method. Next, homogenization techniques are utilized to form a structure-property linkage, followed by the goal-oriented search in the microstructure space to identify optimal microstructures. The crystal plasticity finite element technique is one of the most widely used tools in this field. Fast et al. fast2008application demonstrated the first successful design case studies to optimize performance of structural components made from polycrystalline metals with hexagonal close-packed crystal lattices. Priddy et al. priddy2017strategies employed the Materials Knowledge System framework to significantly reduce the computational cost of crystal plasticity finite element methods and efficiently explore the extreme value distribution of the fatigue indicator parameters with various microstructures. Paulson et al. paulson2018data ; paulson2019reduced followed Priddy et al. priddy2017strategies in applying the Materials Knowledge System framework in the transition fatigue regime. Sundararaghavan and Zabaras sundararaghavan2006design proposed a finite element multiscale homogenization model to capture the elasto-viscoplastic behavior of a microstructure and applied gradient-based optimization methods to achieve the optimal microstructure. Recently, Li et al. li2018deep and Yang et al. yang2018microstructural

employed the generative adversarial framework, which is a deep learning approach, to model the microstructure with a set of low-dimensional latent variables, and subsequently optimize the structure-property where the latent microstructure variables and the materials properties are the inputs and outputs, respectively.

3 Methodology

In this section, the inverse problem of calibrating processing parameters via microstructure is described in an active learning context, where a high-throughput optimization framework can be utilized to accelerate the optimization procedure. Section 3.1 defines the problem and provides an overview perspective of the proposed method. Section 3.2 discusses the microstructure descriptors, the conditional microstructure descriptors using filters, and the necessity of microstructure filters in some applications. Section 3.4 describes the quantitative measures for deterministic and statistical microstructure descriptors. Section 3.5 describes the conversion of the formulated problem in the materials science context to the multi-objective optimization problems in the mathematical context. Section 3.6 discusses the possibility of parallelizing the Bayesian optimization method as an active learning approach to accelerate the optimization procedure in the high-performance computing environment.

3.1 Problem formulation

Figure 1: An overview perspective of the high-throughput microstructure calibration framework as a multi-objective optimization problem. The microstructure descriptors are denoted as . For statistical microstructure descriptors , for some , the microstructure descriptor for are represented as a probability density function . The microstructure descriptors characterizing the target microstructure and the candidate microstructure are denoted as and , respectively. For each microstructure descriptor , the difference between on the target microstructure and on the candidate is measured through some mathematical or statistical metric , depending on whether is deterministic or statistical. The objective functions, denoted as , are the difference measured through . In particular, if a statistical microstructure descriptor, then We assume that there is a one-to-one mapping between the descriptor and . The microstructure calibration framework can be generalized to a high-throughput computational materials characterization, where the input can be queried through the prediction model concurrently in an asynchronously parallel manner. Readers are referred to the online version for color visualization.

Figure 1 presents an overview of the methodology. We are concerned with the inverse problem of process-structure linkage, where the target microstructure (denoted as targetMs) is known beforehand, and a computational process-structure model that is capable of predicting the microstructure is available. The model can be either deterministic or stochastic. The goal is to identify the processing parameters that produce the target microstructure. In the optimization process, the optimization algorithm proposes many candidate microstructures (denoted as candidateMs), and compares quantitatively between microstructures in order to eventually propose an optimal microstructure that matches very closely with the target microstructure in the statistical sense. In other words, the optimal microstructure should be statistically equivalent to the target microstructure.

3.2 Materials characterization by microstructure descriptors

Microstructure is intrinsically noisy due to its stochastic polycrystalline nature, particularly in metallic systems. Thus, it is necessary to describe the microstructure using statistical descriptors. Bostanabad et al. bostanabad2018computational , Liu et al. liu2013computational , Li li2014review , and Bargmann et al. bargmann2018generation provided a comprehensive review for computational microstructure reconstruction, generation, and characterization techniques, with statistical and deterministic physics-based microstructure descriptors.

The main difference between physics-based and data-driven microstructure descriptors is their interpretability and explainability. On one hand, physics-based microstructure descriptors are explainable and associated with certain physical attributes of the microstructure, e.g. volume of fraction, grain size distribution, aspect ratio, etc. The majority of the work done so far in materials science community falls in the spectrum of physics-based microstructure descriptors. On another hand, data-driven microstructure descriptors are impossibly hard to explain, but are capable of fully describing the microstructure. These data-driven microstructure descriptors are often associated with the latent variables in unsupervised machine learning approaches, in particular, deep learning approaches such as generative adversarial networks

latief2010continuum ; mosser2017reconstruction ; mosser2018stochastic . These latent variables are capable of accurately describing the microstructure and are thus considered as data-driven microstructure descriptors, to contrast with the physics-based microstructure descriptors.

To compare between two microstructures quantitatively and measure how far they are away from each, one needs to impose many microstructure descriptors, including both statistical and deterministic. The microstructure descriptors must be able to distinguish one microstructure from another, quantitatively. Because of the stochastic nature of microstructure, the statistical microstructure descriptors are used more often than the deterministic ones. Suppose that there are microstructure descriptors, denoted as , in Figure 1. Given a microstructure, one can collect a population of grains, and subsequently build a statistics on the grain population with a probability density function representation. For example, one can (i) compute the grain area for each grain in the microstructure, (ii) collect all the observations, and (iii) approximate the probability density function of the observational grain areas of the microstructure.

3.3 Conditional microstructure descriptors

It may be beneficial and necessary to impose some conditions to compare the microstructure through the lens of statistics, particularly in choosing the right statistics that can distinguish one microstructure from another. Conventional statistical microstructure descriptors do not always work very well, mostly due to numerical errors during the computation process. If not well chosen, it may be impossible to quantitatively compare microstructures, and thus the optimization framework does not work as one expected. Here, we propose to “filter” the microstructures to eliminate grains or features that are too common and may negatively impact the optimization process. The filter is implemented by imposing some conditions on conventional statistical microstructure descriptors to create conditional statistical microstructure descriptors.

The process to approximate the conditional statistical microstructure descriptors is described as follows, and is summarized in Algorithm 1. For each grain in the filtered microstructure, the grain filter is first applied to eliminate grains that are common, by only considering if the filter is passed, i.e. . Second, a microstructure descriptor is applied on the grain after it has passed the filter

. The quantitative descriptors are then collected for all grains to build a population of samples, where kernel density estimation can be applied to reconstruct the probability density function with the optimal bandwidth. Thus, for each statistical microstructure descriptor, there is a corresponding probability density function, constructed via the kernel density estimation method.

Input: microstructure consisting of grains , a grain-based descriptor for , a grain-based filter for

Output: statistical microstructure descriptors of

1: initialize sample population
2:for every grain in the microstructure do
3:     if  then if the grain passes through the filter
4:          aggregate the population sample
5:     end if
6:end for
7: approx. the density using kernel density estimation
Algorithm 1 Approximation of grain-based microstructure descriptors with probability density functions.

In general, given a completely heterogeneous and anisotropic microstructure, the conditional statistics of chord-length density on some particular sampling locations and orientations allow one to quantify the microstructure quantitatively, thus enabling quantitative comparison between one microstructure to another.

3.4 Measures of differences between microstructures via microstructure descriptors

Through microstructure descriptors, one can impose distance metrics to measure the difference between microstructure descriptors of the two microstructures, e.g target and sample microstructure. If the microstructure descriptor is deterministic, some mathematical metrics, such as

-norm, can be utilized to measure the difference. If the microstructure descriptor is statistical and can be represented as a probability density function, some statistical metrics and divergences, for example, Wasserstein distance and Kullback-Leibler divergence, can be utilized to measure the difference between probability density functions. In particular, if the Kullback-Leibler divergence is utilized to measure the statistical difference

, then the statistical difference between microstructures for a specific microstructure is computed as


where KL denotes the Kullback-Leibler divergence and is the microstructure descriptor.

3.5 Objective functions in optimization

The differences between these microstructure descriptors are then considered as the objective functions, i.e. where the goal is to minimize objectives, , simultaneously. Even though a high-dimensional probability density function can be formed by combining several microstructure descriptors together, in this framework, we opt to treat them separately by forming a one-to-one map between microstructure descriptor and objective function. It is noteworthy that these objective functions are typically noisy due to the stochastic nature of microstructure, particularly in polycrystalline materials systems. As a result, the problem of microstructure calibration can be cast into a noisy multi-objective optimization problem, which can be solved numerically and efficiently on high-performance computing platforms.

3.6 An active learning high-throughput Bayesian optimization framework

Active learning is an area of machine learning that studies the optimal strategies for allocating finite resources. The main motivation of active learning approaches is to intelligently select informative points at which the forward objective function should be queried. Often in practice, the same accuracy of a supervised machine learning model is achieved, while the size of the training dataset is significantly reduced settles2009active ; settles2011theories . In this context, the Bayesian optimization method arises and is widely regarded as an active learning approach, by balancing the trade-offs between exploration and exploitation. On one hand, the exploration advocates for sampling at the least unknown region in the input space. On another hand, the exploitation advocates for quick convergence in the best-so-far sample in the training dataset. Balancing exploration and exploitation is the key element of the Bayesian optimization to avoid being trapped in local optima through exploration and quickly converge to the global optima through exploitation. In the Bayesian optimization method, the trade-off between exploration and exploitation is embedded and reconciled in the acquisition function.

In order to further exploit the available computational resources and expedite the optimization process, multiple instances of the simulations are queried concurrently in a high-performance parallel computing environment to minimize the physical waiting time. Parallel optimization methods can be utilized to query the input

concurrently to accelerate the optimization procedure. For meta-heuristic optimization methods, Memetic et al.

memeti2019using and Mahdavi et al. mahdavi2015metaheuristics provide a comprehensive literature reviews of meta-heuristic optimization methods on high-performance and cloud platforms.

We discuss the high-throughput optimization method based on surrogate models, in particular, Gaussian process model, in enabling asynchronously parallel queries. The Gaussian process model is a powerful and widely used tool for approximating black-box functions. Bayesian optimization is a natural extension of a Gaussian process model to globally optimize an unknown black-box function, which is assumed to be smooth. By implementing an acquisition function to balance between exploitation (i.e., focusing on promising regions) vs. exploration (i.e., exploring uncertain regions), the traditional Bayesian optimization framework can rely on the Gaussian process model, as a surrogate model, to search for the global optimal point effectively. However, the traditional Bayesian optimization is restricted to sequential query, in which only one input can be queried at once.

This limitation is indeed a burden on high-performance computing platform, where multiple inputs can be theoretically queried at the same time. In our previous work, we have extended the traditional Bayesian optimization framework into synchronously batch-sequential parallel Bayesian optimization tran2019pbo for constrained optimization problems, called pBO-2GP-3B, where the constraints are generalized to cover a broad spectrum of applications by considering both known and unknown constraints. We deployed a further improved implementation of pBO-2GP-3B, called aphBO-2GP-3B tran2020aphbo , to asynchronously parallelize on the high-performance computing environment with multiple acquisition functions considered. Readers are referred to our previous work tran2019pbo ; tran2019constrained ; tran2019sbfbo2cogp ; tran2020smfbo2cogp ; tran2020weargp ; tran2018weargp ; tran2018efficient and others brochu2010tutorial ; shahriari2016taking ; frazier2018tutorial ; jones1998efficient for rigorous literature reviews on Gaussian process model and Bayesian optimization methods and its variants.

Regarding multi-objective optimization problems, Alexandropoulos et al. alexandropoulos2019multi and Li et al. li2015many provide a comprehensive literature reviews on solving multi-objective optimization problems, where the goal is to identify the Pareto frontier. Multi-objective optimization problems are typically solved by converting a multi-objective problem to a single-objective problem, where the single objective is a weighted sum of multiple objectives rojas2019survey , such as weighted Chebyshev scalarization function ( represents the ideal value for the objective ), augmented Chebyshev scalarization function , and the weighted sum scalarization function: .

While there is no restriction on the optimization method that can be used to minimize the difference between the candidate and the target microstructures, in this work, we use the aphBO-2GP-3B Bayesian optimization framework, which is an asynchronously parallel constrained multi-acquisition Bayesian optimization algorithm to further improve its efficiency on high-performance computing platform. The aphBO-2GP-3B subdivides the computational budget into three batches, supported by two Gaussian process models. One Gaussian process model is used to model the objective function, whereas another Gaussian process model

is used as a probabilistic binary classifier for hidden constraints. The first batch focuses on optimizing the objective functions by sampling at the locations where the acquisition function value of the Bayesian optimization is maximized. The second batch focuses on the exploration, by sampling at the locations where the posterior variance is maximized. The third batch focuses on the feasibility classification to learn hidden constraints from the optimization problem.

4 Case study #1: kMC simulation for additive manufacturing and Bayesian optimization

In this section, we demonstrate our proposed framework through a case study that combines an advanced extension of the Bayesian optimization method and a kMC model for additive manufacturing.

4.1 Kinetic Monte Carlo simulation

We adopt the kMC simulation using SPPARKS plimpton2009crossing ; plimpton2012spparks , where the welding model is developed by Rodgers et al. rodgers2016predicting ; rodgers2017monte ; rodgers2017simulation to predict the microstructure in welding. For the sake of completeness of the work, we briefly summarize the kMC used as follows. The model simulates melting, solidification, and solid-state microstructural evolution of regions surrounding the weld pool. Grain evolution is modeled spatially with three different regions, the base metal, the heat-affected zone, and the fusion zone. The base metal is unaffected by the weld pool and thus retains its microstructure. In the heat-affected zone, the temperature evolves and induces microstructural evolution, but the temperature in this zone does not exceed the melting temperature. In the fusion zone, the temperature is higher than the melting temperature, thus material is completely melted. The weld pool geometry is modeled using Bézier curves to capture a wide range of pool shapes.

The grain evolution is modeled using a modified Potts Monte Carlo model to capture the process temperature profile as a function of position and time, which is obtained by translating the modeled weld pool and temperature gradient through the simulation domain. Grain growth is simulated by the normal curvature-driven grain growth model with grain boundary mobility as a function of temperature. In this curvature-driven grain growth model, grain boundary energy is the driving force for grain growth and evolution. Grain growth is simulated by minimizing the total grain boundary energy, calculated as the sum of all bonds between neighboring sites belonging to two different grains, multiplied by the bond energy. Figure 2 presents two samples of welding microstructure with the kMC model in final state, where each microstructure corresponds to a different set of processing parameters.

(a) Sample 1 (teardrop welding pool shape).
(b) Sample 2 (ellipse welding pool shape).
Figure 2: Two samples of microstructure produced by kMC simulation via SPPARKS.

The input for the 2D kMC simulation is the velocity, the dimension of the heat-affected zone, and the width of the melting pool. For 3D kMC simulations, two more parameters modeling the geometric shape of the welding pool can be considered. Table 1 lists the input parameters used to generate the target microstructure in this case study, which closely matches with the work of Rodgers et al. rodgers2016predicting ; rodgers2017monte ; rodgers2017simulation . The dimension parameters , , and are assumed to be constant and thus are fixed during the optimization process.

Input Physical description Unit Lower bound Upper bound
width of the microstructure site 805 805
length of the microstructure site 1575 1575
height of the microstructure site 1 1
velocity site/MCS 15 30
heat-affected zone site 120 200
pool width (teardrop welding pool) site 50 250
Table 1: Input parameters of the kMC/SPPARKS simulation with teardrop shape welding pool.

4.2 Statistical microstructure descriptors as outputs

In this case study, multiple statistical physics-based microstructure descriptors are used to quantify the kMC simulated microstructures. In the kMC simulation, the microstructure is represented as a “spin”, which is analogous to the identification number for grains. Sites which correspond to the same “spin” belong to the same grain. Using this representation, the grains are segmented, where the grain boundaries is captured. The whole microstructure is considered as an ensemble of grains, where the probability density functions are inferred based on the grain observations.

Notation Related quantities Physical meaning Type Nature Region
: major dim. of best fit ellipse dist. of grain-based statistic global
: minor dim. of best fit ellipse dist. of grain-based statistic global
: ellipse orientation dist. of grain-based statistics global
grain area dist. of grain area grain-based statistics global
chord-length in - direction dist. of chord-length grain-based statistics global
chord-length in - direction dist. of chord-length grain-based statistics global
local chord-length dist., band 0 dist. of chord-length grain-based statistics local
local chord-length dist., band 1 dist. of chord-length grain-based statistics local
local chord-length dist., band 2 dist. of chord-length grain-based statistics local
local chord-length dist., band 3 dist. of chord-length grain-based statistics local
local chord-length dist., band 4 dist. of chord-length grain-based statistics local
Table 2: List of microstructure descriptors, their nature, and physical meaning.

The microstructure descriptors used are listed in Table 2, which contains 11 microstructure descriptors. The microstructure descriptors associated with the best fit ellipse are described as follows. Each grain is approximated using a best fit 2D ellipse. The parameters describing the ellipse are then used as physics-based microstructure descriptors, including the dimensions of major and minor axes, the orientation, the coordinates of the best fit ellipse for the grain, as well as the grain area. The best fit ellipse is found by minimizing the total square error, which measures the difference between the grain boundary and the ellipse. The grain boundary is obtained by segmenting the grain from the microstructure. Figure 3 describes the process of fitting an ellipse to the grain, and demonstrates it using a representative grain (Figure 2(b)) in the microstructure (Figure 2(a)).

(a) A rasterized ms. representation with dimension.
(b) Grain 16166 in the microstructure shown in Figure 2(a).
(c) Best fit ellipse (rotated ccw) (Figure 2(b)).
Figure 3: Fitting ellipse process for a single grain in a microstructure. For each grain, the best fit ellipse is found by minimizing the total least square error. The distributions of the major dimension , minor dimension , locations of the center , orientation of the ellipses , are used to characterize the microstructure. There are also other microstructure descriptors.
Figure 4: Local sampling chord-length distribution along welding direction (-direction) for the microstructure shown in Figure 2(a). Input parameters include the width of the bands, the spacing between bands, and the number of the bands. Distributions are assumed to be symmetric across the welding axis (denoted as green arrow).

Since the microstructure descriptor choices are left to the users, it is possible to choose local microstructure descriptors in such a way so that the microstructures can be compared locally, e.g. by fixing at a particular location or angle. As a result of the local microstructure descriptors, the heterogeneous and anisotropic behaviors can be captured. Because the welding microstructure is translationally invariant along the welding axis, theoretically, the density of the chord-length does not change when it is sampled at any -location, as in Figure 4. However, the chord-length statistics change drastically, depending on the -location. It is reasonable to assume symmetricity across the welding axis, but shifting up or down would affect the chord-length statistics. Thus, the -location dependence allows one to capture the heterogeneity of the microstructure.

In Figure 4, the local chord-length statistics are obtained through the horizontal bands, which are imposed in parallel with the welding axis. The width of the band, the spacing between bands, and the number of bands are used to generate the bands. When one microstructure is compared with another microstructure, the chord-length distribution for each band is built and compared individually with another. This approach allows one to compare microstructure locally, and thus also allows one to compare microstructure in a heterogeneous and anisotropic manner. For example, the directions and locations of the bands can change to capture the most distinguished microstructure features.

4.3 Conditional statistical microstructure descriptors by imposing filters

Before comparing the microstructure quantitatively, it is important to eliminate some of the microstructure attributed by the base metal. In general, the base metal contributes most of the grains, which are small, in the grain population. When comparing two microstructures, if the base metal grains are not excluded, then the two microstructures are very similar because their base metal grains are similar and the base metal grains is the majority of the microstructure. This effect is undesirable, as the base metal grains are considered as background and not the main focus when comparing two microstructures. Thus, it is necessary to exclude the base metal microstructure. Since the base metal grains are small, a grain area filter is used to exclude these grains.

Figure 5 shows the effects of a low and high threshold in filtering out the grains on a representative microstructure before the probability density functions of the statistical microstructure descriptors are computed. The number of samples decreases monotonically, as 4624 grains, 2672 grains, 1546 grains, 968 grains, 659 grains, as the threshold on grain area increases from 50, 100, 150, 200, and 250 pixel. When filtering on grain area is disabled, the probability density function is heavily shifted to the left, because the grain population associated with the base metal region is dominant, compared to the grain population associated with the heat-affected zone and the melting zone.

It is straightforward to see that the difference metric, i.e. the scalarized single-objective function, and noise increase with increasing threshold. On one hand, if the threshold is too large, the statistical descriptors are noisy because there might not be enough samples to approximate a probability density function

, thus resulting in noisy objectives, and therefore the multi-objective optimization problem becomes very challenging to solve due to the noise. On the other hand, if the threshold is too small, for example, no threshold, the statistical descriptors are close to each other, and the differences between objectives are almost at the same magnitude as the noise, and again, the optimization problem is too noisy to solve robustly. Thus, there is an optimal region of threshold that induces the highest signal-to-noise ratio. This optimal threshold is hard to find, but can be roughly estimated based on the preliminary examination of the microstructures. As the main goal is to eliminate the effect of base metal microstructure, the threshold can be slightly larger than the expectation of the physical descriptors of the base metal microstructure. The signal-to-noise ratio does not have to achieve a maximum number in order to utilize the proposed framework.

Therefore, it may be necessary to filter most of the base metal microstructure since the effects of scale separation are fairly severe. In this work, a grain area filter is implemented. If the grain area is larger than the threshold, then it is considered as a sample in the population. Otherwise, it is simply disregarded when microstructure descriptors are computed. The threshold of 150 pixel is chosen by analyzing its effect on the studied microstructures.

Figure 5: Approximated conditional probability density function of grain area (a microstructure descriptor) with various thresholds on grain area to filter grains. If the threshold is large, the number of samples (grains) decreases, resulting in a higher approximation error. If the threshold is low, it is hard to quantitatively distinguish the candidate and target microstructures, due to the dominant portion of samples from the base metals.

4.4 Quantifying aleatory uncertainty of the objectives for the target microstructure

Associated Obj. Fun. Mean Variance
0.0112 3.7129e-03
0.0157 1.0870e-02
0.0002 6.7116e-05
0.0100 2.3328e-03
0.0065 3.4961e-03
0.0029 1.1344e-03
0.0114 1.0709e-02
0.0024 1.7863e-03
0.0065 9.2228e-04
0.0037 1.2436e-03
0.0098 4.1146e-03
Table 3: Quantified aleatory uncertainty of each single-objective that corresponds with the microstructure descriptors described in Table 2.

To characterize the intrinsic noise of the kMC simulation, we sample the kMC simulation 25 times for the same set of input parameters corresponding to the target microstructure. For each time, a different seed in the pseudo-random number generator is chosen. We then compare the in the same manner with Equation 3.4; however, in this case, the sets of input parameters of sampleTargetMs and targetMs are identical. Table 3 lists the mean (i.e., ) and variance (i.e. ) for 11 objective functions across 25 samples. It is noteworthy to point out that the noise associated the objective functions is fairly small, resulting in an acceptably large signal-to-noise ratio, so that the noise does not have a significant impact on the optimization process. Summing all the means and variances of each individual microstructure descriptors, the mean and the variance of the single-objective are 0.0809 and , respectively.

4.5 Noisy asynchronously-parallel multi-objective Bayesian optimization

After the 11 probability density functions of microstructure descriptors are collected, the Kullback-Leibler divergence, described in Equation 3.4, is utilized to measure the difference between the candidate and the target microstructures. To convert from a multi-objective optimization problem to a single-objective optimization problem, multi-objective functions are simply summed to form a single-objective . The parallel Bayesian optimization aphBO-2GP-3B tran2020aphbo framework is utilized, with 20 concurrent kMC simulations utilized for the first batch of the framework, while 5 concurrent kMC simulations are used to explore the input space of the processing parameters. Because all the simulations are feasible and there are no hidden constraints, the size of the third batch is set as zero, since no classification effort is needed. Python scripts are devised to quantify the microstructure descriptors from SPPARKS dump files. Scipy toolbox scipy is used to reconstruct the kernel density estimation. 3965 samples are collected at the end of this study, which takes roughly about 7-8 days of simulations. The Bayesian optimization package is prototyped in MATLAB, and interfaces between Python, MATLAB, and SPPARKS are constructed so that the case study can run continuously.

4.6 Optimal results

Figure 6 shows the convergence plot of the optimization problem, where the single-objective is minimized as the optimization process advances. For the kMC welding case study, the following sampling points are used through Monte Carlo sampling to approximate the initial Gaussian process model response: (16.0, 135.0, 180.7), (18.0, 134.0, 165.2), (25.0, 145.0, 148.0), (29.0, 175.0, 155.2), (29.0, 169.0, 237.7), (19.0, 179.0, 103.50), (30.0, 200.0, 224.0). Three optimization runs are carried out to demonstrate the robustness of the microstructure calibration framework, where two out of these three runs last about 400 iterations for the purpose of demonstration, while the main one advances 4000 iterations. Here, one iteration corresponds to a single kMC simulation with different parameters. The converged objectives with respect to iterations are plotted in the blue envelope, representing the upper and lower bounds of objectives. In the following text, we will carefully analyze the main optimization run of 4000 iterations. Each simulation is performed on a computing cluster with the SLURM scheduler, where 36 cores are utilized on one node. On the average, the computational cost of one kMC simulation is around 9 CPU hours. 25 simulations are performed concurrently to reduce the physical waiting wall-time. The single-objective starts at the value of 0.29981499 at iteration 1, and subsequently converges to 0.15940848 at iteration 5, 0.14780493 at iteration 21, 0.12549670 at iteration 75, 0.09523728 at iteration 142, 0.08577485 at iteration 212, 0.07945405 at iteration 1762, and 0.07921463 at iteration 3864. Compared to the quantified aleatory uncertainty of the target microstructure in Table 3, , the optimal objective is less than the quantified aleatory uncertainty, i.e. . It is obvious to observe that the uncertainty of the single-objective has reached the quantified aleatory uncertainty; thus, it is unlikely to achieve better. Due to the low-dimensional problem, which contains only 3 parameterized inputs, the optimization has almost converged at iteration 212. The convergence of the optimization can be observed from the beginning to iteration 212. We conclude that the optimal microstructure is comparable to the target microstructure.

Figure 6: Convergence plot of the microstructure calibration framework in kMC/welding problem. The objective is minimized as the optimization process advances.

Figure 6(a) and Figure 6(b) show the optimal microstructure, obtained in iteration 3864, and the target microstructure, respectively. The inputs of the target microstructure are , as described in Table 1, whereas the inputs of the optimal microstructure are . As observed in Figure 7, the dimension of the welding pool is accurately identified, as well as the velocity of the welding pool. Thus, we conclude that the microstructures in Figure 6(a) and in Figure 6(b) are (almost) statistically equivalent.

(a) Optimal microstructure (Iteration 3864).
(b) Target microstructure.
Figure 7: Comparison between optimal and target microstructures.

To further study the relationship between 11 objectives , their correlations are investigated by reconstructing the joint density between and . Figure 8 shows the correlation between 11 objectives , where the lower part of the plot shows the joint density between and for , and the upper part of the plot shows the scatter plot between and . The diagonal of the plot shows the density of .

Figure 8: Scatterplot matrix between objectives (where the microstructure descriptor associated with each objective is listed in Table 2 with the same order) using 3965 samples collected at the end of the optimization process. The lower part of the plot shows the joint density between and for , and the upper part of the plot shows the scatter plot between and . The diagonal of the plot shows the density of .

Figure 8(a) shows the quantified correlation coefficient between two objectives. Figure 8(b) shows the magnified plot of the lower part in Figure 8. Most of the microstructure descriptors are very strongly correlated, except for the microstructure descriptor, which characterizes the dimension of minor axis of the best fit ellipse for the grain. In our experience, even though many microstructure descriptors are strongly correlated with others, including more descriptors does help in the optimization process.

(a) Diagonal correlation matrix between objectives , where the coefficient of correlation is shown at the center of the square.
(b) Magnified plot of the lower part in Figure 8, which shows the scatterplot matrix between objectives.
Figure 9: Correlation matrix with the coefficient of correlation and the joint density function between 11 objectives listed in Table 2 with the same order. Most of them are strongly correlated with each others, while is nearly uncorrelated with the rest of the objectives.

5 Case study #2: kMC simulation for grain growth and Bayesian optimization

In this section, we demonstrate the applicability of our microstructure calibration framework using another kMC simulation for grain growth problem, where a numerical temperature parameter is the input, and the microstructure is the output. The numerical temperature parameter is successfully recovered via our microstructure calibration framework.

5.1 Kinetic Monte Carlo simulation

The details of temperature-dependent kMC simulation for grain growth and its implementation in SPPARKS is described in Garcia et al garcia2008three , and is summarized here for the sake of completeness. In the grain growth simulation, the Potts model wu1982potts is used to simulate curvature-driven grain growth. The Arrhenius equation describes the relationship between grain boundary mobility and temperature as


where is the Boltzmann constant, is the Arrhenius prefactor. The Metropolis algorithm is used to determine the probability of successful change in grain site orientation as


where is the total grain boundary energy calculated by summing all the neighbor interaction energies, can be regarded as the activation, and is the simulation temperature. It is worthy to note that the simulation temperature is not the real system temperature: is an energy that defines the thermal fluctuation, i.e. noise, presented in the kMC simulation garcia2008three .

5.2 Statistical microstructure descriptors as outputs

Figure 10: Effect of the numerical temperature in the kMC grain growth problem.

Figure 10 shows the effect of on different grain growth microstructure. In particular, larger is associated with microstructures with smaller grain size for (nearly) the same amount of time. This suggests that the grain size distribution may be a good microstructure descriptor to describe and distinguish one microstructure from another.

In this case study, the distribution of grain area ( in Table 2) is used as the microstructure descriptor, where the Kullback-Leibler divergence measuring the distance between candidate and target microstructures is used as an output, where is the numerical input describing simulation temperature. Because the number of grains are fairly small (few thousands), no filter is imposed on the statistical microstructure descriptors. In the target microstructure, is set as 0.70, where the range of candidate microstructures is . The size of the simulation domain is 10241024, which takes about 300s with 18 processors ( CPU hours).

5.3 Optimal results

Three optimization runs are carried out to demonstrate the robustness of the microstructure calibration framework for grain growth problem in kMC, where each last about 40-50 runs to obtain the correct target microstructure with . The upper and lower bounds of objectives for three optimization runs are plotted as blue envelope in Figure 11. Three sampling points at are initialized. Four kMC simulations are performed concurrently across a high-performance computing platform. Among these three runs, we analyze thoroughly one run. Here, one iteration corresponds to a single kMC simulation with different parameters. The single-objective starts at the the value of 0.03965973 at iteration 1, and subsequently converges to the value of 0.01092325 at iteration 4, the value of 0.00776526 at iteration 8, the value of 0.00512348 at iteration 31, the value of 0.00408244 at iteration 33, and the value of 0.00000000 at iteration 38. The optimal input parameter is 0.70393395, which agrees very well with the target input parameter of 0.70.

Figure 11: Convergence plot of the microstructure calibration framework in kMC/grain growth problem. The objective is minimized as the optimization process advances.

6 Discussion

Microstructure descriptors are the key aspect of the microstructure calibration problem. First, the population of the microstructure descriptors is limited; for example, the number of samples is limited by the number of grains. Increasing the number of grains by increasing specimen size helps to reduce the approximation error in reconstructing the probability density function. Second, the microstructure in itself is also stochastic, where noise is due to an underlying aleatory uncertainty. Depending on specific problems, the selection (i.e. inclusion or exclusion) of microstructure descriptors may positively or negatively affect the efficiency of the microstructure calibration framework. In our experience, including as many microstructure descriptors as possible may yield a better results, because some microstructure descriptors are capable of quantitatively distinguishing microstructures and some are not. Without any prior knowledge regarding the performance of the microstructure descriptors, it may be beneficial to include as many microstructure descriptors as possible. If the microstructure descriptor is not capable of distinguishing microstructure, it may be regarded as a noise in the objective function. Therefore, the microstructure calibration framework provides a mathematically sound and general approach to solve the inverse structure-process problems in the sense that it would always seek for the global optimal point, which corresponds to processing parameters of the target microstructure, given any predictive forward ICME model.

As shown previously, it is desirable to quantitatively distinguish microstructures. In the best scenario, the microstructure descriptors should be uncorrelated, so that they can fully characterize the microstructure. However, in practice, this is typically not the case; for example, grain area should correlate at least fairly well with chord-length, since larger grains have longer chord-lengths and larger grain areas and vice versa. One of the advantages of the microstructure calibration framework is that microstructures can be compared quantitatively and not just qualitatively. In many cases, the human perception for qualitative comparison may be intuitively misleading and biased. Although the human eye may not be able to distinguish between these intermediate microstructures, they do have different microstructure features and are associated with different microstructure descriptors. Microstructures may be quantitatively different although they may look qualitatively similar. Quantitative comparison of microstructures through microstructure descriptors is therefore a robust measure to distinguish microstructures.

A weighted formulation can be applied to convert multi-objective to single-objective problems. The weights can be adaptively computed based on cross-correlations, as well as the spatial gradients of microstructure descriptors. Choosing weights and norms, such as , , or , to promote the difference between microstructures is a non-trivial research question and requires further study. In the first case study, the hyper-parameter bandWidth (as shown in Figure 4) used to locally capture the microstructure descriptor cannot be too large, because the homogeneous assumption is made across each band. This parameter should also not be too small, because it would lead to a small number of grains, thus resulting in a large approximation error in the approximated probability density function. The hyper-parameters for the microstructure descriptors to characterize the microstructure are chosen in an ad-hoc manner. A more generalized framework for adaptively selecting microstructure descriptors may be helpful and more general.

For additive manufacturing applications, if the tool path can be parameterized, a similar approach for microstructure calibration can be adopted to identify the tool path. The parameters generating the G-code, which defines the geometric model in additive manufacturing, can also be included as a part of the inputs, and the same microstructure calibration framework can be deployed to search for the target set of processing parameters, which includes the parameters to generate G-code and thus tool path.

For 3D problems, a similar approach can be employed by generalizing the best fit ellipse algorithm in 2D to the best fit ellipsoid algorithm in 3D. Such approach requires a stable algorithm for fitting the ellipsoid in 3D and thus is posed as a future work. We also point out that 3D problems, in general, are much more complicated than 2D problems, in term of both computational complexity and microstructure descriptors. This topic is therefore posed as a potential future research.

In this work, each microstructure descriptor is treated individually. However, a high-dimensional density function can also be adopted to the same formulation, by considering the joint probability density function of a set of microstructure descriptors. It is noteworthy that the approximation of the probability density function, in general, scales poorly with dimensionality. Further, the selection of microstructure descriptors is a non-trivial research question, which can be also considered in future work. For example, several groups of microstructure descriptors can be formed, where each group is associated with a high-dimensional probability density function, and subsequently an objective function in the multi-objective optimization framework. This interesting idea remains to be explored in future research.

Since the microstructure is stochastic in nature, the microstructure calibration framework can be aligned with optimization under uncertainty. In particular, for a specific set of input parameters, multiple samples can be obtained to quantify aleatory uncertainty of microstructure descriptors. With experimental data, a multi-fidelity approach, such as CoKriging tran2019sbfbo2cogp ; tran2020smfbo2cogp , can be adopted to fuse predictions of computational and experimental data at the level of high-fidelity tran2019data

. For inputs with discrete and categorical variables, a mixed-integer Bayesian optimization approach

tran2019constrained can be utilized to search for the processing parameters, using the same formulation in this work. Multi-objective Bayesian optimization methods shu2020new ; tran2020srmobo3gp can be deployed to find the Pareto frontiers, which would provide computational insights for selecting objectives and weights for scalarization-based multi-objective Bayesian optimization approaches. Batch parallel Bayesian optimization methods, including both asynchronously parallel tran2020aphbo and batch-sequential parallel tran2019pbo , can be used to accelerate the optimization process. For some ICME applications where the intrinsic variance for microstructure descriptors is quite large and varies dependently with respect to the input parameters, more rigorous approaches, such as stochastic Gaussian process model (or stochastic kriging) ankenman2010stochastic ; chen2017sequential and composite Gaussian process model ba2012composite , would be more suitable.

It is noteworthy to point out that without parallelizing the Bayesian optimization by running 25 simulations concurrently, the sequential Bayesian optimization approach would still be at around iteration 200, while the parallel Bayesian optimization has already advanced to iteration 5000. For high-dimensional optimization problems, the improvement in convergence is more significant. The high-throughput feature in this framework is enabled by both parallelisms in optimization and in simulations.

The microstructure calibration framework, generally, is not restricted to the process-structure linkage, but also is highly applicable to model calibration applications, when model parameters are inferred based on the similarity of microstructures. Such problems are common in phase-field simulations, where thermodynamics parameters are often computed or fitted using some computational thermodynamic package xiong2010phase . The microstructure calibration framework provides an alternative and viable solution to such model calibration problems, where microstructures are considered as objective functions.

7 Conclusion

In this work, a microstructure calibration framework is proposed to efficiently solve the inverse problem in process-structure linkage. A set of microstructure descriptors is used to characterize and quantitatively compare microstructures. By measuring differences of microstructure descriptors densities, the inverse problem is formulated as a noisy multi-objective optimization problem. To that end, a parallel Bayesian optimization framework is utilized to minimize differences. As a result, the optimal microstructure is obtained and compared with the target microstructure. Two case studies are demonstrated, where kMC simulations are employed in forward prediction. In both cases, the input parameters associated with the target microstructure are successfully recovered, thus demonstrating the usefulness of the proposed framework.


A.T. thanks Theron Rodgers (SNL) for numerous helpful discussions. The authors are grateful to the anonymous reviewer for his or her constructive comments. The views expressed in the article do not necessarily represent the views of the U.S. Department of Energy or the United States Government. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA-0003525. This research was supported by the U.S. Department of Energy, Office of Science, Early Career Research Program, under award 17020246, and by Sandia LDRD program 219144.


  • (1) S. R. Kalidindi, A. J. Medford, D. L. McDowell, Vision for data and informatics in the future materials innovation ecosystem, JOM 68 (8) (2016) 2126–2137.
  • (2) N. Science, T. C. (US), Materials Genome Initiative for global competitiveness, Executive Office of the President, National Science and Technology Council, 2011.
  • (3) D. G. Backman, D. Y. Wei, D. D. Whitis, M. B. Buczek, P. M. Finnigan, D. Gao, ICME at GE: accelerating the insertion of new materials and processes, JOM 58 (11) (2006) 36–41.
  • (4) B. Chaparro, S. Thuillier, L. Menezes, P.-Y. Manach, J. Fernandes, Material parameters identification: Gradient-based, genetic and hybrid optimization algorithms, Computational Materials Science 44 (2) (2008) 339–346.
  • (5) B. C. Salzbrenner, J. M. Rodelas, J. D. Madison, B. H. Jared, L. P. Swiler, Y.-L. Shen, B. L. Boyce, High-throughput stochastic tensile performance of additively manufactured stainless steel, Journal of Materials Processing Technology 241 (2017) 1–12.
  • (6) S. P. Ong, Accelerating materials science with high-throughput computations and machine learning, Computational Materials Science 161 (2019) 143–150.
  • (7) S. B. Sinnott, Material design and discovery with computational materials science, Journal of Vacuum Science & Technology A: Vacuum, Surfaces, and Films 31 (5) (2013) 050812.
  • (8) R. Arróyave, D. L. McDowell, Systems approaches to materials design: Past, present, and future, Annual Review of Materials Research 49.
  • (9) G. Olson, Genomic materials design: the ferrous frontier, Acta Materialia 61 (3) (2013) 771–781.
  • (10) B. L. Adams, S. Kalidindi, D. T. Fullwood, Microstructure sensitive design for performance optimization, Butterworth-Heinemann, 2012.
  • (11) D. T. Fullwood, S. R. Niezgoda, B. L. Adams, S. R. Kalidindi, Microstructure sensitive design for performance optimization, Progress in Materials Science 55 (6) (2010) 477–562.
  • (12) T. M. Rodgers, J. D. Madison, V. Tikare, M. C. Maguire, Predicting mesoscale microstructural evolution in electron beam welding, JOM 68 (5) (2016) 1419–1426.
  • (13) T. M. Rodgers, J. A. Mitchell, V. Tikare, A Monte Carlo model for 3D grain evolution during welding, Modelling and Simulation in Materials Science and Engineering 25 (6) (2017) 064006.
  • (14) W. Li, M. Soshi, Modeling analysis of the effect of laser transverse speed on grain morphology during directed energy deposition process, The International Journal of Advanced Manufacturing Technology (2019) 1–13.
  • (15) J. Zhang, F. Liou, W. Seufzer, K. Taminger, A coupled finite element cellular automaton model to predict thermal history and grain morphology of Ti-6Al-4V during direct metal deposition (DMD), Additive Manufacturing 11 (2016) 32–39.
  • (16) A. Rai, H. Helmer, C. Körner, Simulation of grain structure evolution during powder bed based additive manufacturing, Additive Manufacturing 13 (2017) 124–134.
  • (17) R. Acharya, J. A. Sharon, A. Staroselsky, Prediction of microstructure in laser powder bed fusion process, Acta Materialia 124 (2017) 360–371.
  • (18) Y. Ji, L. Chen, L.-Q. Chen, Understanding microstructure evolution during additive manufacturing of metallic alloys using phase-field modeling, in: Thermo-Mechanical Modeling of Additive Manufacturing, Elsevier, 2018, pp. 93–116.
  • (19) F. Yu, Y. Wei, Y. Ji, L.-Q. Chen, Phase field modeling of solidification microstructure evolution during welding, Journal of Materials Processing Technology 255 (2018) 285–293.
  • (20) A. V. Tran, D. Liu, H. A. Tran, Y. Wang, Quantifying uncertainty in the process-structure relationship for Al-Cu solidification, Modelling and Simulation in Materials Science and Engineering 27 (6) (2019) 064005.
  • (21) L. Cao, D. Liu, P. Jiang, X. Shao, Q. Zhou, Y. Wang, Multi-physics simulation of dendritic growth in magnetic field assisted solidification, International Journal of Heat and Mass Transfer 144 (2019) 118673.
  • (22) D. Liu, Y. Wang, Mesoscale multi-physics simulation of rapid solidification of Ti-6Al-4V alloy, Additive Manufacturing 25 (2019) 551–562.
  • (23) Y. C. Yabansu, P. Steinmetz, J. Hötzer, S. R. Kalidindi, B. Nestler, Extraction of reduced-order process-structure linkages from phase-field simulations, Acta Materialia 124 (2017) 182–194.
  • (24) D. B. Brough, D. Wheeler, J. A. Warren, S. R. Kalidindi, Microstructure-based knowledge systems for capturing process-structure evolution linkages, Current Opinion in Solid State and Materials Science 21 (3) (2017) 129–140.
  • (25) E. Popova, T. M. Rodgers, X. Gong, A. Cecen, J. D. Madison, S. R. Kalidindi, Process-structure linkages using a data science approach: application to simulated additive manufacturing data, Integrating Materials and Manufacturing Innovation 6 (1) (2017) 54–68.
  • (26) P. Jiang, C. Wang, Q. Zhou, X. Shao, L. Shu, X. Li, Optimization of laser welding process parameters of stainless steel 316L using FEM, Kriging and NSGA-II, Advances in Engineering Software 99 (2016) 147–160.
  • (27) H. W. Park, Y. B. Park, S. Y. Liang, Multi-procedure design optimization and analysis of mesoscale machine tools, The International Journal of Advanced Manufacturing Technology 56 (1-4) (2011) 1–12.
  • (28) Y. Zhang, P. Zou, B. Li, S. Liang, Study on optimized principles of process parameters for environmentally friendly machining austenitic stainless steel with high efficiency and little energy consumption, The International Journal of Advanced Manufacturing Technology 79 (1-4) (2015) 89–99.
  • (29) S. Wang, Z. Jia, X. Lu, H. Zhang, C. Zhang, S. Y. Liang, Simultaneous optimization of fixture and cutting parameters of thin-walled workpieces based on particle swarm optimization algorithm, Simulation 94 (1) (2018) 67–76.
  • (30) S. Pfeifer, B. S. S. Pokuri, P. Du, B. Ganapathysubramanian, Process optimization for microstructure-dependent properties in thin film organic electronics, Materials Discovery 11 (2018) 6–13.
  • (31) P. Fernandez-Zelaia, S. N. Melkote, Process-structure-property relationships in bimodal machined microstructures using robust structure descriptors, Journal of Materials Processing Technology.
  • (32) J. H. Forsmark, J. W. Zindel, L. Godlewski, J. Zheng, J. E. Allison, M. Li, Using quality mapping to predict spatial variation in local properties and component performance in Mg alloy thin-walled high-pressure die castings: an ICME approach and case study, Integrating Materials and Manufacturing Innovation 4 (1) (2015) 6.
  • (33) P. Nath, Z. Hu, S. Mahadevan, Uncertainty quantification of grain morphology in laser direct metal deposition, Modelling and Simulation in Materials Science and Engineering 27 (4) (2019) 044003.
  • (34) J. Ling, M. Hutchinson, E. Antono, S. Paradiso, B. Meredig, High-dimensional materials and process optimization using data-driven experimental design with well-calibrated uncertainty estimates, Integrating Materials and Manufacturing Innovation 6 (3) (2017) 207–217.
  • (35) A. Paul, P. Acar, W.-k. Liao, A. Choudhary, V. Sundararaghavan, A. Agrawal, Microstructure optimization with constrained design objectives using machine learning-based feedback-aware data-generation, Computational Materials Science 160 (2019) 334–351.
  • (36) L. Johnson, R. Arróyave, An inverse design framework for prescribing precipitation heat treatments from a target microstructure, Materials & Design 107 (2016) 7–17.
  • (37) T. Fast, M. Knezevic, S. R. Kalidindi, Application of microstructure sensitive design to structural components produced from hexagonal polycrystalline metals, Computational Materials Science 43 (2) (2008) 374–383.
  • (38) M. W. Priddy, N. H. Paulson, S. R. Kalidindi, D. L. McDowell, Strategies for rapid parametric assessment of microstructure-sensitive fatigue for HCP polycrystals, International Journal of Fatigue 104 (2017) 231–242.
  • (39) N. H. Paulson, M. W. Priddy, D. L. McDowell, S. R. Kalidindi, Data-driven reduced-order models for rank-ordering the high cycle fatigue performance of polycrystalline microstructures, Materials & Design 154 (2018) 170–183.
  • (40) N. H. Paulson, M. W. Priddy, D. L. McDowell, S. R. Kalidindi, Reduced-order microstructure-sensitive protocols to rank-order the transition fatigue resistance of polycrystalline microstructures, International Journal of Fatigue 119 (2019) 1–10.
  • (41) V. Sundararaghavan, N. Zabaras, Design of microstructure-sensitive properties in elasto-viscoplastic polycrystals using multi-scale homogenization, International Journal of Plasticity 22 (10) (2006) 1799–1824.
  • (42) X. Li, Z. Yang, L. C. Brinson, A. Choudhary, A. Agrawal, W. Chen, A deep adversarial learning methodology for designing microstructural material systems, in: ASME 2018 international design engineering technical conferences and computers and information in engineering conference, American Society of Mechanical Engineers, 2018, pp. V02BT03A008–V02BT03A008.
  • (43) Z. Yang, X. Li, L. Catherine Brinson, A. N. Choudhary, W. Chen, A. Agrawal, Microstructural materials design via deep adversarial learning methodology, Journal of Mechanical Design 140 (11).
  • (44) R. Bostanabad, Y. Zhang, X. Li, T. Kearney, L. C. Brinson, D. W. Apley, W. K. Liu, W. Chen, Computational microstructure characterization and reconstruction: Review of the state-of-the-art techniques, Progress in Materials Science 95 (2018) 1–41.
  • (45) Y. Liu, M. S. Greene, W. Chen, D. A. Dikin, W. K. Liu, Computational microstructure characterization and reconstruction for stochastic multiscale material design, Computer-Aided Design 45 (1) (2013) 65–76.
  • (46) D. Li, Review of structure representation and reconstruction on mesoscale and microscale, JOM 66 (3) (2014) 444–454.
  • (47) S. Bargmann, B. Klusemann, J. Markmann, J. E. Schnabel, K. Schneider, C. Soyarslan, J. Wilmers, Generation of 3D representative volume elements for heterogeneous materials: A review, Progress in Materials Science 96 (2018) 322–384.
  • (48) F. Latief, B. Biswal, U. Fauzi, R. Hilfer, Continuum reconstruction of the pore scale microstructure for fontainebleau sandstone, Physica A: Statistical Mechanics and its Applications 389 (8) (2010) 1607–1618.
  • (49)

    L. Mosser, O. Dubrule, M. J. Blunt, Reconstruction of three-dimensional porous media using generative adversarial neural networks, Physical Review E 96 (4) (2017) 043309.

  • (50) L. Mosser, O. Dubrule, M. J. Blunt, Stochastic reconstruction of an oolitic limestone by generative adversarial networks, Transport in Porous Media 125 (1) (2018) 81–103.
  • (51) B. Settles, Active learning literature survey, Tech. rep., University of Wisconsin-Madison Department of Computer Sciences (2009).
  • (52) B. Settles, From theories to queries: Active learning in practice, in: Active Learning and Experimental Design workshop In conjunction with AISTATS 2010, 2011, pp. 1–18.
  • (53) S. Memeti, S. Pllana, A. Binotto, J. Kołodziej, I. Brandic, Using meta-heuristics and machine learning for software optimization of parallel computing systems: a systematic literature review, Computing 101 (8) (2019) 893–936.
  • (54) S. Mahdavi, M. E. Shiri, S. Rahnamayan, Metaheuristics in large-scale global continues optimization: A survey, Information Sciences 295 (2015) 407–428.
  • (55) A. Tran, J. Sun, J. M. Furlan, K. V. Pagalthivarthi, R. J. Visintainer, Y. Wang, pBO-2GP-3B: A batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics, Computer Methods in Applied Mechanics and Engineering 347 (2019) 827–852.
  • (56) A. Tran, S. McCann, J. M. Furlan, K. V. Pagalthivarthi, R. J. Visintainer, T. Wildey, aphBO-2GP-3B: A budgeted asynchronously-parallel multi-acquisition for known/unknown constrained Bayesian optimization on high-performing computing architecture, arXiv preprint arXiv:2003.09436.
  • (57) A. Tran, M. Tran, Y. Wang, Constrained mixed-integer Gaussian mixture Bayesian optimization and its applications in designing fractal and auxetic metamaterials, Structural and Multidisciplinary Optimization (2019) 1–24.
  • (58) A. Tran, T. Wildey, S. McCann, sBF-BO-2CoGP: A sequential bi-fidelity constrained Bayesian optimization for design applications, in: Proceedings of the ASME 2019 IDETC/CIE, Vol. Volume 1: 39th Computers and Information in Engineering Conference of International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, American Society of Mechanical Engineers, 2019, v001T02A073.
  • (59) A. Tran, T. Wildey, S. McCann, sMF-BO-2CoGP: A sequential multi-fidelity constrained Bayesian optimization for design applications, Journal of Computing and Information Science in Engineering (2020) 1–15.
  • (60) A. Tran, Y. Wang, J. Furlan, K. V. Pagalthivarthi, M. Garman, A. Cutright, R. J. Visintainer, WearGP: A UQ/ML wear prediction framework for slurry pump impellers and casings, in: ASME 2020 Fluids Engineering Division Summer Meeting, American Society of Mechanical Engineers, 2020.
  • (61) A. Tran, J. M. Furlan, K. V. Pagalthivarthi, R. J. Visintainer, T. Wildey, Y. Wang, WearGP: A computationally efficient machine learning framework for local erosive wear predictions via nodal Gaussian processes, Wear 422 (2019) 9–26.
  • (62) A. Tran, L. He, Y. Wang, An efficient first-principles saddle point searching method based on distributed kriging metamodels, ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part B: Mechanical Engineering 4 (1) (2018) 011006.
  • (63) E. Brochu, V. M. Cora, N. de Freitas, A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning, arXiv preprint arXiv:1012.2599.
  • (64) B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, N. de Freitas, Taking the human out of the loop: A review of Bayesian optimization, Proceedings of the IEEE 104 (1) (2016) 148–175.
  • (65) P. I. Frazier, A tutorial on Bayesian optimization, arXiv preprint arXiv:1807.02811.
  • (66) D. R. Jones, M. Schonlau, W. J. Welch, Efficient global optimization of expensive black-box functions, Journal of Global Optimization 13 (4) (1998) 455–492.
  • (67) S.-A. N. Alexandropoulos, C. K. Aridas, S. B. Kotsiantis, M. N. Vrahatis, Multi-objective evolutionary optimization algorithms for machine learning: A recent survey, in: Approximation and Optimization, Springer, 2019, pp. 35–55.
  • (68)

    B. Li, J. Li, K. Tang, X. Yao, Many-objective evolutionary algorithms: A survey, ACM Computing Surveys (CSUR) 48 (1) (2015) 13.

  • (69) S. Rojas-Gonzalez, I. Van Nieuwenhuyse, A survey on kriging-based infill algorithms for multi-objective simulation optimization, FEB Research Report KBI_1907.
  • (70) S. Plimpton, C. Battaile, M. Chandross, L. Holm, A. Thompson, V. Tikare, G. Wagner, E. Webb, X. Zhou, C. G. Cardona, et al., Crossing the mesoscale no-man’s land via parallel kinetic Monte Carlo, Sandia Report SAND2009-6226.
  • (71) S. Plimpton, A. Thompson, A. Slepoy, SPPARKS kinetic monte carlo simulator (2012).
  • (72) T. M. Rodgers, J. D. Madison, V. Tikare, Simulation of metal additive manufacturing microstructures using kinetic Monte Carlo, Computational Materials Science 135 (2017) 78–89.
  • (73) E. Jones, T. Oliphant, P. Peterson, et al., SciPy: Open source scientific tools for Python, [Online; accessed April 28, 2020] (2001–).
  • (74) A. L. Garcia, V. Tikare, E. A. Holm, Three-dimensional simulation of grain growth in a thermal gradient with non-uniform grain boundary mobility, Scripta Materialia 59 (6) (2008) 661–664.
  • (75) F.-Y. Wu, The Potts model, Reviews of modern physics 54 (1) (1982) 235.
  • (76)

    A. Tran, H. Tran, Data-driven high-fidelity 2D microstructure reconstruction via non-local patch-based image inpainting, Acta Materialia 178 (2019) 207–218.

  • (77) L. Shu, P. Jiang, X. Shao, Y. Wang, A new multi-objective bayesian optimization formulation with the acquisition function for convergence and diversity, Journal of Mechanical Design 127 (6) (2020) 1–38.
  • (78) A. Tran, M. Eldred, Y. Wang, S. McCann, srMO-BO-3GP: A sequential regularized multi-objective constrained Bayesian optimization for design applications, in: Proceedings of the ASME 2020 IDETC/CIE, Vol. Volume 1: 40th Computers and Information in Engineering Conference of International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, American Society of Mechanical Engineers, 2020.
  • (79) B. Ankenman, B. L. Nelson, J. Staum, Stochastic kriging for simulation metamodeling, Operations research 58 (2) (2010) 371–382.
  • (80) X. Chen, Q. Zhou, Sequential design strategies for mean response surface metamodeling via stochastic kriging with adaptive exploration and exploitation, European Journal of Operational Research 262 (2) (2017) 575–585.
  • (81) S. Ba, V. R. Joseph, Composite Gaussian process models for emulating expensive functions, The Annals of Applied Statistics (2012) 1838–1860.
  • (82) W. Xiong, M. Selleby, Q. Chen, J. Odqvist, Y. Du, Phase equilibria and thermodynamic properties in the Fe-Cr system, Critical Reviews in Solid State and Materials Sciences 35 (2) (2010) 125–152.