Inverse design of crystals using generalized invertible crystallographic representation

05/15/2020 ∙ by Zekun Ren, et al. ∙ 0

Deep learning has fostered many novel applications in materials informatics. However, the inverse design of inorganic crystals, i.e. generating new crystal structure with targeted properties, remains a grand challenge. An important ingredient for such generative models is an invertible representation that accesses the full periodic table. This is challenging due to limited data availability and the complexity of 3D periodic crystal structures. In this paper, we present a generalized invertible representation that encodes the crystallographic information into the descriptors in both real space and reciprocal space. Combining with a generative variational autoencoder (VAE), a wide range of crystallographic structures and chemistries with desired properties can be inverse-designed. We show that our VAE model predicts novel crystal structures that do not exist in the training and test database (Materials Project) with targeted formation energies and band gaps. We validate those predicted crystals by first-principles calculations. Finally, to design solids with practical applications, we address the sparse label problem by building a semi-supervised VAE and demonstrate its successful prediction of unique thermoelectric materials

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Traditionally, the design of solid-state materials has heavily relied on human intuition and heuristic chemical rules. Considering the number of elements in the periodic table and symmetries in a crystal, the materials space is rather unexplored given only

solid materials have been reported (Noh et al., 2019a). Historically, the rate of discovery and development for conventional solid-state materials has been slow (Correa-Baena et al., 2018). Recently, this process has been accelerated by high throughput screening based on density functional theory (DFT) calculations. High throughput screening has allowed the discovery of a significant number of crystals with functional properties (Brandt et al., 2015; Zhao et al., 2017; Sun et al., 2019; Noh et al., 2019b). Because of the heavy computational burden of DFT, application of machine learning methods to accelerate DFT calculations has seen initial success in screening solid-state crystals (Xie and Grossman, 2018; Chen et al., 2019; Schütt et al., 2018; Rupp et al., 2012; Faber et al., 2015; Huo and Rupp, 2017; Behler, 2011; Bartók et al., 2013). However, these works focus on regressing crystal representations to desired properties, and subsequently screening the existing crystals in the database. To accelerate new material discovery, an inverse design framework of 3D crystals, mapping desired functionality to the crystalline structure, needs to be developed (Zunger, 2018).

One promising branch to design unique crystals is crystal structure prediction using global optimization (Oganov et al., 2019). Various global optimization methods including simulated annealing (Wille, 1986; Kirkpatrick, 1984; Schön and Jansen, 1996), minima hopping (Goedecker, 2004)

and genetic algorithms

(Wang et al., 2012; Glass et al., 2006) have been investigated. Those methods generally require an initial structural pool of specific chemical composition and thus are limited user’s initial choices. Another emerging branch is building probabilistic generative models to encode the existing materials to a continuous latent space (Sanchez-Lengeling and Aspuru-Guzik, 2018). By mapping the latent space to material properties, inverse design with a targeted property can be realized.

Figure 1: Invertible crystal representation using both real space and Fourier space crystal properties

Prior work on inverse design using generative models has been extensively studied in molecules (Gómez-Bombarelli et al., 2018; Liu et al., 2018; Jin et al., 2018; Samanta et al., 2018; Assouel et al., 2018; Kusner et al., 2017; Polykovskiy et al., 2018). Those invertible molecular descriptors are either represented in a 1D string (SMILES representation) or 2D graphs. However, they are not suitable to describe a 3D periodic lattice, such as the one in a crystal. Building generative models for crystals is challenging due to limited available data and its 3D periodic structure (Noh et al., 2020). Several specialized inverse design tools for solid-state materials have been proposed, with limited structural and chemical range (Zhao et al., 2017; Nouira et al., 2018; Noh et al., 2019a; Kim et al., 2020)

. Noh et al. featurizes a single unit cell in a 3D grid and builds a hierarchical image-based VAE to generate new vanadium oxides. Beyond autoencoders, Generative Adversarial Neural Networks have been investigated to inverse design new crystals with different elemental compositions

(Sawada et al., 2019; Dan et al., 2019; Nouira et al., 2018; Kim et al., 2020). Kim and Noh et al. represent the crystal as a set of atomic coordinates and cell parameters. The crystal generation in the above inverse design methods is achieved by selecting a subset of elements from the periodic table. Moreover, the periodic nature of the crystals is not reflected in the current crystal representations. To account for crystal periodicity, Hoffmann et al. directly encode the repeated unit cells in a 3D cube to train a VAE (Hoffmann et al., 2019). Training the model using a 3D representation of the repeating unit can be computationally inefficient, and limited to specific crystal symmetries.

In this work, we propose an efficient and invertible 2D representation for 3D crystals that accounts for crystal periodicity and accesses the full periodic table. The motion of electrons for crystals in Quantum Mechanics is generally described in both the real space e.g. surface energy, and the momentum space (k space) e.g. E-k diagram. We embed this domain knowledge to construct a crystallographic representation that can be used to train a generative model by combining the atomic and structural properties in both real space and momentum space (also called k space, Fourier space or frequency space). We build a VAE (Kingma and Welling, 2013) using these 2D crystallographic representations with the latent space organized according to different material properties. This material-semantic latent representation allows the generation of new crystals with targeted materials properties. We investigate different sampling strategies to generate new crystals from the latent space. The comparison is assessed by the reconstruction error in rediscovered crystals from the test dataset. We find that applying a local perturbation to existing crystals yields the lowest crystal reconstruction error in rediscovered crystals. Given evaluation of new materials is expensive, local perturbation is used for new crystal generation.

We demonstrate three success cases to design new crystals with desired properties that are not in the training database (Materials Project). As we trained our model with DFT calculations only, we believe performing independent DFT calculations of new crystals is an adequate and actionable validation procedure. In the first demonstration, we generated 97 unique ternary crystals with various targets of formation energy per atom (). We perform additional DFT calculations to validate those crystals and find 27 crystals have targeted within -0.1 eV/atom error. In the second demonstration, we generate 16 new crystals with targeted bandgap () and . 6 out of 16 generated crystals are validated to have close to the target within the model prediction accuracy while the majority of the unique crystals meet the target. Lastly, we extend our generative model to predict good thermoelectric crystals. An additional challenge for such a demonstration is the small dataset. The application-relevant properties such as effective mass, Seebeck coefficient and power factor are computationally expensive to compute with DFT or require different computation platforms (Madsen and Singh, 2006), and existing data is limited. We tackle this data scarcity problem by jointly training a large unlabelled dataset with a small labelled subset via a semi-supervised VAE. We show that 2 out of a total 27 predicted unique crystals have state-of-the-art power factor validated using Boltzmann Transport calculations. In summary, in this work:

  1. We present a generalized invertible crystallographic representation for 3D crystals.

  2. We build a VAE that allows the generation of new crystals with targeted materials properties.

  3. We demonstrate three successful applications of inverse design of unique functional crystals.

2 Invertible crystallographic representation for crystals

Atoms in a crystal are arranged in a periodic pattern. According to Bloch’s theorem in solid-state physics, the wave function can be expressed as the product of a plane wave and a function that has the same periodicity as the crystal (Martín-Palma et al., 2006). The energy of electrons in the crystal is both studied in the real space (energy surface) and momentum space (energy dispersion relation). We embed such physics knowledge to construct the crystallographic descriptors for the 3D periodic crystal. Figure 1

shows the schematic of our crystallographic representation. Firstly, we represent the unit cell (the basic building block of the crystal) using the length and angles of the three fundamental translation vectors (lattice vectors) and all atomic coordinates in the real space. Then, we introduce Fourier-transformed crystal properties (FTCP) to capture the elemental and structural information in the momentum space. We construct the crystallographic representations by concatenating both real space and momentum space crystal representations.

2.1 Real space representation

To represent the crystals in the real space, we extract the text information in the crystallographic information file (CIF) of 3D crystals. The text primarily corresponds to two matrices: cell matrix (the length and angle of three translation lattice vectors), with size , and basis matrix (all atom coordinates in the unit cell) with size .

is the upper bound of the number of atoms in the unit cell. We use zero padding in crystals with less atoms. In addition, we bin and one-hot encode each element in the unit cell as a vector

Z which has K features which are the number of atomic properties such as group number, electronegativity, first ionization energy, etc. (Xie and Grossman, 2018). For featurizing the elements in ternary compounds, we concatenate Z into a matrix. Hereafter, we construct the real space representation by further concatenating the atomic vector Z of each element, the cell matrix, the basis matrix and the one-hot encoded element matrix. The concatenated real space representation matrix is with size where is the sum of the length of the cell matrix, the basis matrix and the element matrix.

2.2 Momentum space representation:

The most common momentum space representation of a crystal is its diffraction pattern. X-ray crystallography is the primary method to study periodic crystals (Ladd et al., 1985)

. Modified diffraction images for periodic crystals have been shown to classify its geometry accurately

(Ziletti et al., 2018). We enrich the information in diffraction images in the momentum space by projecting one-hot encoded elemental properties Z to different crystal planes using a discrete Fourier transform.

(1)

where is the atom in the unit cell, is the atom coordinates in real space and is the Miller index of a crystal plane (a point in momentum space). is the Fourier transformed crystal properties for plane , which is the projection of atomic properties in one plane in the crystal lattice and a point in the reciprocal lattice. As the unit cells are defined as the repeating unit in the crystal, there exist an infinite number of planes. We limit the number of planes by applying a filter of the sum of less than 4. The Fourier-transformed properties are arranged according to orders of their Miller index and become a matrix where is the number of planes with the sum of less than 4 and is the number of atomic features. Lastly, we concatenate the real space crystal representation (V with size ) and momentum space representation (F with size ) with zero padding. The combined representation has a matrix size of .

3 Generative model and inverse design

Hereafter, we build a generative model to encode and decode the concatenated crystallographic descriptors shown in Figure 2. The detailed structure of the neural network is shown in Supplementary S6.

3.1 Encoder

We treat the concatenated crystallographic representation as a 1D signals with K channels. The signal in each channel represents the atomic property projection such as electronegativity, ionization energy, etc., in both real and momentum space. We use the 1D convolutional neural network (CNN) as the sequence of atoms and crystal planes are contained in the 1D signal. The 1D CNN in this work is inspired by point clouds used for 3D image classification

(Kim et al., 2020; Qi et al., 2017). There is a spatial 1D relationship in our crystal representation. Along this spatial axis, our Fourier space descriptor is arranged according to symmetry points that are universal in describing the electronic band structure. Because of the pooling operations in CNNs, the arrangement of those geometry points becomes translationally invariant (Gu et al., 2018; Liu et al., 2019). However, there is no universal order for the real space descriptor and thus the real space representation has neither translational or rotational invariance. Kim et al (Kim et al., 2020)

have shown that performing data augmentation for the real space descriptors will lead to a more balanced generation of crystal structures for Mg-Mn-O systems. Given a much larger chemical range is considered in this study, data augmentation is not feasible. Instead, we encode 1D signals with K channels into a normal distribution with a diagonal covariance matrix using 1D CNN. The latent vector is sampled from those distributions and is regularized by the Kullback–Leibler (KL) divergence between the latent distribution and the standard Gaussian prior.

3.2 Latent to material property regression

In addition to constraining the latent distribution to the multivariant normal distribution with diagonal covariance, we train a gated regression model to map the latent space to material properties (Gómez-Bombarelli et al., 2018). The regression is shown in Eq.2 where and are neural networks and

is the sigmoid function,

is the encoded latent. We use the L2 loss between and the materials properties. The regression shapes the latent space with a gradient by material property values (Figure S1) and allows to generate new crystals with targeted material properties.

(2)

3.3 Decoder

The decoder mirrors the encoder using transposed convolutional layers. Lengths, angles of the three translational vectors and coordinates of atoms in the unit cell are part of the decoded crystallographic descriptors. Therefore, the 3D structure of the unit cell can be fully reconstructed.

3.4 Loss

Training of the model is done by optimization of the tractable evidence lower bound (ELBO) of VAE and regression loss for material properties prediction. The overall loss is

(3)

where is the reconstruction loss, is the distribution loss, is the regression loss, and are the coefficients. To learn a disentangled representation, we allow heavy penalization of the latent distribution () (Higgins et al., 2017).

Figure 2: VAE architecture using invertible crystal representations of both real space and Fourier space crystal properties.
reconstruction metric Lp Slerp Random
Rediscovery percentage [%] 27.1 35.9 50.9
Lattice constant PMAE [%] 14.4 21.7 27.3
Lattice angles PMAE [%] 9.75 17.1 24.5
Atom fractional coordinates MAE[] 0.14 0.15 0.16
Table 1: Validation of VAE generated crystals using three sampling methods.

3.5 Inverse design

As the latent space of the VAE is mapped to material properties using the regression model, the latent space or reduced material space becomes an organized and continuous crystal representation with different material properties (Figure S1). Inverse design of new crystals leverages this structured latent space by sampling in regions with desired properties. To avoid the possibility that generated crystals with minor structural change is considered as a new crystal, we only consider generated crystals whose chemical formula changes as unique samples. To ensure the correctness of inverse-designed crystals, we investigate three different sampling strategies to the latent space. The comparison is assessed by the crystal reconstruction errors for the crystals in the test dataset.

4 Experiments

4.1 Dataset

To generate crystals that access the chemistries in the full periodic table, we use the Materials Project dataset (1). In the first experiment, we consider ternary compounds with the number of atoms in the unit cell () is less than or equal to 20 and energy above hull () less than 50 meV in the database. The number of crystals selected is reduced to 24,785. We ensure that there is no identical structure (same composition and same geometry) in the dataset. We randomly split those unique structures into 80% for training and 20% for testing. The VAE model is trained with 80% of the data and validated with 20% of the data. DFT calculated and of those crystals are taken as the output of the material property prediction model.

4.2 Generating new crystals

We featurize the 24,785 .cif files into the concatenated 2D crystallographic representations following the procedure shown in Figure 1. We then train VAE using the loss described in Eq.3. We reconstruct the crystals from the trained latent space using the decoder, and then compare key lattice parameters with the original crystal. For the test dataset, we achieve 99.1% accuracy for crystal chemical formula prediction, percentage mean absolute error (PMAE) of 7.41% for the length of lattice constant, PMAE of 3.99% for the lattice angles. PMAE for atomic coordinates results in unreasonable numbers as there are many zeros in the coordinates. We use Mean absolute error (MAE) for the atomic coordinates and 0.001 error is reported.

Moving beyond reconstruction, we investigate the generation of unique crystals that are not in the training dataset by sampling out-of-the-distribution latent space. Three different sampling methods: local perturbation (Lp), spherical linear interpolation (Slerp) and random sampling (Random) are implemented to sample points that are different from training dataset in the latent space.

As our crystal representation contains both elemental and structural information, we invert those two parts separately to recover the full 3D crystal. The elemental representation is one-hot encoded, and we perform additional argmax

to convert the decoded elemental representation to discrete labels (chemical formula). The structural information contains scalar values of the atomic coordinates and lattice parameters, and is directly taken from the decoded representation. Thus, despite every sampled latent representation decoding into a mathematically new crystal representation, many of those have the same chemical formula with minor structural changes compared to the crystals in the training dataset. Those crystals that have the same chemical formula to the training dataset are excluded and not counted as new crystals. For Lp and Slerp, known crystals with targeted material properties are used as initial guesses, Lp applies random local perturbation to those crystals’ latent representation and Slerp performs interpolation between each pair of crystals. Random sampling is done from Gaussian distribution with the sample’s mean and variance. We compare the three sampling methods by attempting to rediscover the 4,957 crystals in the test dataset. Discovery of new crystals in the test dataset also validates our generative model. We use two sets of metrics, percentage of crystals in the test dataset that can be rediscovered (rediscovery percentage) and reconstruction error of the rediscovered crystals to the test crystals (rediscovery error) to evaluate the sampling strategy.

Table 1 summarizes the rediscovery percentage and error of the three sampling methods. For a fair comparison, we use the same number of crystals (500) in the test dataset for all methods in calculating rediscovered crystals errors. Sampling based on local perturbation of latent representations has the lowest reconstruction error for rediscovered crystals but only recovers 27% of the test crystals. We find that new crystals obtained by local perturbation sampling mainly experiences elemental substitution compared to its root crystal. This agrees well with the prevailing design principles for new crystalline materials, which make use of manual substitution of certain elements in the unit cell (Zhao et al., 2017). Slerp interpolation and random sampling generate samples that experience both elemental substitution and significant structural change, however, the crystal reconstruction error in the rediscovered materials is much higher. Given the long development cycle for solid-state materials (Correa-Baena et al., 2018), it is of critical importance to have an accurate initial guess. Even the DFT calculations to validate VAE generated crystals are computationally expensive. DFT validations conducted in this study takes around 1-2 hours per single crystal with 16 CPU cores. However, DFT calculations can require significant time (in the order of weeks) to setup and optimize. More complex calculations, such as defect calculations, can take days to complete (Atilgan and Hu, 2018). Given all the above considerations, we choose local perturbation sampling which gives us the lowest rediscovery error to generate new crystals with targeted material properties.

Input MAE() MAE()
Real space
representations 0.092 0.311
Momentum space
representations 0.090 0.267
Concatenated
representations 0.063 0.231
CGCNN 0.062 0.254
Table 2: ablation study of crystal representations for forward prediction.
Metrics Real space representation concatenated representation
Element
Categorical
Accuracy [%] 99.1 97.9
Lattice constant
PMAE[%] 7.41 12.5
Lattice angles
PMAE[%] 3.99 6.75
Atom fractional
coordinates
MAE [a.u.] 0.001 0.002
Table 3: Ablation study of crystal representations for VAE reconstruction.

Figure 3: Examples of VAE-generated 3D crystals with targeted =-0.5 eV after DFT relaxation. There are 8 different crystal structures with more than 30 different elements in those 17 crystals (Figure S2). This shows the VAE can generate crystals accessing a wide range of structures and chemistries.

Figure 4: DFT calculated material properties for inverse designed unique crystals (a) Case 1: ternary crystals with a targeted =-0.5 eV. The black dashed line is the target of . 7 out of 17 generated crystals have close to the target (-0.5eV) within 0.1 eV accuracy (b) Case 2: ternary and quarternary crystals with targeted -1.5 eV and =1.5 eV. Most of the generated crystals satisfies -1.5 eV requirement and the median of those crystals is around 1.5 eV

4.3 Ablation study

As previous work on crystal representation mainly focuses on material property prediction or inverse design with limited chemistry, there is no benchmark in the inverse design of crystals with desired properties. Herein, we perform the ablation study on our crystal representation and compare a property prediction error with the state-of-the-art graph representation. The forward property prediction only consists of the encoder which embeds the crystallographic descriptors in the invertible latent representation and the regression model that maps the latent space to the material properties. The forward model is trained with the same ternary crystals dataset that is used for training the VAE. The results in Table 2 show that the concatenated crystallographic representations have an MAE of 0.063 eV/atom in predicting the formation energy per atom and MAE of 0.231 eV in predicting the bandgap. The error from this model is comparable to the intrinsic error from DFT (Kirklin et al., 2015). Only considering representation in real/momentum space results in higher prediction error indicating the importance of incorporating both real space and Fourier space descriptors. The graph-based representation Crystal Graph Convolutional Neural Networks (CGCNN) (Xie and Grossman, 2018) have similar performance compared to our crystallographic representations when trained using the ternary dataset, however, our crystal representation is invertible and allows for training generative models.

Furthermore, we compare the reconstruction error between the VAE model trained using both real and momentum space representation and model trained only with real space representations. The error significantly increases when Fourier transformed crystal representation is not considered. The improvement in reconstruction error of the lengths, angles of the three basis vector and atomic coordinates by incorporating the crystal representations in the momentum space Table 3 shows featurizing the periodic crystal in Fourier space is also important for training the generative model.

Figure 5: (a) and (b) show the predicted ternary crystals generated by our inverse design algorithm. These materials don’t exist in any database. Their power factor values (divided by the relaxation time) are plotted as a function of doping level at room temperature, both for n- and p-doping. (c) shows the comparison to state-of-the-art high-performance thermoelectric material, cubic germanium telluride (GeTe), with comparable results.

5 Demonstrations: inverse design of crystals with targeted properties

The goal of inverse design is to generate new crystals, with desired properties that are not in the database. In this study, we showcase three demonstrations. Firstly, we generate new crystals with targeted values. We perturb the latent representation of 100 existing ternary crystals with close to the target. 17 unique ternary crystals with new chemical formula that do not exist in the Materials Project database are generated. A few of those unique crystal structures are shown in Figure 3. There are 8 different crystal structures with more than 30 different elements in those 17 crystals (Figure S2). This shows the VAE can generate crystals accessing a wide range of structures and chemistries. We perform independent DFT calculations to optimize the geometry and compute the of the generated crystals. We notice that the generated atomic coordinates changes after DFT relaxation indicating that the atoms of generated crystals are not at the minimum energy surface. Post-processing the .cif files with DFT relaxation is required and the same approach is reported in (Noh et al., 2019a; Kim et al., 2020). This is because all the atoms in our training data (Materials Project) are relaxed and our generative model is not trained to find structure close to the local minimum of the energy surface. Further development of the model is needed to learn the relaxation of atoms in generated crystals. Figure 4 (a) shows that with additional DFT relaxation calculations, 7 out of 17 generated crystals have close to the target (-0.5eV) within 0.1 eV accuracy. The black dashed line is the target of . In addition, we generate another 80 ternary compounds with staggered values (-0.3eV, -0.6eV and -0.7eV). 20 out of those 80 compounds have close to the target within 0.1 eV accuracy. The results are shown in Figure S3. All the generated 97 compounds in this demonstration have a unique chemical formula that does not exist in the Materials Project database. In addition, we investigate the structural uniqueness and ensure not all compounds can be discovered by manual elemental substitution. We use a dissimilarity value that is assessed on the basis of local coordination information from all sites in the two structures (Zimmermann and Jain, 2020), the crystal structure is the same if the dissimilarity is zero. Figure S4 shows the minimum dissimilarity value compared to every crystal in the training dataset. The median dissimilarity value is 0.64 where a number of compounds have a dissimilarity value above 0.75 indicating the structural uniqueness of those generated compounds.

In the second demonstration, we extend the design criteria to multi-objective by producing ternary and quaternary crystals for potential photovoltaic applications. The design criteria are to have a bandgap equal to 1.5 eV and formation energy per atom smaller than -1.5 eV. We locally perturb the latent representation of 100 existing ternary crystals and generate 16 unique crystals. Figure 4(b) shows that the majority of the generated crystals satisfies the formation energy target (-1.5 eV) and the median bandgap value () of those crystals is around 1.5 eV. However, the variance of is significantly larger. We postulate that this is due to large training error for prediction (Table 2). Six out of 16 generated crystals have close to the 1.5 eV target within the bandgap forward model prediction accuracy. Figure S5 shows the histogram of and

values of all crystals in the training dataset. The probability of randomly sampling crystals satisfies

1.5 eV and =1.5 eV is less than 1% justifying the non-triviality of designing crystals of those targeted property values.

5.1 Designing thermoelectric materials using semi-supervised VAE

A grand challenge in the field of thermoelectrics, where materials convert heat to electrical energy and vice-versa, is to perform inverse design with multi-objectives, e.g. the compound must be stable and has high power factor with the desired bandgap. This is a difficult task because, in addition to the ground state properties of the inorganic solids, we also have to learn the key features that tell us which solids have good functional properties. The electronic power factor is a non-linear mathematical function that depends on the electronic band structure as well as charge scattering parameters. Generating crystals for thermoelectric applications is much more challenging because of data scarcity: we use the database from (Ricci et al., 2017) where the constant relaxation time approximation under the Boltzmann Transport Equations is used to calculate the thermoelectric properties of inorganic crystals from Materials Project. The final dataset has 34,784 crystal structures with ground-state properties (, ). Only 4,284 crystal have calculated power factor labels. We tackle this limited data problem by jointly training the VAE and mapping its latent space to ground-state property labels, and a small labelled subset with thermoelectric property labels. The training loss of the semi-supervised VAE is shown in Eq.4. We demonstrate that our method can generate crystal structures that meet those objectives. Here, the doping level and temperature are treated as user inputs. The ability to dope the material to the desired carrier concentration is beyond the scope of our approach.

(4)

where ,

is the reconstruction and Kullback–Leibler divergence for all crystals in the data,

is the regression loss for ground state material properties (,,) of all crystals and is the regression loss for 4,284 crystals that have calculated power factor labels.

Based on domain expertise, we select 100 ternary and quaternary crystals with the following section criteria: high power factors, energy above convex hull less than 0.03 eV, and bandgap between 0.3 and 1.5 eV. Applying local perturbation to those compounds generates 27 unique crystals that are not in the database. To test the quality of the crystals generated, we performed DFT calculations of the predicted materials. We also performed thermal relaxation to obtain the final atomic coordinates and then performed Boltztrap calculations (Madsen and Singh, 2006). We find that 2 out of the 27 crystals show state-of-the-art power factors, comparable to the best thermoelectric materials. The results are shown in Figure 5, as a function of doping concentration.

6 Future work

Despite many recent studies reporting great results through theoretical screening using forward prediction and inverse design using generative methods in the realm of computations, the synthesizability of these newly found materials still poses a great challenge to the community. Therefore, while our inverse design tool can discover new materials unprecedented in the Materials Project with targeted properties, those structures are lacking real experimental validation. This is because of the fundamental limitations of DFT, or any other first-principles calculation technique, to fully capture real materials. As we trained our model with DFT calculations only, performing independent DFT calculations (outside of the training database, with different DFT functionals) is an adequate and actionable approach to validate our predicted crystal structures. Validation using experimental crystals will be future work, as it requires longer development time frames, including physical process tuning, synthesizability optimization, and characterization, etc.

7 Conclusions

We present a generalized invertible crystallographic representation for 3D periodic crystals. An ablation study of our crystallographic representation suggests an advantage of using both real space crystal representations and momentum space representations for material property prediction and new crystal generation. We propose an inverse design paradigm for generating new solid-state materials with targeted material properties using VAE. We demonstrated three unique applications of our model by generating new chemistry formulas that are not in the database and validate them using DFT and Boltzmann Transport Calculations. Our future work will be experimentally synthesizing the predicted crystals, and bridging the synthesizability gap between theoretically generated crystals and experimental crystals by using semi-supervised learning of experimental datasets.

8 Acknowledgement

We acknowledge Shijing Sun from MIT and Vladan Stevanovic from Colorado School of Mines for helpful discussions. This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) program through the Singapore Massachusetts Institute of Technology (MIT) Alliance for Research and Technology’s Low Energy Electronic Systems research program. J.N. and Y.J. acknowledge support from NRF Korea (NRF-2017R1A2B3010176). Y.L. is supported by the National Key Research and Development Program of China (Grant Nos. 2017YFB0702901 and 2017YFB0701502) and the National Nature Science Foundation of China (Grant No. 91641128). Q.L., S.J. and K.H. acknowledge funding from the Accelerated Materials Development for Manufacturing Program at A*STAR via the AME Programmatic Fund by the Agency for Science, Technology and Research under Grant No. A1898b0043.

References

  • [1] Cited by: §4.1.
  • R. Assouel, M. Ahmed, M. H. Segler, A. Saffari, and Y. Bengio (2018) Defactor: differentiable edge factorization-based probabilistic graph generation. arXiv preprint arXiv:1811.09766. Cited by: §1.
  • E. Atilgan and J. Hu (2018) First-principle-based computational doping of srtio3 using combinatorial genetic algorithms. Bulletin of Materials Science 41 (1), pp. 1. Cited by: §4.2.
  • A. P. Bartók, R. Kondor, and G. Csányi (2013) On representing chemical environments. Physical Review B 87 (18), pp. 184115. Cited by: §1.
  • J. Behler (2011) Atom-centered symmetry functions for constructing high-dimensional neural network potentials. The Journal of chemical physics 134 (7), pp. 074106. External Links: ISSN 0021-9606 Cited by: §1.
  • R. E. Brandt, V. Stevanović, D. S. Ginley, and T. Buonassisi (2015) Identifying defect-tolerant semiconductors with high minority-carrier lifetimes: beyond hybrid lead halide perovskites. Mrs Communications 5 (2), pp. 265–275. Cited by: §1.
  • C. Chen, W. Ye, Y. Zuo, C. Zheng, and S. P. Ong (2019) Graph networks as a universal machine learning framework for molecules and crystals. Chemistry of Materials 31 (9), pp. 3564–3572. External Links: ISSN 0897-4756 Cited by: §1.
  • J. Correa-Baena, K. Hippalgaonkar, J. van Duren, S. Jaffer, V. R. Chandrasekhar, V. Stevanovic, C. Wadia, S. Guha, and T. Buonassisi (2018) Accelerating materials development via automation, machine learning, and high-performance computing. Joule. External Links: ISSN 2542-4351 Cited by: §1, §4.2.
  • Y. Dan, Y. Zhao, X. Li, S. Li, M. Hu, and J. Hu (2019) Generative adversarial networks (gan) based efficient sampling of chemical space for inverse design of inorganic materials. arXiv preprint arXiv:1911.05020. Cited by: §1.
  • F. Faber, A. Lindmaa, O. A. von Lilienfeld, and R. Armiento (2015) Crystal structure representations for machine learning models of formation energies. International Journal of Quantum Chemistry 115 (16), pp. 1094–1101. External Links: ISSN 0020-7608 Cited by: §1.
  • C. W. Glass, A. R. Oganov, and N. Hansen (2006) USPEX—evolutionary crystal structure prediction. Computer physics communications 175 (11-12), pp. 713–720. Cited by: §1.
  • S. Goedecker (2004) Minima hopping: an efficient search method for the global minimum of the potential energy surface of complex molecular systems. The Journal of chemical physics 120 (21), pp. 9911–9917. Cited by: §1.
  • R. Gómez-Bombarelli, J. N. Wei, D. Duvenaud, J. M. Hernández-Lobato, B. Sánchez-Lengeling, D. Sheberla, J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams, and A. Aspuru-Guzik (2018) Automatic chemical design using a data-driven continuous representation of molecules. ACS central science 4 (2), pp. 268–276. External Links: ISSN 2374-7943 Cited by: §1, §3.2.
  • J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, and J. Cai (2018) Recent advances in convolutional neural networks. Pattern Recognition 77, pp. 354–377. External Links: ISSN 0031-3203 Cited by: §3.1.
  • I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner (2017) Beta-vae: learning basic visual concepts with a constrained variational framework. Iclr 2 (5), pp. 6. Cited by: §3.4.
  • J. Hoffmann, L. Maestrati, Y. Sawada, J. Tang, J. M. Sellier, and Y. Bengio (2019) Data-driven approach to encoding and decoding 3-d crystal structures. arXiv preprint arXiv:1909.00949. Cited by: §1.
  • H. Huo and M. Rupp (2017) Unified representation for machine learning of molecules and crystals. arXiv preprint arXiv:1704.06439 13754. Cited by: §1.
  • W. Jin, R. Barzilay, and T. Jaakkola (2018) Junction tree variational autoencoder for molecular graph generation. arXiv preprint arXiv:1802.04364. Cited by: §1.
  • S. Kim, J. Noh, G. H. Gu, A. Aspuru-Guzik, and Y. Jung (2020) Generative adversarial networks for crystal structure prediction. arXiv preprint arXiv:2004.01396. Cited by: §1, §3.1, §5.
  • D. P. Kingma and M. Welling (2013) Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114. Cited by: §1.
  • S. Kirklin, J. E. Saal, B. Meredig, A. Thompson, J. W. Doak, M. Aykol, S. Rühl, and C. Wolverton (2015) The open quantum materials database (oqmd): assessing the accuracy of dft formation energies. npj Computational Materials 1 (1), pp. 1–15. External Links: ISSN 2057-3960 Cited by: §4.3.
  • S. Kirkpatrick (1984) Optimization by simulated annealing: quantitative studies. Journal of statistical physics 34 (5-6), pp. 975–986. Cited by: §1.
  • M. J. Kusner, B. Paige, and J. M. Hernández-Lobato (2017) Grammar variational autoencoder. Conference Proceedings In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 1945–1954. Cited by: §1.
  • M. F. C. Ladd, R. A. Palmer, and R. A. Palmer (1985) Structure determination by x-ray crystallography. Springer. Cited by: §2.2.
  • Q. Liu, M. Allamanis, M. Brockschmidt, and A. Gaunt (2018) Constrained graph variational autoencoders for molecule design. Conference Proceedings In Advances in neural information processing systems, pp. 7795–7804. Cited by: §1.
  • W. Liu, J. Sun, W. Li, T. Hu, and P. Wang (2019) Deep learning on point clouds and its application: a survey. Sensors 19 (19), pp. 4188. Cited by: §3.1.
  • G. K. Madsen and D. J. Singh (2006) BoltzTraP. a code for calculating band-structure dependent quantities. Computer Physics Communications 175 (1), pp. 67–71. External Links: ISSN 0010-4655 Cited by: §1, §5.1.
  • R. J. Martín-Palma, J. Martínez-Duart, and F. Agulló-Rueda (2006) Nanotechnology for microelectronics and optoelectronics. Book, Elsevier. External Links: ISBN 0080456952 Cited by: §2.
  • J. Noh, G. H. Gu, S. Kim, and Y. Jung (2020) Machine-enabled inverse design of inorganic solid materials: promises and challenges. Chemical Science. Cited by: §1.
  • J. Noh, J. Kim, H. S. Stein, B. Sanchez-Lengeling, J. M. Gregoire, A. Aspuru-Guzik, and Y. Jung (2019a) Inverse design of solid-state materials via a continuous representation. Matter 1 (5), pp. 1370–1384. External Links: ISSN 2590-2385 Cited by: §1, §1, §5.
  • J. Noh, S. Kim, G. ho Gu, A. Shinde, L. Zhou, J. M. Gregoire, and Y. Jung (2019b) Unveiling new stable manganese based photoanode materials via theoretical high-throughput screening and experiments. Chemical Communications 55 (89), pp. 13418–13421. Cited by: §1.
  • A. Nouira, N. Sokolovska, and J. Crivello (2018) Crystalgan: learning to discover crystallographic structures with generative adversarial networks. arXiv preprint arXiv:1810.11203. Cited by: §1.
  • A. R. Oganov, C. J. Pickard, Q. Zhu, and R. J. Needs (2019) Structure prediction drives materials discovery. Nature Reviews Materials 4 (5), pp. 331–348. Cited by: §1.
  • D. Polykovskiy, A. Zhebrak, D. Vetrov, Y. Ivanenkov, V. Aladinskiy, P. Mamoshina, M. Bozdaganyan, A. Aliper, A. Zhavoronkov, and A. Kadurin (2018) Entangled conditional adversarial autoencoder for de novo drug discovery. Molecular pharmaceutics 15 (10), pp. 4398–4405. External Links: ISSN 1543-8384 Cited by: §1.
  • C. R. Qi, H. Su, K. Mo, and L. J. Guibas (2017) Pointnet: deep learning on point sets for 3d classification and segmentation. In

    Proceedings of the IEEE conference on computer vision and pattern recognition

    ,
    pp. 652–660. Cited by: §3.1.
  • F. Ricci, W. Chen, U. Aydemir, G. J. Snyder, G. Rignanese, A. Jain, and G. Hautier (2017) An ab initio electronic transport database for inorganic materials. Scientific data 4, pp. 170085. External Links: ISSN 2052-4463 Cited by: §5.1.
  • M. Rupp, A. Tkatchenko, K. Müller, and O. A. Von Lilienfeld (2012) Fast and accurate modeling of molecular atomization energies with machine learning. Physical review letters 108 (5), pp. 058301. Cited by: §1.
  • B. Samanta, D. Abir, G. Jana, P. K. Chattaraj, N. Ganguly, and M. G. Rodriguez (2018) Nevae: a deep generative model for molecular graphs. Conference Proceedings In

    Proceedings of the AAAI Conference on Artificial Intelligence

    ,
    Vol. 33, pp. 1110–1117. External Links: ISBN 2374-3468 Cited by: §1.
  • B. Sanchez-Lengeling and A. Aspuru-Guzik (2018) Inverse molecular design using machine learning: generative models for matter engineering. Science 361 (6400), pp. 360–365. Cited by: §1.
  • Y. Sawada, K. Morikawa, and M. Fujii (2019) Study of deep generative models for inorganic chemical compositions. arXiv preprint arXiv:1910.11499. Cited by: §1.
  • J. C. Schön and M. Jansen (1996) First step towards planning of syntheses in solid-state chemistry: determination of promising structure candidates by global optimization. Angewandte Chemie International Edition in English 35 (12), pp. 1286–1304. Cited by: §1.
  • K. T. Schütt, H. E. Sauceda, P. Kindermans, A. Tkatchenko, and K. Müller (2018) SchNet–a deep learning architecture for molecules and materials. The Journal of Chemical Physics 148 (24), pp. 241722. External Links: ISSN 0021-9606 Cited by: §1.
  • S. Sun, N. T. Hartono, Z. D. Ren, F. Oviedo, A. M. Buscemi, M. Layurova, D. X. Chen, T. Ogunfunmi, J. Thapa, S. Ramasamy, et al. (2019) Accelerated development of perovskite-inspired materials via high-throughput synthesis and machine-learning diagnosis. Joule 3 (6), pp. 1437–1451. Cited by: §1.
  • Y. Wang, J. Lv, L. Zhu, and Y. Ma (2012) CALYPSO: a method for crystal structure prediction. Computer Physics Communications 183 (10), pp. 2063–2070. Cited by: §1.
  • L. Wille (1986) Searching potential energy surfaces by simulated annealing. Nature 324 (6092), pp. 46–48. Cited by: §1.
  • T. Xie and J. C. Grossman (2018) Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Physical review letters 120 (14), pp. 145301. Cited by: §1, §2.1, §4.3.
  • X. Zhao, J. Yang, Y. Fu, D. Yang, Q. Xu, L. Yu, S. Wei, and L. Zhang (2017) Design of lead-free inorganic halide perovskites for solar cells via cation-transmutation. Journal of the American Chemical Society 139 (7), pp. 2630–2638. External Links: ISSN 0002-7863 Cited by: §1, §1, §4.2.
  • A. Ziletti, D. Kumar, M. Scheffler, and L. M. Ghiringhelli (2018) Insightful classification of crystal structures using deep learning. Nature communications 9 (1), pp. 2775. External Links: ISSN 2041-1723 Cited by: §2.2.
  • N. E. Zimmermann and A. Jain (2020) Local structure order parameters and site fingerprints for quantification of coordination environment and crystal structure similarity. RSC Advances 10 (10), pp. 6063–6081. Cited by: §5.
  • A. Zunger (2018) Inverse design in search of materials with target functionalities. Nature Reviews Chemistry 2 (4), pp. 1–16. Cited by: §1.