Self-learning Emulators and Eigenvector Continuation

by   Avik Sarkar, et al.
Michigan State University

Emulators that can bypass computationally expensive scientific calculations with high accuracy and speed can enable new studies of fundamental science as well as more potential applications. In this work we focus on solving a system of constraint equations efficiently using a new machine learning approach that we call self-learning emulation. A self-learning emulator is an active learning protocol that can rapidly solve a system of equations over some range of control parameters. The key ingredient is a fast estimate of the emulator error that becomes progressively more accurate as the emulator improves. This acceleration is possible because the emulator itself is used to estimate the error, and we illustrate with two examples. The first uses cubic spline interpolation to find the roots of a polynomial with variable coefficients. The second example uses eigenvector continuation to find the eigenvectors and eigenvalues of a large Hamiltonian matrix that depends on several control parameters. We envision future applications of self-learning emulators for solving systems of algebraic equations, linear and nonlinear differential equations, and linear and nonlinear eigenvalue problems.


Singular algebraic equations with empirical data

Singular equations with rank-deficient Jacobians arise frequently in alg...

Secant acceleration of sequential residual methods for solving large-scale nonlinear systems of equations

Sequential Residual Methods try to solve nonlinear systems of equations ...

Gaussian Process Assisted Active Learning of Physical Laws

In many areas of science and engineering, discovering the governing diff...

Polynomial spline collocation method for solving weakly regular Volterra integral equations of the first kind

The polynomial spline collocation method is proposed for solution of Vol...

Exact splitting methods for kinetic and Schrödinger equations

In [8], some exact splittings are proposed for inhomogeneous quadratic d...

The intrinsic Toeplitz structure and its applications in algebraic Riccati equations

In this paper we propose a new algorithm for solving large-scale algebra...

Learning from the Kernel and the Range Space

In this article, a novel approach to learning a complex function which c...


I Supplemental Material

i.1 Geometrical picture of eigenvector continuation error

We will present a geometrical picture of eigenvector continuation (EC) error as well as some additional insight into the error estimate that appears in Eq. (3) of the main text. We consider a Hamiltonian manifold that depends on the control parameters . We write for the eigenvector of interest and for the corresponding energy eigenvalue. Suppose we know the eigenvectors at different training points, . We label the set of training eigenvectors as . Let us define the norm matrix as


and let be the determinant of . Then corresponds to the square of the volume of the

-dimensional parallelopiped defined by the vectors in the set

. If all the eigenvectors are normalized, then the maximum possible volume is 1, which is attained when all the eigenvectors are orthogonal.

Let us now consider selecting the next training point, . Let be the projection operator onto the linear span of , and let be the orthogonal complement so that . Suppose we now expand our training set by adding another training vector to form . Let us define the perpendicular projection vector as


Since is the squared volume of the parallelopiped defined by the vectors in and is the squared volume of the parallelopiped defined by the vectors in , it follows that the ratio to is given by the squared norm of ,


Let us define the projections of onto and subspaces as


The EC approximation is nothing more than the approximation of by some eigenvector of , which we denote as . Let the corresponding energy be labelled so that


We also label the eigenvectors of contained in the orthogonal complement as,


When the difference between the exact eigenvector and the eigenvector continuation approximation of the eigenvector is small, we can use first order perturbation theory to write


To first order in perturbation theory, the residual vector is just . We therefore have


If we now combine with Eq. (S3), we get


We can now connect this result with the error or loss function in the main text. The second part of the equation gives an expression for the error term using first-order perturbation theory, and the first part of the equation is a geometrical interpretation of the error term as the ratio of the squared volumes, to . Taking the logarithm of the square root, we get


The term in the numerator,


will go to zero at each of the training points, causing large variations in the logarithm of the error as we add more and more training points. In contrast, the term in the denominator, , will be smooth as a function of . Similarly, will also be a smooth function of . We can write


where is a constant and averages to zero over the entire domain of . While the function is unknown, it will be dominated by the large variations in the logarithm of the error as more and more training points are added. We note that


We therefore arrive at the variance error estimate used in the main text,


i.2 Model 2 Hamiltonian

Model 2 describes four-distinguishable particles with equal masses on a three-dimensional lattice with pairwise point interactions with coefficients for each pair . We use lattice units where physical quantities are multiplied by powers of the spatial lattice spacing to make the combinations dimensionless. We take the common mass to equal in lattice units. We let denote the spatial lattice points on our three dimensional periodic lattice. Let the lattice annihilation and creation operators for particle be written as and respectively. The free non-relativistic lattice Hamiltonian has the form


We add to the free Hamiltonian single-site contact interactions, and the resulting Hamiltonian then has the form


where is the density operator for particle ,