Validation design I: construction of validation designs via kernel herding
share
We construct validation designs Z_m aimed at estimating the integrated squared prediction error of a given design X_n. Our approach is based on the minimization of a maximum mean discrepancy for a particular kernel, conditional on X_n, so that sequences of nested validation designs can be constructed incrementally by kernel herding. Numerical experiments show that key features for a good validation design are its space-filling properties, in order to fill the holes left by X_n and properly explore the whole design space, and the suitable weighting of its points, since evaluations far from X_n tend to overestimate the global error. A dedicated weighting method, based on a particular kernel, is proposed. Numerical simulations with random functions show the superiority the method over more traditional validation based on random designs, low-discrepancy sequences, or leave-one-out cross validation.
READ FULL TEXT