Synthesizing Machine Learning Programs with PAC Guarantees via Statistical Sketching

10/11/2021
by   Osbert Bastani, et al.
0

We study the problem of synthesizing programs that include machine learning components such as deep neural networks (DNNs). We focus on statistical properties, which are properties expected to hold with high probability – e.g., that an image classification model correctly identifies people in images with high probability. We propose novel algorithms for sketching and synthesizing such programs by leveraging ideas from statistical learning theory to provide statistical soundness guarantees. We evaluate our approach on synthesizing list processing programs that include DNN components used to process image inputs, as well as case studies on image classification and on precision medicine. Our results demonstrate that our approach can be used to synthesize programs with probabilistic guarantees.

READ FULL TEXT
research
12/31/2019

PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction

We propose an algorithm combining calibrated prediction and generalizati...
research
08/31/2021

Synbit: Synthesizing Bidirectional Programs using Unidirectional Sketches

We propose a technique for synthesizing bidirectional programs from the ...
research
10/06/2020

A Note on High-Probability versus In-Expectation Guarantees of Generalization Bounds in Machine Learning

Statistical machine learning theory often tries to give generalization g...
research
11/02/2020

PAC Confidence Predictions for Deep Neural Network Classifiers

A key challenge for deploying deep neural networks (DNNs) in safety crit...
research
12/02/2016

Probabilistic Neural Programs

We present probabilistic neural programs, a framework for program induct...
research
01/27/2023

Alien Coding

We introduce a self-learning algorithm for synthesizing programs for OEI...
research
07/29/2023

An Automata-Theoretic Approach to Synthesizing Binarized Neural Networks

Deep neural networks, (DNNs, a.k.a. NNs), have been widely used in vario...

Please sign up or login with your details

Forgot password? Click here to reset