research
∙
11/25/2022
Solving math word problems with process- and outcome-based feedback
Recent work has shown that asking language models to generate reasoning ...
share
research
∙
10/04/2022
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
The field of AI alignment is concerned with AI systems that pursue unint...
research
∙
09/28/2022
Improving alignment of dialogue agents via targeted human judgements
We present Sparrow, an information-seeking dialogue agent trained to be ...
research
∙
06/16/2022
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models
Large language models produce human-like text that drive a growing numbe...
research
∙
12/08/2021
Ethical and social risks of harm from Language Models
This paper aims to help structure the risk landscape associated with lar...
research
∙
10/04/2021
An Empirical Investigation of Learning from Biased Toxicity Labels
Collecting annotations from human raters often results in a trade-off be...
research
∙
09/15/2021
Challenges in Detoxifying Language Models
Large language models (LM) generate remarkably fluent text and can be ef...
research
∙
02/18/2021
Verifying Probabilistic Specifications with Functional Lagrangians
We propose a general framework for verifying input-output specifications...
research
∙
11/17/2020
Avoiding Tampering Incentives in Deep RL via Decoupled Approval
How can we design agents that pursue a given objective when all feedback...
research
∙
11/17/2020
REALab: An Embedded Perspective on Tampering
This paper describes REALab, a platform for embedded agency research in ...
research
∙
10/22/2020
Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming
Convex relaxations have emerged as a promising approach for verifying de...
research
∙
10/07/2020
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
Adversarial training and its variants have become de facto standards for...
research
∙
10/21/2019
An Alternative Surrogate Loss for PGD-based Adversarial Testing
Adversarial testing methods based on Projected Gradient Descent (PGD) ar...
research
∙
05/31/2019
Are Labels Required for Improving Adversarial Robustness?
Recent work has uncovered the interesting (and somewhat surprising) find...
research
∙
02/25/2019
Verification of Non-Linear Specifications for Neural Networks
Prior work on neural network verification has focused on specifications ...
research
∙
12/04/2018
Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures
This paper addresses the problem of evaluating learning systems in safet...
research
∙
11/23/2018
Robustness via curvature regularization, and vice versa
State-of-the-art classifiers have been shown to be largely vulnerable to...
research
∙
11/22/2018
Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
While deep learning has led to remarkable results on a number of challen...
research
∙
10/30/2018
On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
Recent works have shown that it is possible to train models that are ver...
research
∙
05/25/2018
Training verified learners with learned verifiers
This paper proposes a new algorithmic framework,predictor-verifier train...
research
∙
02/15/2018
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
This paper investigates recently proposed approaches for defending again...
research
∙
10/30/2017