Model evaluation for extreme risks

05/24/2023
by   Toby Shevlane, et al.
0

Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities. Further progress in AI development could lead to capabilities that pose extreme risks, such as offensive cyber capabilities or strong manipulation skills. We explain why model evaluation is critical for addressing extreme risks. Developers must be able to identify dangerous capabilities (through "dangerous capability evaluations") and the propensity of models to apply their capabilities for harm (through "alignment evaluations"). These evaluations will become critical for keeping policymakers and other stakeholders informed, and for making responsible decisions about model training, deployment, and security.

READ FULL TEXT

page 1

page 3

page 5

page 6

research
07/06/2023

Frontier AI Regulation: Managing Emerging Risks to Public Safety

Advanced AI models hold the promise of tremendous benefits for humanity,...
research
10/20/2020

Artificial Tikkun Olam: AI Can Be Our Best Friend in Building an Open Human-Computer Society

Technological advances of virtually every kind pose risks to society inc...
research
04/02/2016

The AGI Containment Problem

There is considerable uncertainty about what properties, capabilities an...
research
06/24/2023

Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools

As advancements in artificial intelligence propel progress in the life s...
research
03/15/2021

Modeling Multivariate Cyber Risks: Deep Learning Dating Extreme Value Theory

Modeling cyber risks has been an important but challenging task in the d...
research
11/26/2021

Machines and Influence

Policymakers face a broader challenge of how to view AI capabilities tod...
research
12/01/2021

Collaborative AI Needs Stronger Assurances Driven by Risks

Collaborative AI systems (CAISs) aim at working together with humans in ...

Please sign up or login with your details

Forgot password? Click here to reset