Kevin Leyton-Brown

research

∙ 07/13/2023

Generating Benchmarks for Factuality Evaluation of Language Models

Before deploying a language model (LM) within a given domain, it is impo...

0 Dor Muhlgay, et al. ∙

research

∙ 06/07/2023

Loss Functions for Behavioral Game Theory

Behavioral game theorists all use experimental data to evaluate predicti...

0 Greg d'Eon, et al. ∙

research

∙ 01/31/2023

In-Context Retrieval-Augmented Language Models

Retrieval-Augmented Language Modeling (RALM) methods, that condition a l...

0 Ori Ram, et al. ∙

research

∙ 12/21/2022

Parallel Context Windows Improve In-Context Learning of Large Language Models

For applications that require processing large amounts of text at infere...

0 Nir Ratner, et al. ∙

research

∙ 11/22/2022

Monte Carlo Forest Search: UNSAT Solver Synthesis via Reinforcement learning

We introduce Monte Carlo Forest Search (MCFS), an offline algorithm for ...

0 Chris Cameron, et al. ∙

research

∙ 10/31/2022

Artificial Intelligence and Life in 2030: The One Hundred Year Study on Artificial Intelligence

In September 2016, Stanford's "One Hundred Year Study on Artificial Inte...

0 Peter Stone, et al. ∙

research

∙ 09/02/2022

Better Peer Grading through Bayesian Inference

Peer grading systems aggregate noisy reports from multiple students to a...

18 Hedayat Zarkoob, et al. ∙

research

∙ 05/25/2022

Formalizing Preferences Over Runtime Distributions

When trying to solve a computational problem we are often faced with a c...

5 Devon R Graham, et al. ∙

research

∙ 05/01/2022

MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning

Huge language models (LMs) have ushered in a new era for AI, serving as ...

6 Ehud Karpas, et al. ∙

research

∙ 02/24/2022

Matching Papers and Reviewers at Large Conferences

This paper studies a novel reviewer-paper matching approach that was rec...

0 Kevin Leyton-Brown, et al. ∙

research

∙ 07/01/2021

The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models

Supervised learning models often make systematic errors on rare subsets ...

16 Greg d'Eon, et al. ∙

research

∙ 06/18/2021

The Perils of Learning Before Optimizing

Formulating real-world optimization problems often begins with making pr...

10 Chris Cameron, et al. ∙

research

∙ 01/18/2021

Mechanical TA 2: A System for Peer Grading with TA Support

Mechanical TA 2 (MTA2) is an open source web-based peer grading applicat...

0 Hedayat Zarkoob, et al. ∙

research

∙ 12/01/2020

Dynamic Weighted Matching with Heterogeneous Arrival and Departure Rates

We study a dynamic non-bipartite matching problem. There is a fixed set ...

0 Natalie Collina, et al. ∙

research

∙ 11/02/2020

Exemplar Guided Active Learning

We consider the problem of wisely using a limited budget to label a smal...

8 Jason Hartford, et al. ∙

research

∙ 10/05/2020

PMI-Masking: Principled masking of correlated spans

Masking tokens uniformly at random constitutes a common flaw in the pret...

5 Yoav Levine, et al. ∙

research

∙ 06/19/2020

Valid Causal Inference with (Some) Invalid Instruments

Instrumental variable methods provide a powerful approach to estimating ...

8 Jason Hartford, et al. ∙

research

∙ 06/08/2020

Learning under Invariable Bayesian Safety

A recent body of work addresses safety constraints in explore-and-exploi...

7 Gal Bahar, et al. ∙

research

∙ 03/21/2020

Smarter Parking: Using AI to Identify Parking Inefficiencies in Vancouver

On-street parking is convenient, but has many disadvantages: on-street s...

7 Devon Graham, et al. ∙

research

∙ 06/13/2019

Report-Sensitive Spot-Checking in Peer-Grading Systems

Peer grading systems make large courses more scalable, provide students ...

0 Hedayat Zarkoob, et al. ∙

research

∙ 05/16/2019

Fiduciary Bandits

Recommendation systems often face exploration-exploitation tradeoffs: th...

0 Gal Bahar, et al. ∙

research

∙ 02/14/2019

Procrastinating with Confidence: Near-Optimal, Anytime, Adaptive Algorithm Configuration

Algorithm configuration methods optimize the performance of a parameteri...

6 Robert Kleinberg, et al. ∙

research

∙ 12/30/2018

A Formal Separation Between Strategic and Nonstrategic Behavior

It is common to make a distinction between `strategic' behavior and othe...

0 James R. Wright, et al. ∙

research

∙ 12/30/2018

Formalizing the Boundary Between Strategic and Nonstrategic Reasoning

Research in multiagent systems often does not draw a clear distinction b...

0 James R. Wright, et al. ∙

research

∙ 03/07/2018

Deep Models of Interactions Across Sets

We use deep learning to model interactions across two or more sets of ob...

0 Jason Hartford, et al. ∙

research

∙ 06/11/2017

Deep Optimization for Spectrum Repacking

Over 13 months in 2016-17 the FCC conducted an "incentive auction" to re...

0 Neil Newman, et al. ∙

research

∙ 03/30/2017

Efficient Benchmarking of Algorithm Configuration Procedures via Model-Based Surrogates

The optimization of algorithm (hyper-)parameters is crucial for achievin...

0 Katharina Eggensperger, et al. ∙

research

∙ 12/30/2016

Counterfactual Prediction with Deep Instrumental Variables Networks

We are in the middle of a remarkable rise in the use and capability of a...

0 Jason Hartford, et al. ∙

research

∙ 06/08/2015

ASlib: A Benchmark Library for Algorithm Selection

The task of algorithm selection involves choosing an algorithm from a se...

0 Bernd Bischl, et al. ∙

research

∙ 05/05/2015

The Configurable SAT Solver Challenge (CSSC)

It is well known that different solution strategies work well for differ...

0 Frank Hutter, et al. ∙

research

∙ 08/04/2014

Computational Analysis of Perfect-Information Position Auctions

After experimentation with other designs, the major search engines conve...

0 David R. M Thompson, et al. ∙

research

∙ 01/15/2014

ParamILS: An Automatic Algorithm Configuration Framework

The identification of performance-optimizing parameter settings is an im...

0 Frank Hutter, et al. ∙

research

∙ 10/07/2013

Bayesian Optimization With Censored Response Data

Bayesian optimization (BO) aims to minimize a given blackbox function us...

0 Frank Hutter, et al. ∙

research

∙ 11/05/2012

Algorithm Runtime Prediction: Methods & Evaluation

Perhaps surprisingly, it is possible to predict how long an algorithm wi...

0 Frank Hutter, et al. ∙

research

∙ 05/09/2012

Temporal Action-Graph Games: A New Representation for Dynamic Games

In this paper we introduce temporal action graph games (TAGGs), a novel ...

0 Albert Xin Jiang, et al. ∙

research

∙ 10/31/2011

SATzilla: Portfolio-based Algorithm Selection for SAT

It has been widely observed that there is no single "dominant" SAT solve...

0 Lin Xu, et al. ∙

Kevin Leyton-Brown

Featured Co-authors