research
∙
07/25/2023
QuIP: 2-Bit Quantization of Large Language Models With Guarantees
This work studies post-training parameter quantization in large language...
share
research
∙
06/14/2023
InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models
While diffusion models excel at generating high-quality samples, their l...
research
∙
06/13/2023
TART: A plug-and-play Transformer module for task-agnostic reasoning
Large language models (LLMs) exhibit in-context learning abilities which...
research
∙
06/01/2023
Coneheads: Hierarchy Aware Attention
Attention networks such as transformers have achieved state-of-the-art p...
research
∙
05/24/2023
Shadow Cones: Unveiling Partial Orders in Hyperbolic Space
Hyperbolic space has been shown to produce superior low-dimensional embe...
research
∙
01/27/2023
Variance, Self-Consistency, and Arbitrariness in Fair Classification
In fair classification, it is common to train a model, and to compare an...
research
∙
07/18/2022
MCTensor: A High-Precision Deep Learning Library with Multi-Component Floating-Point
In this paper, we introduce MCTensor, a library based on PyTorch for pro...
research
∙
06/23/2022
Non-Determinism and the Lawlessness of ML Code
Legal literature on machine learning (ML) tends to focus on harms, and a...
research
∙
06/20/2022
Low-Precision Stochastic Gradient Langevin Dynamics
While low-precision optimization has been widely used to accelerate deep...
research
∙
03/04/2022
Structured Pruning is All You Need for Pruning CNNs at Initialization
Pruning is a popular technique for reducing the model size and computati...
research
∙
02/14/2022
HyLa: Hyperbolic Laplacian Features For Graph Learning
Due to its geometric properties, hyperbolic space can support high-fidel...
research
∙
02/10/2022
Understanding Hyperdimensional Computing for Parallel Single-Pass Learning
Hyperdimensional computing (HDC) is an emerging learning paradigm that c...
research
∙
09/22/2021
Tecnologica cosa: Modeling Storyteller Personalities in Boccaccio's Decameron
We explore Boccaccio's Decameron to see how digital humanities tools can...
research
∙
07/19/2021
Equivariant Manifold Flows
Tractably modelling distributions over manifolds has long been an import...
research
∙
06/17/2021
How Low Can We Go: Trading Memory for Error in Low-Precision Training
Low-precision arithmetic trains deep learning models using less energy, ...
research
∙
02/26/2021
Low-Precision Reinforcement Learning
Low-precision training has become a popular approach to reduce computati...
research
∙
10/13/2020
Revisiting BFloat16 Training
State-of-the-art generic low-precision training algorithms use a mix of ...
research
∙
07/06/2020
Meta-Learning for Variational Inference
Variational inference (VI) plays an essential role in approximate Bayesi...
research
∙
07/04/2020
Regulating Accuracy-Efficiency Trade-Offs in Distributed Machine Learning Systems
In this paper we discuss the trade-off between accuracy and efficiency i...
research
∙
06/20/2020
Asymptotically Optimal Exact Minibatch Metropolis-Hastings
Metropolis-Hastings (MH) is a commonly-used MCMC algorithm, but it can b...
research
∙
06/18/2020
Neural Manifold Ordinary Differential Equations
To better conform to data geometry, recent deep generative modelling tec...
research
∙
05/14/2020
MixML: A Unified Analysis of Weakly Consistent Parallel Learning
Parallelism is a ubiquitous method for accelerating machine learning alg...
research
∙
03/05/2020
Optimizing JPEG Quantization for Classification Networks
Deep learning for computer vision depends on lossy image compression: it...
research
∙
02/29/2020
Differentiating through the Fréchet Mean
Recent advances in deep representation learning on Riemannian manifolds ...
research
∙
02/29/2020
AMAGOLD: Amortized Metropolis Adjustment for Efficient Stochastic Gradient MCMC
Stochastic gradient Hamiltonian Monte Carlo (SGHMC) is an efficient meth...
research
∙
02/26/2020
Moniqua: Modulo Quantized Communication in Decentralized SGD
Running Stochastic Gradient Descent (SGD) in a decentralized fashion has...
research
∙
11/21/2019
Poisson-Minibatching for Gibbs Sampling with Convergence Rate Guarantees
Gibbs sampling is a Markov chain Monte Carlo method that is often used f...
research
∙
10/13/2019
Overwrite Quantization: Opportunistic Outlier Handling for Neural Network Accelerators
Outliers in weights and activations pose a key challenge for fixed-point...
research
∙
10/09/2019
PipeMare: Asynchronous Pipeline Parallel DNN Training
Recently there has been a flurry of interest around using pipeline paral...
research
∙
10/09/2019
QPyTorch: A Low-Precision Arithmetic Simulation Framework
Low-precision training reduces computational cost and produces efficient...
research
∙
04/26/2019
SWALP : Stochastic Weight Averaging in Low-Precision Training
Low precision operations can provide scalability, memory savings, portab...
research
∙
03/29/2019
SysML: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoptio...
research
∙
02/28/2019
Distributed Learning with Sublinear Communication
In distributed statistical learning, N samples are split across m machin...
research
∙
01/28/2019
Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
Quantization can improve the execution latency and energy efficiency of ...
research
∙
01/28/2019
Improving Neural Network Quantization using Outlier Channel Splitting
Quantization can improve the execution latency and energy efficiency of ...
research
∙
11/19/2018
Building Efficient Deep Neural Networks with Unitary Group Convolutions
We propose unitary group convolutions (UGConvs), a building block for CN...
research
∙
06/15/2018
Minibatch Gibbs Sampling on Large Graphical Models
Gibbs sampling is the de facto Markov chain Monte Carlo method used for ...
research
∙
05/29/2018
Channel Gating Neural Networks
Employing deep neural networks to obtain state-of-the-art performance on...
research
∙
04/10/2018
Representation Tradeoffs for Hyperbolic Embeddings
Hyperbolic embeddings offer excellent quality with few dimensions when e...
research
∙
03/23/2018
The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory
Stochastic Gradient Descent (SGD) is a fundamental algorithm in machine ...
research
∙
03/16/2018
A Kernel Theory of Modern Data Augmentation
Data augmentation, a technique in which a training set is expanded with ...
research
∙
03/09/2018
High-Accuracy Low-Precision Training
Low-precision computation is often used to lower the time and energy cos...
research
∙
01/21/2018
A Formal Framework For Probabilistic Unclean Databases
Traditional modeling of inconsistency in database theory casts all possi...
research
∙
07/10/2017
Accelerated Stochastic Power Iteration
Principal component analysis (PCA) is one of the most powerful tools in ...
research
∙
10/25/2016
Socratic Learning: Augmenting Generative Models to Incorporate Latent Subsets in Training Data
A challenge in training discriminative models like neural networks is ob...
research
∙
06/23/2016
Parallel SGD: When does averaging help?
Consider a number of workers running SGD independently on the same pool ...
research
∙
06/10/2016
Scan Order in Gibbs Sampling: Models in Which it Matters and Bounds on How Much
Gibbs sampling is a Markov Chain Monte Carlo sampling technique that ite...
research
∙
05/25/2016
Data Programming: Creating Large Training Sets, Quickly
Large labeled training sets are the critical building blocks of supervis...
research
∙
06/22/2015
Taming the Wild: A Unified Analysis of Hogwild!-Style Algorithms
Stochastic gradient descent (SGD) is a ubiquitous algorithm for a variet...
research
∙
02/03/2015