Thomas Hofmann

research

∙ 09/20/2023

The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute

The Languini Kitchen serves as both a research collective and codebase d...

0 Aleksandar Stanić, et al. ∙

research

∙ 06/30/2023

The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit

In deep learning theory, the covariance matrix of the representations se...

0 Lorenzo Noci, et al. ∙

research

∙ 06/23/2023

Scaling MLPs: A Tale of Inductive Bias

In this work we revisit the most fundamental building block in deep lear...

0 Gregor Bachmann, et al. ∙

research

∙ 06/04/2023

Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D Scenes

Training models to apply common-sense linguistic knowledge and visual co...

0 Alexandros Delitzas, et al. ∙

research

∙ 05/16/2023

The Hessian perspective into the Nature of Convolutional Neural Networks

While Convolutional Neural Networks (CNNs) have long been investigated a...

0 Sidak Pal Singh, et al. ∙

research

∙ 04/12/2023

CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes

Training models to apply linguistic knowledge and visual concepts from 2...

0 Maria Parelli, et al. ∙

research

∙ 03/16/2023

Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learning

In contrast to the natural capabilities of humans to learn new tasks in ...

0 Sanghwan Kim, et al. ∙

research

∙ 02/23/2023

Random Teachers are Good Teachers

In this work, we investigate the implicit regularization induced by teac...

0 Felix Sarnthein, et al. ∙

research

∙ 11/22/2022

Cosmology from Galaxy Redshift Surveys with PointNet

In recent years, deep learning approaches have achieved state-of-the-art...

0 Sotiris Anagnostidis, et al. ∙

research

∙ 10/25/2022

The Curious Case of Benign Memorization

Despite the empirical advances of deep learning across a variety of lear...

0 Sotiris Anagnostidis, et al. ∙

research

∙ 10/21/2022

Decoding a Neural Retriever's Latent Space for Query Suggestion

Neural retrieval models have superseded classic bag-of-words methods suc...

0 Leonard Adolphs, et al. ∙

research

∙ 10/03/2022

Mastering Spatial Graph Prediction of Road Networks

Accurately predicting road networks from satellite images requires a glo...

0 Sotiris Anagnostidis, et al. ∙

research

∙ 07/19/2022

OpenFilter: A Framework to Democratize Research Access to Social Media AR Filters

Augmented Reality or AR filters on selfies have become very popular on s...

11 Piera Riccio, et al. ∙

research

∙ 05/27/2022

How Tempering Fixes Data Augmentation in Bayesian Neural Networks

While Bayesian neural networks (BNNs) provide a sound and principled alt...

0 Gregor Bachmann, et al. ∙

research

∙ 03/14/2022

Phenomenology of Double Descent in Finite-Width Neural Networks

`Double descent' delineates the generalization behaviour of models depen...

4 Sidak Pal Singh, et al. ∙

research

∙ 03/07/2022

Generalization Through The Lens Of Leave-One-Out Error

Despite the tremendous empirical success of deep learning models to solv...

0 Gregor Bachmann, et al. ∙

research

∙ 01/26/2022

FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control

Generating music with deep neural networks has been an area of active re...

23 Dimitri von Rütte, et al. ∙

research

∙ 01/02/2022

Randomized Signature Layers for Signal Extraction in Time Series Data

Time series analysis is a widespread task in Natural Sciences, Social Sc...

0 Enea Monzio Compagnoni, et al. ∙

research

∙ 09/01/2021

Boosting Search Engines with Interactive Agents

Can machines learn to use a search engine as an interactive tool for fin...

83 Leonard Adolphs, et al. ∙

research

∙ 08/04/2021

How to Query Language Models?

Large pre-trained language models (LMs) are capable of not only recoveri...

0 Leonard Adolphs, et al. ∙

research

∙ 06/30/2021

Analytic Insights into Structure and Rank of Neural Network Hessian Maps

The Hessian of a neural network captures parameter interactions through ...

0 Sidak Pal Singh, et al. ∙

research

∙ 06/11/2021

Precise characterization of the prior predictive distribution of deep ReLU networks

Recent works on Bayesian neural networks (BNNs) have highlighted the nee...

0 Lorenzo Noci, et al. ∙

research

∙ 06/11/2021

Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effect

The "cold posterior effect" (CPE) in Bayesian deep learning describes th...

0 Lorenzo Noci, et al. ∙

research

∙ 05/07/2021

Uniform Convergence, Adversarial Spheres and a Simple Remedy

Previous work has cast doubt on the general framework of uniform converg...

0 Gregor Bachmann, et al. ∙

research

∙ 03/23/2021

Generative Minimization Networks: Training GANs Without Competition

Many applications in machine learning can be framed as minimization prob...

31 Paulina Grnarova, et al. ∙

research

∙ 03/21/2021

SwissDial: Parallel Multidialectal Corpus of Spoken Swiss German

Swiss German is a dialect continuum whose natively acquired dialects sig...

0 Pelin Dogan-Schönberger, et al. ∙

research

∙ 02/23/2021

Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization

Viewing optimization methods as numerical integrators for ordinary diffe...

0 Peiyuan Zhang, et al. ∙

research

∙ 06/13/2020

Convolutional Generation of Textured 3D Meshes

Recent generative models for 2D images achieve impressive visual results...

7 Dario Pavllo, et al. ∙

research

∙ 03/05/2020

BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward

Measuring the quality of a generated sequence against a set of reference...

0 Florian Schmidt, et al. ∙

research

∙ 12/06/2019

Controlling Style and Semantics in Weakly-Supervised Image Generation

We propose a weakly-supervised approach for conditional image generation...

20 Dario Pavllo, et al. ∙

research

∙ 10/31/2019

Mixing of Stochastic Accelerated Gradient Descent

We study the mixing properties for stochastic accelerated gradient desce...

0 Peiyuan Zhang, et al. ∙

research

∙ 09/04/2019

LeDeepChef: Deep Reinforcement Learning Agent for Families of Text-Based Games

While Reinforcement Learning (RL) approaches lead to significant achieve...

0 Leonard Adolphs, et al. ∙

research

∙ 08/30/2019

Autoregressive Text Generation Beyond Feedback Loops

Autoregressive state transitions, where predictions are conditioned on p...

0 Florian Schmidt, et al. ∙

research

∙ 08/15/2019

Cosmological N-body simulations: a challenge for scalable generative models

Deep generative models, such as Generative Adversarial Networks (GANs) o...

0 Nathanaël Perraudin, et al. ∙

research

∙ 06/04/2019

Adversarial Training Generalizes Data-dependent Spectral Norm Regularization

We establish a theoretical link between adversarial training and operato...

12 Kevin Roth, et al. ∙

research

∙ 02/13/2019

The Odds are Odd: A Statistical Test for Detecting Adversarial Examples

We investigate conditions under which test statistics exist that can rel...

18 Kevin Roth, et al. ∙

research

∙ 11/13/2018

Evaluating GANs via Duality

Generative Adversarial Networks (GANs) have shown great results in accur...

6 Paulina Grnarova, et al. ∙

research

∙ 09/23/2018

Learning and Evaluating Sparse Interpretable Sentence Embeddings

Previous research on word embeddings has shown that sparse representatio...

0 Valentin Trifonov, et al. ∙

research

∙ 08/23/2018

End-to-End Neural Entity Linking

Entity Linking (EL) is an essential task for semantic text understanding...

0 Nikolaos Kolitsas, et al. ∙

research

∙ 06/20/2018

A Distributed Second-Order Algorithm You Can Trust

Due to the rapid growth of data and computational resources, distributed...

2 Celestine Dünner, et al. ∙

research

∙ 06/12/2018

Deep State Space Models for Unconditional Word Generation

Autoregressive feedback is considered a necessity for successful uncondi...

0 Florian Schmidt, et al. ∙

research

∙ 05/27/2018

Towards a Theoretical Understanding of Batch Normalization

Normalization techniques such as Batch Normalization have been applied v...

0 Hadi Daneshmand, et al. ∙

research

∙ 05/25/2018

Zero-Shot Dual Machine Translation

Neural Machine Translation (NMT) systems rely on large amounts of parall...

0 Lierni Sestorain, et al. ∙

research

∙ 05/23/2018

Hyperbolic Neural Networks

Hyperbolic spaces have recently gained momentum in the context of machin...

0 Octavian-Eugen Ganea, et al. ∙

research

∙ 05/22/2018

Adversarially Robust Training through Structured Gradient Regularization

We propose a novel data-dependent structured gradient regularizer to inc...

2 Kevin Roth, et al. ∙

research

∙ 05/15/2018

Local Saddle Point Optimization: A Curvature Exploitation Approach

Gradient-based optimization methods are the most popular choice for find...

0 Leonard Adolphs, et al. ∙

research

∙ 04/03/2018

Hyperbolic Entailment Cones for Learning Hierarchical Embeddings

Learning graph representations via low-dimensional embeddings that prese...

0 Octavian-Eugen Ganea, et al. ∙

research

∙ 03/15/2018

Escaping Saddles with Stochastic Gradients

We analyze the variance of stochastic gradients along negative curvature...

0 Hadi Daneshmand, et al. ∙

research

∙ 01/27/2018

Fast Cosmic Web Simulations with Generative Adversarial Networks

Dark matter in the universe evolves through gravity to form a complex ne...

0 Andres C Rodriguez, et al. ∙

research

∙ 11/15/2017

The best defense is a good offense: Countering black box attacks by predicting slightly wrong labels

Black-Box attacks on machine learning models occur when an attacker, des...

0 Yannic Kilcher, et al. ∙

Thomas Hofmann

Featured Co-authors