LMdiff: A Visual Diff Tool to Compare Language Models

11/02/2021
by   Hendrik Strobelt, et al.
7

While different language models are ubiquitous in NLP, it is hard to contrast their outputs and identify which contexts one can handle better than the other. To address this question, we introduce LMdiff, a tool that visually compares probability distributions of two models that differ, e.g., through finetuning, distillation, or simply training with different parameter sizes. LMdiff allows the generation of hypotheses about model behavior by investigating text instances token by token and further assists in choosing these interesting text instances by identifying the most interesting phrases from large corpora. We showcase the applicability of LMdiff for hypothesis generation across multiple case studies. A demo is available at http://lmdiff.net .

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/21/2022

Language models are better than humans at next-token prediction

Current language models are considered to have sub-human capabilities at...
research
05/31/2021

Language Model Evaluation Beyond Perplexity

We propose an alternate approach to quantifying how well language models...
research
08/13/2023

Token-Scaled Logit Distillation for Ternary Weight Generative Language Models

Generative Language Models (GLMs) have shown impressive performance in t...
research
06/01/2023

EEL: Efficiently Encoding Lattices for Reranking

Standard decoding approaches for conditional text generation tasks typic...
research
12/15/2022

Visually-augmented pretrained language models for NLP tasks without images

Although pre-trained language models (PLMs) have shown impressive perfor...
research
09/15/2023

Headless Language Models: Learning without Predicting with Contrastive Weight Tying

Self-supervised pre-training of language models usually consists in pred...
research
07/10/2020

Neural Composition: Learning to Generate from Multiple Models

Decomposing models into multiple components is critically important in m...

Please sign up or login with your details

Forgot password? Click here to reset