DeepAI AI Chat
Log In Sign Up

Towards Personalized Intelligence at Scale

by   Yiping Kang, et al.
University of Michigan

Personalized Intelligence (PI) is the problem of providing customized AI experiences tailored to each individual user. In many applications, PI is preferred or even required. Existing personalization approaches involve fine-tuning pre-trained models to create new customized models. However, these approaches require a significant amount of computation to train, scaling with model size and the number of users, inhibiting PI to be realized widely. In this work, we introduce a novel model architecture and training/inference framework to enable Personalized Intelligence at scale. We achieve this by attaching a Personalization Head (PH) to pre-trained language models (LM). During training, the base LMs are frozen and only the parameters in PH are updated and are unique per user. This results in significantly smaller overall model sizes and training cost than traditional fine-tuning approaches when scaled across many users. We evaluate PHs on academia and industry-focused datasets and show that the PHs outperform zeroshot baseline in F1 score and are significantly more scalable than traditional fine-tuning approaches. We identify key factors required for effective PH design and training.


page 1

page 2

page 3

page 4


The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models

In this paper, we explore the effects of language variants, data sizes, ...

When does Parameter-Efficient Transfer Learning Work for Machine Translation?

Parameter-efficient fine-tuning methods (PEFTs) offer the promise of ada...

A Stability Analysis of Fine-Tuning a Pre-Trained Model

Fine-tuning a pre-trained model (such as BERT, ALBERT, RoBERTa, T5, GPT,...

DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning

Diffusion models have proven to be highly effective in generating high-q...

My3DGen: Building Lightweight Personalized 3D Generative Model

Our paper presents My3DGen, a practical system for creating a personaliz...

Large Language Models and the Reverse Turing Test

Large Language Models (LLMs) have been transformative. They are pre-trai...