GRAM: Fast Fine-tuning of Pre-trained Language Models for Content-based Collaborative Filtering

04/08/2022
by   Yoonseok Yang, et al.
0

Content-based collaborative filtering (CCF) predicts user-item interactions based on both users' interaction history and items' content information. Recently, pre-trained language models (PLM) have been used to extract high-quality item encodings for CCF. However, it is resource-intensive to train a PLM-based CCF model in an end-to-end (E2E) manner, since optimization involves back-propagating through every content encoding within a given user interaction sequence. To tackle this issue, we propose GRAM (GRadient Accumulation for Multi-modality in CCF), which exploits the fact that a given item often appears multiple times within a batch of interaction histories. Specifically, Single-step GRAM aggregates each item encoding's gradients for back-propagation, with theoretic equivalence to the standard E2E training. As an extension of Single-step GRAM, we propose Multi-step GRAM, which increases the gradient update latency, achieving a further speedup with drastically less GPU memory. GRAM significantly improves training efficiency (up to 146x) on five datasets from two task domains of Knowledge Tracing and News Recommendation. Our code is available at https://github.com/yoonseok312/GRAM.

READ FULL TEXT
research
09/03/2023

Pre-trained Neural Recommenders: A Transferable Zero-Shot Framework for Recommendation Systems

Modern neural collaborative filtering techniques are critical to the suc...
research
08/02/2023

Knowledge-aware Collaborative Filtering with Pre-trained Language Model for Personalized Review-based Rating Prediction

Personalized review-based rating prediction aims at leveraging existing ...
research
09/10/2019

Wasserstein Collaborative Filtering for Item Cold-start Recommendation

The item cold-start problem seriously limits the recommendation performa...
research
04/06/2023

Graph Collaborative Signals Denoising and Augmentation for Recommendation

Graph collaborative filtering (GCF) is a popular technique for capturing...
research
07/03/2020

Disentangled Graph Collaborative Filtering

Learning informative representations of users and items from the interac...
research
07/13/2021

Learning to Recommend Items to Wikidata Editors

Wikidata is an open knowledge graph built by a global community of volun...
research
12/08/2021

Zero-Shot Recommendation as Language Modeling

Recommendation is the task of ranking items (e.g. movies or products) ac...

Please sign up or login with your details

Forgot password? Click here to reset