Large-scale Pre-trained Models are Surprisingly Strong in Incremental Novel Class Discovery

03/28/2023
by   Mingxuan Liu, et al.
0

Discovering novel concepts from unlabelled data and in a continuous manner is an important desideratum of lifelong learners. In the literature such problems have been partially addressed under very restricted settings, where either access to labelled data is provided for discovering novel concepts (e.g., NCD) or learning occurs for a limited number of incremental steps (e.g., class-iNCD). In this work we challenge the status quo and propose a more challenging and practical learning paradigm called MSc-iNCD, where learning occurs continuously and unsupervisedly, while exploiting the rich priors from large-scale pre-trained models. To this end, we propose simple baselines that are not only resilient under longer learning scenarios, but are surprisingly strong when compared with sophisticated state-of-the-art methods. We conduct extensive empirical evaluation on a multitude of benchmarks and show the effectiveness of our proposed baselines, which significantly raises the bar.

READ FULL TEXT
research
04/07/2022

Class-Incremental Learning with Strong Pre-trained Models

Class-incremental learning (CIL) has been widely studied under the setti...
research
03/13/2023

Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need

Class-incremental learning (CIL) aims to adapt to emerging new classes w...
research
12/24/2022

Boosting Out-of-Distribution Detection with Multiple Pre-trained Models

Out-of-Distribution (OOD) detection, i.e., identifying whether an input ...
research
04/20/2023

Learning Sample Difficulty from Pre-trained Models for Reliable Prediction

Large-scale pre-trained models have achieved remarkable success in a var...
research
07/18/2022

Class-incremental Novel Class Discovery

We study the new task of class-incremental Novel Class Discovery (class-...
research
04/06/2023

Learning Neural Eigenfunctions for Unsupervised Semantic Segmentation

Unsupervised semantic segmentation is a long-standing challenge in compu...
research
08/21/2023

Towards Accelerated Model Training via Bayesian Data Selection

Mislabeled, duplicated, or biased data in real-world scenarios can lead ...

Please sign up or login with your details

Forgot password? Click here to reset