Multi-Level Fine-Tuning, Data Augmentation, and Few-Shot Learning for Specialized Cyber Threat Intelligence

07/22/2022
by   Markus Bayer, et al.
0

Gathering cyber threat intelligence from open sources is becoming increasingly important for maintaining and achieving a high level of security as systems become larger and more complex. However, these open sources are often subject to information overload. It is therefore useful to apply machine learning models that condense the amount of information to what is necessary. Yet, previous studies and applications have shown that existing classifiers are not able to extract specific information about emerging cybersecurity events due to their low generalization ability. Therefore, we propose a system to overcome this problem by training a new classifier for each new incident. Since this requires a lot of labelled data using standard training methods, we combine three different low-data regime techniques - transfer learning, data augmentation, and few-shot learning - to train a high-quality classifier from very few labelled instances. We evaluated our approach using a novel dataset derived from the Microsoft Exchange Server data breach of 2021 which was labelled by three experts. Our findings reveal an increase in F1 score of more than 21 points compared to standard training methods and more than 18 points compared to a state-of-the-art method in few-shot learning. Furthermore, the classifier trained with this method and 32 instances is only less than 5 F1 score points worse than a classifier trained with 1800 instances.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/28/2019

All you need is a good representation: A multi-level and classifier-centric representation for few-shot learning

The main problems of few-shot learning are how to learn a generalized re...
research
10/18/2021

Ortho-Shot: Low Displacement Rank Regularization with Data Augmentation for Few-Shot Learning

In few-shot classification, the primary goal is to learn representations...
research
04/01/2020

Self-Augmentation: Generalizing Deep Networks to Unseen Classes for Few-Shot Learning

Few-shot learning aims to classify unseen classes with a few training ex...
research
04/04/2019

HoloDetect: Few-Shot Learning for Error Detection

We introduce a few-shot learning framework for error detection. We show ...
research
03/17/2021

Towards Few-Shot Fact-Checking via Perplexity

Few-shot learning has drawn researchers' attention to overcome the probl...
research
02/09/2023

Zero-Shot Learning for Requirements Classification: An Exploratory Study

Context: Requirements engineering researchers have been experimenting wi...

Please sign up or login with your details

Forgot password? Click here to reset