DeepAI AI Chat
Log In Sign Up

Robustness Tests of NLP Machine Learning Models: Search and Semantically Replace

by   Rahul Singh, et al.

This paper proposes a strategy to assess the robustness of different machine learning models that involve natural language processing (NLP). The overall approach relies upon a Search and Semantically Replace strategy that consists of two steps: (1) Search, which identifies important parts in the text; (2) Semantically Replace, which finds replacements for the important parts, and constrains the replaced tokens with semantically similar words. We introduce different types of Search and Semantically Replace methods designed specifically for particular types of machine learning models. We also investigate the effectiveness of this strategy and provide a general framework to assess a variety of machine learning models. Finally, an empirical comparison is provided of robustness performance among three different model types, each with a different text representation.


page 7

page 8

page 9

page 10

page 11

page 16

page 17

page 18


An Overview and Prospective Outlook on Robust Training and Certification of Machine Learning Models

In this discussion paper, we survey recent research surrounding robustne...

Training Natural Language Processing Models on Encrypted Text for Enhanced Privacy

With the increasing use of cloud-based services for training and deployi...

Deep learning models are not robust against noise in clinical text

Artificial Intelligence (AI) systems are attracting increasing interest ...

Replace or Retrieve Keywords In Documents at Scale

In this paper we introduce, the FlashText algorithm for replacing keywor...

Natural Backdoor Attack on Text Data

Deep learning has been widely adopted in natural language processing app...

Extending the Abstraction of Personality Types based on MBTI with Machine Learning and Natural Language Processing

A data-centric approach with Natural Language Processing (NLP) to predic...

DeltaGrad: Rapid retraining of machine learning models

Machine learning models are not static and may need to be retrained on s...