Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling

10/01/2020
by   Yan Shvartzshnaider, et al.
0

This paper formulates a new task of extracting privacy parameters from a privacy policy, through the lens of Contextual Integrity, an established social theory framework for reasoning about privacy norms. Privacy policies, written by lawyers, are lengthy and often comprise incomplete and vague statements. In this paper, we show that traditional NLP tasks, including the recently proposed Question-Answering based solutions, are insufficient to address the privacy parameter extraction problem and provide poor precision and recall. We describe 4 different types of conventional methods that can be partially adapted to address the parameter extraction task with varying degrees of success: Hidden Markov Models, BERT fine-tuned models, Dependency Type Parsing (DP) and Semantic Role Labeling (SRL). Based on a detailed evaluation across 36 real-world privacy policies of major enterprises, we demonstrate that a solution combining syntactic DP coupled with type-specific SRL tasks provides the highest accuracy for retrieving contextual privacy parameters from privacy statements. We also observe that incorporating domain-specific knowledge is critical to achieving high precision and recall, thus inspiring new NLP research to address this important problem in the privacy domain.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

09/06/2018

Analyzing Privacy Policies Using Contextual Integrity Annotations

In this paper, we demonstrate the effectiveness of using the theory of c...
10/21/2020

Semantic Role Labeling as Syntactic Dependency Parsing

We reduce the task of (span-based) PropBank-style semantic role labeling...
03/12/2019

Evaluating the Contextual Integrity of Privacy Regulation: Parents' IoT Toy Privacy Norms Versus COPPA

Increased concern about data privacy has prompted new and updated data p...
03/06/2020

Parsing Thai Social Data: A New Challenge for Thai NLP

Dependency parsing (DP) is a task that analyzes text for syntactic struc...
12/15/2021

One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks

Preserving privacy in training modern NLP models comes at a cost. We kno...
03/14/2022

Towards Semantic Search for Community Question Answering for Mortgage Officers

Community Question Answering (CQA) has gained increasing popularity in m...
11/07/2017

The VACCINE Framework for Building DLP Systems

Conventional Data Leakage Prevention (DLP) systems suffer from the follo...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.