Log In Sign Up

Text Mining for Processing Interview Data in Computational Social Science

by   Jussi Karlgren, et al.

We use commercially available text analysis technology to process interview text data from a computational social science study. We find that topical clustering and terminological enrichment provide for convenient exploration and quantification of the responses. This makes it possible to generate and test hypotheses and to compare textual and non-textual variables, and saves analyst effort. We encourage studies in social science to use text analysis, especially for exploratory open-ended studies. We discuss how replicability requirements are met by text analysis technology. We note that the most recent learning models are not designed with transparency in mind, and that research requires a model to be editable and its decisions to be explainable. The tools available today, such as the one used in the present study, are not built for processing interview texts. While many of the variables under consideration are quantifiable using lexical statistics, we find that some interesting and potentially valuable features are difficult or impossible to automatise reliably at present. We note that there are some potentially interesting applications for traditional natural language processing mechanisms such as named entity recognition and anaphora resolution in this application area. We conclude with a suggestion for language technologists to investigate the challenge of processing interview data comprehensively, especially the interplay between question and response, and we encourage social science researchers not to hesitate to use text analysis tools, especially for the exploratory phase of processing interview data.?


A Tidy Data Model for Natural Language Processing using cleanNLP

The package cleanNLP provides a set of fast tools for converting a textu...

Councils in Action: Automating the Curation of Municipal Governance Data for Research

Large scale comparative research into municipal governance is often proh...

A Natural Language Processing Pipeline for Detecting Informal Data References in Academic Literature

Discovering authoritative links between publications and the datasets th...

DELTA: A DEep learning based Language Technology plAtform

In this paper we present DELTA, a deep learning based language technolog...

Transformer Encoder for Social Science

High-quality text data has become an important data source for social sc...

Computational International Relations: What Can Programming, Coding and Internet Research Do for the Discipline?

Computational Social Science emerged as a highly technical and popular d...

Limited Functional Form, Misspecification, and Unreliable Interpretations in Psychology and Social Science

The replicability crisis has drawn attention to numerous weaknesses in p...