DeepAI AI Chat
Log In Sign Up

Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines

by   Daniel Graziotin, et al.
University of Stuttgart
Chalmers University of Technology

Humans are what constitutes the most complex and complicated, yet fascinating, component of a software engineering endeavor. A meaningful and deep understanding of the human aspects of software engineering requires psychological constructs to be taken into account. We argue that psychology and statistics theory can facilitate the development and adoption of valid and reliable instruments to assess these constructs. In particular, to ensure high quality, the psychometric properties of measurement instruments need evaluation. In this paper, we provide an introduction to psychometric theory for the evaluation of measurement instruments (e.g., psychological tests and questionnaires) for software engineering researchers. We present guidelines that enable using existing instruments and developing new ones adequately. We conducted a comprehensive review of the psychology literature, including journal articles, textbooks, and society standards, framed by the Standards for Educational and Psychological Testing (American Educational Research Association et al., 2014). We detail activities used when operationalizing new psychological constructs, such as item analysis, factor analysis, standardization and normalization, reliability, validity, and fairness in testing and test bias. With this paper, we hope to encourage a culture change in software engineering research towards the adoption of established methods from social science. To improve the quality of behavioral research in software engineering, we believe that studies focusing on introducing, validation, and then using psychometric instruments need to be more common. Finally, we present an example of a psychometric evaluation based on our guidelines, to which we openly provide code and dataset.


Behavioral Software Engineering: Methodological Introduction to Psychometrics

Humans are what constitutes the most complex and complicated, yet fascin...

Blueprint and Evaluation Instruments for a Course on Software Engineering for Sustainability

We report on a summer school course on Software Engineering for Sustaina...

Qualities of Quality: A Tertiary Review of Software Quality Measurement Research

This paper presents a tertiary review of software quality measurement re...

Supporting Modularity in Simulink Models

Model-Based Development (MBD) is widely used for embedded controls devel...

Towards Valid and Reliable Privacy Concern Scales: The Example of IUIPC-8

Valid and reliable measurement instruments are crucial for human factors...

Applying Inter-rater Reliability and Agreement in Grounded Theory Studies in Software Engineering

In recent years, the qualitative research on empirical software engineer...