Behavioral Software Engineering: Methodological Introduction to Psychometrics

by   Daniel Graziotin, et al.
University of Stuttgart
Chalmers University of Technology

Humans are what constitutes the most complex and complicated, yet fascinating, component of a software engineering endeavor. A meaningful and deep understanding of the human aspects of software engineering requires psychological constructs to be taken into account. We argue that psychology and statistics theory would facilitate the development and adoption of valid and reliable instruments to assess these constructs. In particular, to ensure high quality, the psychometric properties of measurement instruments need evaluation. In this paper, we provide an introduction to psychometric theory for the evaluation of measurement instruments (e.g., psychological tests and questionnaires) for software engineering researchers. We present guidelines that enable using existing instruments and developing new ones adequately. We conducted a field survey of the psychology literature, including journal articles, textbooks, and society standards, framed by the Standards for Educational and Psychological Testing (American Educational Research Association et al, 2014). We detail activities used when operationalizing new psychological constructs, such as item analysis, factor analysis, standardization and normalization, reliability, validity, and fairness in testing and test bias. With this paper, we hope to encourage a culture change in software engineering research towards the adoption of established methods from social science. To improve the quality of behavioral research, we believe that studies focusing on introducing and psychometrically validating measurement instruments need to be more common. Finally, we present an example of a psychometric evaluation based on our guidelines, to which we openly provide code and dataset.


page 1

page 5


Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines

Humans are what constitutes the most complex and complicated, yet fascin...

Blueprint and Evaluation Instruments for a Course on Software Engineering for Sustainability

We report on a summer school course on Software Engineering for Sustaina...

On Psychometric Instruments in Software Engineering Research: An Ongoing Study

[Context] Although software development is an inherently human activity,...

Towards Valid and Reliable Privacy Concern Scales: The Example of IUIPC-8

Valid and reliable measurement instruments are crucial for human factors...

Psychometric Instruments in Software Engineering Research on Personality: Status Quo After Fifty Years

Context: Although software development is a human activity, Software Eng...

Qualities of Quality: A Tertiary Review of Software Quality Measurement Research

This paper presents a tertiary review of software quality measurement re...

The First 50 Years of Software Reliability Engineering: A History of SRE with First Person Accounts

Software Reliability has just passed the 50-year milestone as a technica...

Please sign up or login with your details

Forgot password? Click here to reset