Standards of Validity and the Validity of Standards in Behavioral Software Engineering Research: The Perspective of Psychological Test Theory

by   Lucas Gren, et al.
Göteborgs universitet

Background. There are some publications in software engineering research that aim at guiding researchers in assessing validity threats to their studies. Still, many researchers fail to address many aspects of validity that are essential to quantitative research on human factors. Goal. This paper has the goal of triggering a change of mindset in what types of studies are the most valuable to the behavioral software engineering field, and also provide more details of what construct validity is. Method. The approach is based on psychological test theory and draws upon methods used in psychology in relation to construct validity. Results. In this paper, I suggest a different approach to validity threats than what is commonplace in behavioral software engineering research. Conclusions. While this paper focuses on behavioral software engineering, I believe other types of software engineering research might also benefit from an increased focus on construct validity.


page 1

page 2

page 3

page 4


Improving the Reporting of Threats to Construct Validity

Background: Construct validity concerns the use of indicators to measure...

Evidence Profiles for Validity Threats in Program Comprehension Experiments

Searching for clues, gathering evidence, and reviewing case files are al...

LLM is Like a Box of Chocolates: the Non-determinism of ChatGPT in Code Generation

There has been a recent explosion of research on Large Language Models (...

Bad Smells in Software Analytics Papers

CONTEXT: There has been a rapid growth in the use of data analytics to u...

Neural Network-based exploration of construct validity for Russian version of the 10-item Big Five Inventory

This study aims to present a new method of exploring construct validity ...

Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines

Humans are what constitutes the most complex and complicated, yet fascin...

Please sign up or login with your details

Forgot password? Click here to reset