Pandemic Programming: How COVID-19 affects software developers and how their organizations can help

05/03/2020 ∙ by Paul Ralph, et al. ∙ JetBrains KAIST 수리과학과 puc-rio The University of Adelaide University of Bari Aldo Moro Design and Development by: Monash University New York Institute of Technology Peking University Göteborgs universitet Dalhousie University Delft University of Technology 0

Context. As a novel coronavirus swept the world in early 2020, thousands of software developers began working from home. Many did so on short notice, under difficult and stressful conditions. Objective. This paper seeks to understand the effects of the pandemic on developers' wellbeing and productivity. Method. A questionnaire survey was created mainly from existing, validated scales. The questionnaire ran in 12 languages, with region-specific advertising strategies. The data was analyzed using non-parametric inferential statistics and structural equation modeling. Results. The questionnaire received 2225 usable responses from 53 countries. Factor analysis supported the validity of the scales and the structural model achieved a good fit (CFI = 0.961, RMSEA = 0.051, SRMR = 0.067). Findings include: (1) developers' wellbeing and productivity are suffering; (2) productivity and wellbeing are closely related; (3) disaster preparedness, fear related to the pandemic and home office ergonomics all affect wellbeing or productivity; (4) women, parents and people with disabilities may be disproportionately affected. Conclusions. To improve employee productivity, software companies should focus on maximizing employee wellbeing and improving the ergonomics of employees' home offices. Women, parents and disabled persons may require extra support.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

In December 2019, a novel coronavirus disease (COVID-19) emerged in Wuhan, China. While symptoms vary, COVID-19 often produces fever, cough, shortness of breath, and in some cases, pneumonia and death. By April 30, 2020, The World Health Organization (WHO) recorded more than 3 million confirmed cases and 217,769 deaths (WHO, 2020). With wide-spread transmissions in 214 countries, territories or areas, the WHO declared it a Public Health Emergency of International Concern (WHO, 2020) and many jurisdictions declared states of emergency or lockdowns (Kaplan et al., 2020). Many technology companies told their employees to work from home (Duffy, 2020).


Quarantine work !== Remote work. I’ve been working remotely with success for 13 years, and I’ve never been close to burn out. I’ve been working quarantined for over a month and I’m feeling a tinge of burn out for the first time in my life. Take care of yourself folks. Really.
–Scott Hanselman (April 19, 2020)

Thinking of this situation as a global natural experiment in working from home—the event that would prove once and for all that working from home, well, works—would be naïve. This is not normal working from home. This is working from home, unexpectedly, during an unprecedented crisis. The normal benefits of working from home do not apply (Donnelly and Proctor-Thomson, 2015). Rather than working in a remote office or well-appointed home office, some people are working in impromptu in bedrooms, at kitchen tables and on sofas while partners, children, siblings, parents, roommates, and pets distract them. Others are spending all day alone in a studio or one-bedroom apartment. With schools and daycare closed, many parents must juggle work with not only childcare but also home schooling or monitoring remote schooling activities and keeping children engaged. Some professionals have the virus or are caring for family members with the virus.

While numerous studies have investigated remote work, few investigate working from home during disasters. There are no modern studies of working from home during a pandemic of this magnitude because there has not been a pandemic of this magnitude since before there was a world wide web. Therefore, software companies have limited evidence on which to base attempts to support their workers through this crisis, which raises the following research question.

Research questions: How is working from home during the COVID-19 pandemic affecting software developers’ emotional wellbeing and productivity?

In addressing this question, this paper generates and evaluates a theoretical model for explaining and predicting changes in wellbeing and productivity while working from home during a crisis. Moreover, we provide recommendations for professionals and organizations to support employees who are working from home due to COVID-19 or future disasters.

This paper is organized as follows. Section 2 provides needed background on pandemics, productivity and working from home. Section 3 introduces our hypotheses and nomological model. Section 4 describes the design of the pandemic programming questionnaire and our sampling strategy. Next, we describe our analysis and results (Section 5), followed by the study’s limitations and implications (Section 6). Section 7 concludes the paper with a summary of its contributions.

2 Background

To fully understand this study, we need to review three areas of related work: (1) pandemics, bioevents and disasters; (2) working from home; (3) productivity.

2.1 Pandemics, bioevents and disasters

Madhav et al. (2017) defines pandemics as “large-scale outbreaks of infectious disease over a wide geographic area that can greatly increase morbidity and mortality and cause significant economic, social, and political disruption” (p. 35). Pandemics can be very stressful not only for those who become infected but also for those caring for the infected and worrying about the health of themselves, their families and their friends (Kim et al., 2015; Prati et al., 2011). In a recent poll, “half of Canadians (50%) report[ed] a worsening of their mental health” during the COVID-19 lockdown (ARI, 2020).

A pandemic can be mitigated in several ways including social distancing (Anderson et al., 2020). “Social distancing refers to a set of practices that aim to reduce disease transmission through physical separation of individuals in community settings” (Rebmann, 2009, p. 120-14). It may include public facility shutdowns, home quarantine, cancelling large public gatherings, working from home or reducing the number of workers in the same place at the same time, maintaining a distance of at least 1.5–2m between people, and other attempts to reduce contact rate (Rebmann, 2009; Anderson et al., 2020).

The extent to which individuals comply with recommendations varies significantly and is affected by many factors. People are more likely to comply when they have more self-efficacy; that is, confidence that they can stay at home or keep working during the pandemic, and when they perceive the risks as high (Teasdale et al., 2012). However, this “threat appraisal” depends on: the psychological process of quantifying risk, sociocultural perspectives (e.g. one’s worldview and beliefs; how worried one’s friends are), “illusiveness of preparedness” (e.g. fatalistic attitudes and denial), beliefs about who is responsible for mitigating risks (e.g. individuals or governments) and how prepared one feels (Yong et al., 2017; Yong and Lemyre, 2019; Prati et al., 2011).

People are less likely to comply when they are facing loss of income, personal logistical problems (e.g. how to get groceries), isolation, and psychological stress (e.g. infection fears, boredom, frustration, stigma) (DiGiovanni et al., 2004). Barriers to following recommendations include job insecurity, lack of childcare, guilt and anxiety about work not being completed, and personal cost of following government advice (Teasdale et al., 2012; Blake et al., 2010).

For employees, experiencing negative life events such as disasters is associated with absenteeism and lower quality of workdays (North et al., 2010). Employers therefore need work-specific strategies and support for their employees. Employers can give employees a sense of security and help them return to work by continuing to pay full salaries on time, reassuring employees they they are not going to lose their job, having flexible work demands, implementing an organized communication strategy, and ensuring access to utilities (e.g. telephone, internet, water, electricity, sanitation) and organisational resources (North et al., 2010; Donnelly and Proctor-Thomson, 2015; Blake et al., 2010).

Work-specific strategies and support are also needed to ensure business continuation and survival. The disruption of activities in disasters simultaneously curtails revenues and reduces productive capacity due to the ambiguity and priorities shifting in individuals, organizations and communities (Donnelly and Proctor-Thomson, 2015). As social distancing closes worksites and reduces commerce, governments face increased economic pressure to end social distancing requirements prematurely (Loose et al., 2010). Maintaining remote workers’ health and productivity is therefore important for maintaining social distancing as long as is necessary (Blake et al., 2010).

2.2 Working from home

Pérez et al. (2004) defines teleworking (also called remote working) as “organisation of work by using information and communication technologies that enable employees and managers to access their labour activities from remote locations, such as home-based teleworking, mobile teleworking, and telecenters or teleworking centers” (p. 280). Telework can help restore and maintain operational capacity and essential services during and after disasters (Blake et al., 2010), especially when workplaces are inaccessible. Indeed, many executives are already planning to shift “at least 5% of previously on-site employees to permanently remote positions post-COVID 19” (Lavelle, 2020).

However, many organisations lack appropriate plans, supportive policies, resources or management practices for practising home-based telework. In disasters such as pandemics where public facilities are closed and people are required to stay at home, their experience and capacity to work can be limited by lack of dedicated workspace at home, caring responsibilities and organisational resources (Donnelly and Proctor-Thomson, 2015).

In general, working from home is often claimed to improve productivity (Davenport and Pearlson, 1998; McInerney, 1999; Cascio, 2000) and teleworkers consistently report increased perceived productivity (Duxbury et al., 1998; Baruch, 2000). Interestingly, Baker et al. (2007) found that organisational and job-related factors (e.g. management culture, human resources support, structure of feedback) are more likely to affect teleworking employees’ satisfaction and perceived productivity than work styles (e.g. planning vs. improvising) and household characteristics (e.g. number of children).

However, individuals’ wellbeing while working remotely is influenced by their emotional stability (that is, a person’s ability to their control emotions when stressed). Working from home gives people with high emotional stability more autonomy and fosters their wellbeing. In contrast, working from home can exacerbate physical, social and psychological strain in employees with low emotional stability (Perry et al., 2018), and the COVID-19 pandemic has not been good for emotional stability (ARI, 2020).

Research on working from home has been criticized for reliance on self-reported perceived productivity, which may inflate the benefits of working from home (Bailey and Kurland, 2002); however, objective measures often lack construct validity (Ralph and Tempero, 2018) and perceived productivity correlates well with managers’ appraisals (Baruch, 1996). (The perceived productivity scale we use below correlates well with objective performance data; cf. Section 4.3).

2.3 Productivity

Productivity is the amount of work done per unit of time.111https://dictionary.cambridge.org/dictionary/english/productivity Measuring time is simple but quantifying the work done by a software developer is not. Some researchers (e.g. Jaspan and Sadowski, 2019) argue for using goal-specific metrics; others reject the whole idea of measuring productivity (e.g. Ko, 2019) not least because people tend to optimize for whatever metric is being used, a phenomenon known as Goodhart’s law (Goodhart, 1984; Chrystal and Mizen, 2003).

Furthermore, simple productivity measures such as counting the number of commits or the number of modified lines of code in a certain period suffer from low construct validity (Ralph and Tempero, 2018). The importance and added value of a commit does not necessarily correlate with its size. Similarly, some developers might prefer very dense one-line solutions, while others like to arrange their contributions in several lines. Comparing the two above-mentioned solutions by counting lines of code must yield biased results. Nevertheless, large companies including Microsoft still use controversial metrics such as number of pull requests as a “proxy for productivity” (Spataro, 2020), and individual developers also use them to monitor their own performance (Baltes and Diehl, 2018). Copious time tracking tools exist for that purpose—some specifically tailored for software developers.222See: https://wakatime.com/

While researchers have adapted existing scales to measure related phenomena like happiness (e.g. Graziotin and Fagerholm, 2019), there is no widespread consensus about how to measure developers’ productivity or the main antecedents thereof. Many researchers use simple, unvalidated productivity scales; for example, Meyer et al. (2017) used a single question asking participants to rate themselves from “not productive” to “very productive.” (The perceived productivity scale we use below has been repeatedly validated in multiple domains; cf. see Section 4.3).

3 Hypotheses

The discussion above suggests numerous hypotheses, as follows. Here we hypothesize about “developers” even though our survey was open to all software professionals because most respondents were developers (see Section 5.3). These hypotheses were generated contemporaneously with questionnaire design—before data collection began.

Hypothesis H1: Developers will have lower wellbeing while working from home due to COVID-19.

Stress, isolation, travel restrictions, business closures and the absence of educational, child care and fitness facilities all take a toll on those working from home. Indeed, a pandemic’s severity and the uncertainty and isolation it induces create frustration, anxiety and fear (Taha et al., 2014; DiGiovanni et al., 2004; Teasdale et al., 2012). It therefore seems likely that many developers will be experiencing reduced emotional wellbeing.

Hypothesis H2: Developers will have lower perceived productivity while working from home due to COVID-19.

Similarly, stress, moving to an impromptu home office, and lack of child care and other amenities may have a negative impact on many developers’ productivity. Many people are likely more distracted by the people they live with and their own worrisome thoughts. People tend to experience lower motivation, productivity and commitment while working from home in a disaster situation (Donnelly and Proctor-Thomson, 2015).


Assuming Hypotheses H1 and H2 are supported, we want to propose a model that explains and predicts changes in wellbeing and productivity (Figure 1). Hypotheses H1 and H2 are encapsulated in the change in wellbeing and change in perceived productivity constructs. The model only makes sense if wellbeing and productivity have changed since developers began working from home.

Figure 1: Theoretical model of developer wellbeing and productivity

Hypothesis H3: Change in wellbeing and change in perceived productivity are directly related.

We expect wellbeing and productivity to exhibit reciprocal causality. That is, as we feel worse, we become less productive, and feeling less productive makes us feel even worse, in a downward spiral. Many studies show that productivity and wellbeign covary (cf. Dall’Ora et al., 2016). Moreover, Evers et al. (2014) found that people with increasing health risks have a lower wellbeing and higher dissatisfaction in life, leading to higher rates of depression and anxiety. On the other hand, decreasing health risk will increase physical and emotional wellbeing and productivity.

Hypotheses H4 and H5: Disaster preparedness is directly related to change in wellbeing and change in perceived productivity.

Disaster preparedness is the degree to which a person is ready for a natural disaster. It includes behaviors like having an emergency supply kit and complying with directions from authorities. We expect lack of preparedness for disasters in general and for COVID-19 in particular to exacerbate reductions in wellbeing and productivity, and vice versa (cf. Paton, 2008; Donnelly and Proctor-Thomson, 2015).

Hypotheses H6 and H7: Fear (of the pandemic) is inversely related to change in wellbeing and change in perceived productivity.

Fear is a common reaction to bioevents like pandemics. Emerging research on COVID-19 is already showing a negative effect on wellbeing, particularly anxiety (Harper et al., 2020; Xiang et al., 2020). Meanwhile, fear of infection and public health measures cause psychosocial distress, increased absenteeism and reduced productivity (Shultz et al., 2016; Thommes et al., 2016).

Hypotheses H8 and H9: Home office ergonomics is directly related to change in wellbeing and change in perceived productivity.

Here we use ergonomics in its broadest sense of the degree to which an environment is safe, comfortable and conducive to the tasks being completed in it. We are not interested in measuring the angle of a developer’s knees and elbows, but in a more general sense of their comfort. Professionals with more ergonomic home offices should have greater wellbeing and be more productive. Donnelly and Proctor-Thomson (2015) found that availability of a dedicated work-space at home, living circumstances, and the availability of organisational resources to work relate to the capacity to return to work after a disaster and employees’ productivity.

Hypothesis H10: Disaster preparedness is inversely related to fear (of the pandemic).

It seems intuitive that the more prepared we are for a disaster, the more resilient and less afraid we will be when the disaster occurs. Indeed, Ronan et al.’s 2015 systematic review found that programs for increasing disaster preparedness had a small- to medium-sized negative effect on fear. People who have high self-efficacy and response-efficacy (i.e. perceive themselves as ready to face a disaster) will be less afraid (Roberto et al., 2009).

4 Method

On March 19, 2020, the first author conceived of a survey to investigate how COVID-19 affects developers, and recruited the second and third authors for help. We created the questionnaire and it was approved by Dalhousie University’s research ethics board in less than 24 hours. We began data collection on March 27th. We then recruited authors 5 through 17, who translated and localized the questionnaire into Arabic, (Mandarin) Chinese, English, French, Italian, Japanese, Korean, Persian, Portuguese, Spanish, Russian and Turkish, and created region-specific advertising strategies. Translations launched between April 5 and 7, and we completed data collection on April 14. Next, we recruited the fourth author to assist with the data analysis, which was completed on April 29th. The manuscript was prepared primarily by the first four authors with edits from the rest of team.

This section details our approach and instrumentation.

4.1 Replication pack

A comprehensive replication package including our (anonymous) dataset, instruments and analysis scripts is stored in the Zonodo open data archive.333https://zenodo.org/record/3780082

4.2 Population and inclusion criteria

This study’s target population is software developers anywhere in the world who switched from working in an office to working from home because of COVID-19. Of course, developers who had been working remotely before the pandemic and developers who continued working in offices throughout the pandemic are also important, but this study is about the switch, and the questions are designed for people who switched from on-site to at-home work.

In principle, the questionnaire was open to all sorts of software professionals, including designers, quality assurance specialists, product managers, architects and business analysis, but we are mainly interested in developers, our marketing focuses on software developers, and therefore most respondents identify as developers (see Section 5.3).

4.3 Instrument design

We created an anonymous questionnaire survey. We did not use URL tracking or tokens. We did not collect contact information.

Questions were organized into blocks corresponding to scale or question type. The order of the items in each multi-item scale was randomized to mitigate primacy and recency effects. The order of blocks was not randomized because our pilot study (Section 4.4) suggested that asking the questionnaire was more clear when the questions that distinguish between before and after the switch to home working came after those that did not.

The questionnaire used a filter question to exclude respondents who do not meet the inclusion criteria. Respondents who had not switched from working in an office to working from home because of COVID-19 simply skipped to the end of the questionnaire. It also included not only traditional demographic variables (e.g. age, gender, country, experience, education) but also how many adults and children (under twelve) participants lived with, the extent to which participants are staying home and whether they or any friends or family had tested positive for COVID-19.

The questionnaire used validated scales as much as possible to improve construct validity. A construct is a quantity that cannot be measured directly. Fear, disaster preparedness, home office ergonomics, wellbeing and productivity are all constructs. In contrast, age, country, and number of children are all directly measurable. Direct measurements are assumed to have inherent validity, but latent variables have to be validated to ensure that they measure the right properties(cf. Ralph and Tempero, 2018).

The exact question wording can be seen in our replication pack. This section describes the scales and additional questions.

Emotional wellbeing (WHO-5).

To assess emotional wellbeing, we used the WHO’s five-item wellbeing index (WHO-5).444https://www.psykiatri-regionh.dk/who-5/Documents/WHO5_English.pdf Each item is assessed on a six-point scale from “at no time” (0) to “all of the time” (5). The scale can be calculated by summing the items or using factor analysis. The WHO-5 scale is widely used, widely applicable, and has high sensitivity and construct validity (Topp et al., 2015). Respondents self-assessed their wellbeing twice: once for the four weeks prior to beginning to work from home, and then again for the time they have been working from home.

Perceived Productivity (HPQ).

To assess perceived productivity we used items from the WHO’s Health at Work Performance Questionnaire (HPQ).555https://www.hcp.med.harvard.edu/hpq/info.php The HPQ measures perceived productivity in two ways: (1) using an eight-item summative scale, with multiple reversed indicators, that assesses overall and relative performance; and (2) using 11-point general ratings of participants’ own performance and typical performance of similar workers. These scales are amenable to factor analysis or summation. Of course, people tend to overestimate their performance relative to their peers, but we are comparing participants to their past selves not to each other. HPQ scores are closely related to objective performance data in diverse fields (Kessler et al., 2003). Again, respondents self-assessed their productivity for both the four weeks prior to working from home, and for the time they have been working from home.

Disaster Preparedness (DP).

To assess disaster preparedness, we adapted Yong et al.’s (2017) individual disaster preparedness scale. Yong et al. developed their five-item, five-point, Likert scale based on common, important behaviors such as complying with government recommendations and having emergency supplies. The scale was validated using a questionnaire survey of a “weighted nationally representative sample” of 1084 Canadians. We adapted the items to refer specifically to COVID-19. It can be computed by summing the responses or using factor analysis.

Fear and Resilience (FR).

The Bracha-Burkle Fear and Resilience (FR) checklist is a triage tool for assessing a patient’s reaction to a bioevent (e.g. infectious disease pandemics, bioterrorism). The FR checklist places the patient on a scale from intense fear to hyper-resilience (Bracha and Burkle, 2006). We dropped some of the more extreme items (e.g. “Right now are you experiencing shortness of breath?”) because respondents are at home taking a survey, not arriving in a hospital emergency room. The FR checklist is a weighted summative scale so it has to be computed manually using Bracha and Burkle’s formula rather than using factor analysis. It has multiple reversed indicators.

Ergonomics.

We could not find a reasonable scale for evaluating home office ergonomics. There is comparatively less research on the ergonomics of home offices (Inalhan and Ng, 2010) and ergonomic instruments tend to be too narrow (e.g. evaluating hip angle). Based on our reading of the ergonomics literature, we made a simple six-item, six-point Likert scale concerning distractions, noise, lighting, temperature, chair comfort and overall ergonomics.

Again, we evaluated the scale’s face and content validity using a pilot study (see Section 4.4) and statistically evaluated convergent and discriminant validity ex post in Section 5.2.

Organizational Support (OS).

This study seeks to produce actionable advice for software companies regarding how to support their employees. Consequently, we need to elicit respondent’s beliefs about helpful actions. We could not find any appropriate instrument, so the first author interviewed three experienced developers with experience in both co-located and distributed teams as well as office work and working from home. Interviewees brainstormed actions companies could take to help, and we used open-coding (Saldaña, 2015) to organize their responses into five themes:

  1. Equipment: providing equipment employees need in their home office (e.g. a second monitor)

  2. Reassurance: adopting a tone that removes doubt and fear (e.g. assuring employees that lower productivity would be understood)

  3. Connectedness: encouraging virtual socializing (e.g. through video chat)

  4. Self-care: providing personal services not directly related to work (e.g. resources for exercising or home-schooling children)

  5. Technical infrastructure and practices: ensuring that remote infrastructure (e.g. VPNs) and practices (e.g. code review) are in place.

We generated a list of 22 actions (four or five per theme) by synthesizing the ideas of interviewees with existing literature on working from home, distributed development and software engineering more generally. For each action, respondents indicate whether their employer is taking the action and whether they think it is or would be helpful. Organizational support is not a construct in our theory per se because we have insufficient a priori information to produce a quantitative estimate, so we analyze these answers separately.

4.4 Pilot

We solicited feedback from twelve colleagues: six software engineering academics and six experienced software developers. Pilot participants made various comments on the questionnaire structure, directions and on the face and content validity of the scales. Based on this feedback we made numerous changes including clarifying directions, making the question order static, moving the WHO-5 and HPQ scales closer to the end, dropping some problematic questions, splitting up an overloaded question, and adding some open response questions. (Free-text answers are not analyzed in this paper; open response questions were included mainly to inform future research; see Section 6.3).

4.5 Sampling, incentives and localization

We advertised our survey on social and conventional media, including Dev.to, Développez.com, DNU.nl, eksisozluk, Facebook, Hacker News, Heise Online, InfoQ, LinkedIn, Twitter, Reddit and WeChat. Upon completion, participants were provided a link and encouraged to share it with colleagues who might also like to take the survey. Because this is an anonymous survey, we did not ask respondents to provide colleagues’ email addresses or send messages on their behalf.

We did not offer cash incentives for participation. Rather, we offered to donate US$500 to an open source project chosen by participants (in one of the open response questions).

We considered several alternatives, including scraping emails from software repositories and stratified random sampling using company lists, but none of these options seemed likely to produce a more representative sample.

Instead, we focused on increasing the diversity of the sample by localizing the survey and promoting it in more jurisdictions. We translated the survey into Arabic, (Mandarin) Chinese, English, French, Italian, Japanese, Korean, Persian, Portuguese, Spanish, Russian and Turkish. We capitalized on each authors’ local knowledge to reach the more people in their jurisdiction. Rather than a single, global campaign, we used a collection of local campaigns.

Each localization involved small changes in wording. Only a few significant changes were needed:

  • The Chinese version had to be re-implemented in a different questionnaire system (wjx.cn) because Google Forms is not available in China.

  • Because the lockdowns in China and Korea are ending, we reworded some questions from “since you began working from home” to “while you were working from home.”

  • The Portuguese version promised to donate to a specific open-source project in Brazil that is related to COVID-19 (which we have done).

5 Analysis and Results

We received 2668 total responses of which 439 did not meet our inclusion criteria and 4 were effectively blank (see below) leaving 2225 responses that fulfilled our inclusion criteria. This section describes how the data was cleaned and analyzed.

5.1 Data cleaning

The data was cleaned as follows.

  1. Delete responses that did not meet our inclusion criteria.

  2. Delete almost empty rows, where the respondent apparently answered the filter question correctly, then skipped all other questions.

  3. Delete the timestamp field (to preserve anonymity), the consent form confirmation field (because participants could not continue without checking these boxes so the answer is always “TRUE”) and the filter question field (because all remaining rows have the same answer).

  4. Add a binary field indicating whether the respondent had entered text in at least one long-answer question (see Section 5.2)

  5. Remove all free-text responses to a separate file (to preserve anonymity).

  6. Recode the raw data (which is in different languages with different alphabets) into a common quantitative coding scheme; for example, from 1 for “strongly disagree” to 5 for “strongly agree” The recoding instructions and related scripts are included in our replication package.

  7. Split select-multiple questions into one binary variable per checkbox. (Google forms inanely saves this data as a comma-separated list of the text of selected answers).

  8. Add a field indicating the language of the response.

  9. Combine the responses into a single data file.

  10. Calculate the FR scale according to its formula (Bracha and Burkle, 2006).

5.2 Validity analysis

We evaluated construct validity using established guidelines (Ralph and Tempero, 2018). First, we assessed content validity using a pilot study (Section 4.4). Next, we assessed convergent and discriminant validity using a principle component analysis (PCA) with Varimax rotation and Kaiser normalization. Bartlett’s test is significant () and our KMO measure of sampling adequacy is high (0.874), indicating that our data is appropriate for factor analysis.

Variable Component
1 2 3 4
Productivity 8 0.740
Productivity 2 0.715
Productivity 9 0.704 0.304
Productivity 6 0.699
Productivity 4 0.669
Productivity 3 0.645
Productivity 5 0.64
Productivity 1 0.563
Productivity 7 0.356
Wellbeing 1 0.838
Wellbeing 2 0.791
Wellbeing 3 0.782
Wellbeing 5 0.734
Wellbeing 4 0.727
Ergonomics 6 0.802
Ergonomics 5 0.748
Ergonomics 2 0.666
Ergonomics 3 0.645
Ergonomics 1 0.306 0.640
Ergonomics 4 0.628
Disaster Preparedness 3 0.688
Disaster Preparedness 1 0.661
Disaster Preparedness 5 0.568
Disaster Preparedness 2 0.565
Disaster Preparedness 4 0.493

*Rotation converged in 5 iterations. Coefficients suppressed.

Table 1: First Principle Components Analysis*

As Table 1 shows, the items load well but not perfectly. The bolded coefficients suggest possible issues with Change in productivity ( Productivity) 7 and 9, as well as Ergonomics 1. We dropped items one at a time until the loadings stabilize, starting with Productivity 7, followed by Productivity 9. As shown in Table 2, dropping these two indicators solved the problem with Ergonomics 1, so the latter is retained.

Variable Component
1 2 3 4
productivity 2 0.721
productivity 8 0.718
productivity 6 0.703
productivity 4 0.679
productivity 3 0.651
productivity 5 0.649
productivity 1 0.566
wellbeing 1 0.845
wellbeing 2 0.797
wellbeing 3 0.790
wellbeing 5 0.740
wellbeing 4 0.732
ergonomics6 0.803
ergonomics5 0.745
ergonomics2 0.669
ergonomics1 0.646
ergonomics3 0.644
ergonomics4 0.629
disasterPreparedness3 0.685
disasterPreparedness1 0.666
disasterPreparedness2 0.570
disasterPreparedness5 0.565
disasterPreparedness4 0.490

Notes: Rotation convergonomicsed in 5 iterations; correlations suppressed.

Table 2: Second Principle Components Analysis

We evaluate predictive validity by testing our hypotheses in Section 5.4.

Response bias.

There are two basic ways to analyze response bias. The first—comparing sample parameters to known population parameters—is impractical because no one has ever established population parameters for software professionals. The second—comparing late respondents to early respondents—cannot be used directly here because we do not know the time between each respondent learning of the survey and completing it. However, we can do something similar: we can compare respondents who answered one or more open response questions (more keen on the survey) with those who skipped the open response questions (less keen on the survey).

As shown in Table 3, only number of adult cohabitants and age are significant, and in both cases, the effect size () is very small. This is consistent with minimal response bias.

Variable F Sig.
age 4.250 0.039 0.002
disability 0.117 0.733 0.000
education 0.153 0.696 0.000
adult cohabitants 19.037 0.000 0.009
child cohabitants 0.358 0.550 0.000
experience 3.381 0.066 0.002
remote experience 0.013 0.910 0.000
organization size 0.330 0.566 0.000
Table 3: Analysis of Response Bias (One-way ANOVA)

5.3 Demographics

Figure 2: Organization sizes
Figure 3: Participants’ education levels

Respondents were disproportionately male (81% vs. 18% female and 1% non-binary) and overwhelmingly employed full-time (94%) with a median age of 30–34. Participants were generally well-educated (Fig. 3). Most respondents, 53%, live with one other adult, while 18% live with no other adults and the rest live with two or more people. 27% live with one or more children under 12. 8% indicate that they may have a disability that affects their work. Mean work experience is 9.3 years (). Mean experience working from home is 1.3 years (); however, 58% of respondents have no experience working from home.

Participants hail from 53 countries (Table 4) and organizations ranging from 0–9 employees to more than 100,000 (Fig 2). Many participants listed multiple roles but 80% included software developer or equivalent among them, while the rest were other kinds of software professionals (e.g. project manager, quality assurance analyst).

Seven participants () tested positive for COVID-19 and six more () live with someone with COVID-19; 4% of respondents indicated that a close friend or family member had tested positive, and 13% were currently or recently quarantined.

Country n % Country n %
Germany 505 22.7% Japan 53 2.4%
Russia 366 16.4% Spain 52 2.3%
Brazil 272 12.2% Iran 40 1.8%
Italy 173 7.8% Austria 29 1.3%
United States 99 4.4% Canada 27 1.2%
South Korea 81 3.6% Switzerland 20 0.9%
Belgium 77 3.5% United Kingdom 20 0.9%
China 76 3.4% n/a 20 0.9%
Turkey 66 3.0% Other 194 8.7%
India 55 2.5%
Table 4: Respondents’ countries of residence

5.4 Change in wellbeing and productivity

Hypothesis H1: Developers will have lower wellbeing while working from home due to COVID-19.

The WHO5 wellbeing scale is a five-item summative scale. Participants answered the scale twice—once referring to the 28-day period before switching to work from home and once referring to the period while working from home. To assess the effect of switching to work from home, we sum each scale and compare their distributions. Since the summed scales deviate significantly from a normal distribution (K-S test;

), we compare the distributions using the Wilcoxon signed rank test.

Hypothesis H1 is supported (Wilcoxon signed rank test ). Effect size for the Wilcoxon signed rank test is calculated as where is the total number of observations over both data points, which gives an effect size of .

Hypothesis H2: Developers will have lower perceived productivity while working from home due to COVID-19.

Like the wellbeing scale, participants answered the HPQ productivity scale twice. We sum the scales (omitting items 7 and 9; see Section 5.2) and compare their distributions. Again, the summed scales deviate significantly from a normal distribution (K-S test; ), so we use the Wilcoxon signed rank test.

Hypothesis H2 is supported (Wilcoxon signed rank test ; effect size).

5.5 Structural equation model

To test our remaining hypotheses, we use structural equation modeling (SEM). Briefly, SEM is used to test theories involving constructs (also called latent variables). A construct is a quantity that cannot be measured directly (Ralph and Tempero, 2018). Fear, disaster preparedness, home office ergonomics, wellbeing and productivity are all constructs. In contrast, age, country, and number of children are all directly measurable.

To design a structural equation model, we first define a measurement model, which maps each reflective indicator

into its corresponding construct. For example, each of the five items comprising the WHO5 wellbeing scale is modeled as a reflective indicator of wellbeing. SEM uses confirmatory factor analysis to estimate each construct as the shared variance of its respective indicators.

Next, we define the structural model, which identifies the expected relationships among the constructs. The constructs we are attempting to predict are referred to as endogenous, while the predictors are exogenous.

SEM uses a path modeling technique (e.g. regression) to build a model that predicts the endogenous (latent) variables based on the exogenous variables, and to estimate both the strength of each relationship and the overall accuracy of the model.666Data was analyzed using the R package lavaan 0.6-5. available at http://lavaan.ugent.be/.

As mentioned, the first step in a SEM analysis is to conduct a confirmatory factor analysis to verify that the measurement model is consistent (Table 5). Here, the latent concepts Ergonomics and DisasterPreparedness are captured by their respective exogenous variables. Fear is not included because it is computed manually (see Section 4.3). is the difference in a participant’s emotional wellbeing before and after the corona outbreak. This latent concept is captured by five exogenous variables, . Similarly, Productivity represents the difference in perceived productivity, before and after the corona outbreak.

Construct Indicator Estimate Std.Err z-value
WB1 1.000
WB2 0.896 0.016 54.518 0
WB3 0.955 0.016 58.917 0
WB4 0.804 0.018 44.686 0
WB5 0.848 0.017 51.041 0
P1 1.000
P2 -1.268 0.053 -24.084 0
P3 -1.120 0.053 -20.979 0
P4 -1.239 0.053 -23.263 0
P5 -1.229 0.055 -22.266 0
P6 -1.306 0.058 -22.677 0
P8 1.460 0.057 25.512 0
Erg1 1.000
Erg2 0.964 0.035 27.395 0
Erg3 0.820 0.037 22.128 0
Erg4 0.937 0.035 26.663 0
Erg5 1.064 0.034 31.535 0
Erg6 1.258 0.035 35.821 0
Disaster DP1 1.000
DP2 0.716 0.089 8.079 0
DP3 1.181 0.112 10.521 0
DP4 0.923 0.105 8.805 0
DP5 1.186 0.120 9.888 0

Notes: converged after 50 iterations with 185 free parameters with (

); estimates may exceed 1.0 because they are regression coefficients, not correlations as in Principal Component Analysis; negative estimates indicate reversed items

Table 5: Confirmatory Factor Analysis

The confirmatory factor analysis converged (not converging would suggest a problem with the measurement model) and all of the indicators load well on their constructs. The lowest estimate, 0.716 for DP2, is still quite good. The estimates for DeltaP2 through 6 are negative because these items were reversed (i.e. higher score = worse productivity). Note that factor loadings greater than one do not indicate a problem because they are regression coefficients, not correlations (Jöreskog, 1999).

Having reached confidence in our measurement model, we construct our structural model by representing all of the hypotheses stated in Section 3 as regressions (e.g. Wellbeing DisasterPreparedness + Fear + Ergonomics).

In principle, we use all control variables as predictors for all latent variables. In practice, however, this leads to too many relationships and prevents the model from converging. Therefore, we evaluate the predictive power of each control variable, one at a time, and include it in a regression only where it makes at least a marginally significant (

) difference. Here, using a higher than normal p-value is more conservative because we are dropping predictors rather than testing hypotheses. Country (of residence) and language (of questionnaire) are not included because SEM does not respond well to nominal categorical variables (see Section

5.6).

Since the exogenous variables are ordinal, the weighted least square mean variance (WLSMV) estimator was used. WLSMV uses diagonally weighted least squares to estimate the model parameters, but it will use the full weight matrix to compute robust standard errors, and a mean- and variance-adjusted test statistic. In short, the WLSMV is a robust estimator which does not assume a normal distribution, and provides the best option for modelling ordinal data in SEM 

(Brown, 2006). We use the default Nonlinear Minimization subject to Box Constraints (NLMINB) optimizer.

For missing data, we use pairwise deletion: we only keep those observations for which both values are observed (this may change from pair to pair). By default, since we are also dealing with categorical exogenous variables, the model is set to be conditional on the exogenous variables.

The model was executed and all diagnostics passed, that is, lavaan ended normally after 97 iterations with 212 free parameters with . We evaluate model fit by inspecting several indicators (cf. Hu and Bentler, 1999, for cut-offs):

  • The ratio of the

    -test statistic and degrees of freedom (

    ) should be greater than two.

  • The Comparative Fit Index () and Tucker-Lewis Index (), which compare the model’s fit against the null model, should be at least 0.95.

  • The Root Mean Square Error of Approximation (, 90% CI [0.048; 0.053]) should be less than 0.06.

  • The Standardized Root Mean Square Residual () should be less than 0.08 (for large sample sizes).

In summary, all diagnostics indicate the model is safe to interpret (i.e. , , , ).

Figure 4 illustrates the supported structural equation model. Numbers are path coefficients, which indicate the relative strength and direction of relationships. Arrows indicate hypothesized causal direction.


Based on this model, Hypotheses H1–H3, H5, H6, H8–H10 are supported; Hypotheses H4 and H7 are not supported. That is, change in wellbeing and change in perceived productivity are directly related; change in perceived productivity depends on home office ergonomics and disaster preparedness; change in wellbeing depends on ergonomics and fear; and disaster preparedness is inversely related to fear.
Figure 4: Supported model of developer wellbeing and productivity

Note: error terms, unsupported hypotheses and control variables are omitted for clarity

5.6 Exploratory findings

Inspecting the detailed SEM results (Table 6) reveals many interesting patterns.

Construct Predictor Estimate Std.Err -value
Disaster adultCohabitants 0.080 0.019 4.234 0.000
Preparedness disability -0.179 0.059 -3.035 0.002
covidStatus 0.073 0.032 2.260 0.024
education -0.050 0.026 -1.882 0.060
Ergonomics children -0.163 0.031 -5.184 0.000
adultCohabitants -0.047 0.019 -2.457 0.014
disability -0.110 0.057 -1.932 0.053
remoteExperience 0.044 0.026 1.709 0.087
Fear isolation 0.502 0.105 4.764 0.000
DisasterPreparedness -0.336 0.106 -3.161 0.002
role -0.356 0.116 -3.056 0.002
covidStatus 0.196 0.075 2.607 0.009
gender 0.273 0.122 2.241 0.025
disability 0.265 0.119 2.227 0.026
education -0.122 0.060 -2.047 0.041
children 0.116 0.063 1.831 0.067
Wellbeing Ergonomics 0.125 0.033 3.813 0.000
covidStatus -0.121 0.040 -3.041 0.002
Fear -0.031 0.012 -2.542 0.011
age 0.097 0.044 2.204 0.028
DisasterPreparedness -0.020 0.049 -0.416 0.678
Productivity Ergonomics 0.242 0.024 10.233 0.000
DisasterPreparedness 0.097 0.035 2.788 0.005
adultCohabitants 0.041 0.015 2.752 0.006
disability 0.124 0.049 2.513 0.012
age 0.070 0.032 2.220 0.026
Fear -0.002 0.009 -0.204 0.838
Wellbeing Performance 0.822 0.045 18.361 0.000

Notes: converged after 97 iterations; Latent variables capitalized (e.g. Fear); direct measurements in camelCase (e.g. age, adultCohabitants)

Table 6: SEM Regressions

Many direct effects are obvious, for example:

  • People who live with small children have significantly less ergonomic home offices. This is not surprising because the ergonomics scale included items related to noise and distractions.

  • Women tend to be more afraid. This is consistent with studies on the SARS epidemic, which found that women tended to perceive the risk as higher (Brug et al., 2004).

  • People with disabilities are less prepared for disasters, have less ergonomic offices and are more afraid.

  • People who live with other adults are more prepared for disasters.

  • People who live alone have more ergonomic home offices.

  • People who have COVID-19 or have family members, housemates or close friends with COVID-19 tend to be more afraid, more prepared, and have worse wellbeing since working from home.

  • People who are more isolated (i.e. not leaving home at all, or only for necessities) tend to be more afraid.

It is more difficult to interpret indirect effects. For example, changes in productivity and wellbeing are closely related. Hypothesis H4 may be unsupported because change in productivity may be mediating the effect of disaster preparedness on change in wellbeing. Similarly, Hypothesis H7 may not be unsupported because change in wellbeing is mediating the relationship between fear and change in productivity. Furthermore, control variables including gender, children and disability may have significant effects on wellbeing or productivity that are not obvious because they are mediated by another construct. Moreover, some variables have conflicting effects. For example, disability has not only a direct positive effect on productivity but also an indirect negative effect (through fear). So, is the pandemic hitting people with disabilities harder? More research is needed to explore these relationships.

Above, we mentioned omitting language and country because SEM does not respond well to nominal categorical variables. We tried anyway, and both language and country were significant predictors for all latent variables, but, as expected, including so many binary dummy variables makes the model impossible to interpret. While our analysis suggests that country, language (and probably culture) have significant effects on disaster preparedness, ergonomics, fear, wellbeing and productivity, more research is need to understand the nature of these effects (see Section

6.3).

5.7 Organizational support

Table 7 shows participants’ opinions of the helpfulness of numerous ways their organizations could support them. Several interesting patterns stand out from this data:

  • Only action #1—paying developer’s home internet charges—is perceived as helpful by more than half of participants and less than 10% of companies appear to be doing that.

  • The action most companies are taking (#20, having regular meetings) is not perceived as helpful by most participants.

  • There appears to be no correlation between things developers believe would help and things employers are actually doing.

  • There is little consensus among developers about what their organizations should do to help them.

# Action Helpful Following
1 My organization will pay for some or all of my internet charges 51.9% 9.8%
2 My organization will buy new equipment we need to work from home 49.2% 30.9%
3 My organization is encouraging staff to use this time for professional training 47.7% 24.3%
4 My organization has reassured me that they understand if my work performance suffers 47.4% 40.5%
5 My organization is providing activities to occupy staff member’s children 46.4% 7.2%
6 My organization is sending food to staff working from home 44.5% 4.0%
7 My organization is providing at-home exercise programs 41.4% 15.8%
8 My organization has reassured me that I will keep my job 40.2% 62.4%
9 My organization has reassured me that I can take time off if I’m sick or need to care for dependents 40.1% 65.5%
10 My organization is improving documentation of its processes (e.g. how code changes are approved) 37.4% 34.7%
11 My organization will pay for software we need to work from home 36.8% 54.7%
12 My team is peer reviewing commits, change requests or pull requests (peer code review) 36.5% 63.1%
13 I can (or could) take equipment (e.g. monitors) home from my workplace 36.0% 73.9%
14 My organization has reassured me that I will continue to be paid 34.7% 75.2%
15 My team uses a build system to automate compilation and testing 34.3% 62.9%
16 Someone is keeping high priority work ready and our backlog organized 33.1% 60.0%
17 My team has good work-from-home infrastructure (e.g. source control, VPN, remote desktop, file sharing) 32.6% 86.4%
18 My team is having virtual social events (e.g. via video chat) 32.1% 56.1%
19 My organization is encouraging staff to touch base regularly with each other 30.8% 62.4%
20 My team is continuing to have regular meetings (e.g. via video chat) 28.5% 88.9%
21 My team is avoiding synchronous communication (e.g. video chat) 25.5% 14.3%
22 For most of the day, I work with an open video or audio call to some or all of my team 23.3% 26.7%

*number of respondents who indicated that this practice is or would be helpful and number of respondents who indicated that their organizations are following this recommendation (n=2225)

Table 7: Organizational support actions in order of perceived helpfulness*

5.8 Summary interpretation

This study shows that software professionals who are working from home during the pandemic are experiencing diminished emotional wellbeing and productivity, which are closely related. Furthermore, poor disaster preparedness, fear related to the pandemic, and poor home office ergonomics are exacerbating this reduction in wellbeing and productivity. Moreover, women, parents and people with disabilities may be disproportionately affected. In addition, dissensus regarding what organizations can do to help suggests that no single action is universally helpful; rather, different people need different kinds of support.

6 Discussion

6.1 Recommendations

Organizations need to accept that expecting normal productivity under these circumstances is unrealistic. Pressuring employees to maintain normal productivity will likely make matters worse. Furthermore, companies should avoid making any decisions (e.g. layoffs, promotions, bonuses) based on productivity during the pandemic because any such decision may be prejudiced against protected groups.

The best way to improve productivity is to help employees maintain their emotional wellbeing. However, no single action appears beneficial to everyone, so organizations should talk to their employees to determine what they need.

Helping employees improve the ergonomics of their work spaces, in particular, should help. However, micromanaging foot positions, armrests and hip angles is not what we mean by ergonomics. Rather, companies should ask broad questions such as “what do you need to limit distractions and be more comfortable?” Shipping an employee a new office chair or noise cancelling headphones could help significantly.

Meanwhile, professionals should try to accept that their productivity may be lower and stop stressing about it. Similarly, professionals should try to remember that different people are experiencing the pandemic in very different ways—some people may be more productive than normal while others struggle to complete any work through no fault of their own. It is crucial to support each other and avoid inciting conflict over who is working harder. If a member of a protected group feels discriminated against due to low productivity at this time, we recommend contacting your local human rights commission or equivalent organization.

6.2 Limitations and threats to validity

The above recommendations should be considered in the context of the study’s limitations.

Sampling bias.

Random sampling of software developers is rare (Amir and Ralph, 2018) because there are no lists of all the software developers, projects, teams or organizations in the world or particular jurisdictions (Baltes and Ralph, 2020). We therefore combined convenience and snowball sampling with a strategy of finding a co-author with local knowledge to translate, localize and advertise the questionnaire in a locally effective way. On one hand, the convenience/snowball strategy may bias the sample in unknown ways. On the other hand, our translation and localization strategy demonstrably increased sample diversity, leading to one of the largest and broadest samples of developers ever studied, possible due to a large, international and diverse research team. Any random sample of English-speaking developers is comparatively ethnocentric.

Response Bias.

Meanwhile, we found minimal evidence of response bias (in Section 5.2

), however, because the questionnaire is anonymous and Google Forms does not record incomplete responses, response bias can only be estimated in a limited way. Someone could have taken the survey more than once or entered fake data. Moreover, some internet service providers in Iran block Google services, but developers tend to use proxies to bypass these restrictions. Additionally, large responses from within a single country could skew the data but we correct for country, company size and language to mitigate this.

Construct validity.

To enhance construct validity, we used validated scales for wellbeing, productivity, disaster preparedness and fear/resilience. Post-hoc construct validity analysis suggests that all four scales, as well as the ergonomics scale we created, are sound (Section 5.2). However, perceived productivity is not the same as actual productivity. Although the HPQ scale correlates well with objective performance data in other fields (Kessler et al., 2003), it may not in software development. Similarly, we asked respondents their opinion of numerous potential mechanisms for organizational support. Just because companies are taking some action (e.g. having regular meetings) or respondents believe in the helpfulness of some action (e.g. paying their internet bills), does not mean that those actions will lead to measurable improvements in productivity or wellbeing.

Measurement Validity.

There is much debate about whether 5-point responses such as used in the WHO5 scale should be treated as ordinal or interval. CFA and SEM are often used with these kinds of variables in social sciences despite assuming at least interval data. Some evidence suggests that CFA is robust against moderate deviations from normality, including arguably-ordinal questionnaire items (Flora and Curran, 2004, cf.). We tend not to worry about treating data as interval as long as, in principle, the data is drawn from a continuous distribution. Additionally, due to a manual error, the Italian version was missing organizational support item 11: “My team uses a build system to automate compilation and testing.” The survey may therefore under-count the frequency and importance of this item by up to 10%.

Conclusion validity.

We use structural equation modeling to fit a theoretical model to the data. Indicators of model fit suggest that the model is sound. Moreover, SEM enhances conclusion validity by correcting for multiple comparisons, testing the entire model as a whole (instead of one hypothesis at a time) and measurement error (by inferring latent variables based on observable variables). SEM is considered superior to alternative path modeling techniques such as partial least squares path modeling (Rönkkö and Evermann, 2013). While a Bayesian approach might have higher conclusion validity (Furia et al., 2019), none of the Bayesian SEM tools (e.g. Blaavan) we are aware of support ordered categorical variables. The main remaining threat to conclusion validity is that the structural model is overfit to the data. More research is needed to determine whether the model overstates any of the supported effects.

Internal validity.

To infer causality, we must demonstrate correlation, precedence and the absence of third variable explanations. SEM demonstrates correlation. Inferring causal relationships from cross-sectional data is fraught because we cannot tell the direction of causality or control for all possible confounds. However, many of our propositions only make sense in one direction. For example, having COVID-19 may reduce one’s productivity, but feeling unproductive cannot give someone a specific virus. Other relationships make a lot more sense in one direction than the other. For example, having a more ergonomic office might make you more productive, but being more productive does not make your office more ergonomic. We can be more confident in causality where precedence only makes sense in one direction. That said, while we include numerous control variables (e.g. age, gender, education level), other third variable explanations cannot be discounted. Developers who work more overtime, for example, might have lower wellbeing, worse home office ergonomics, and reduced disaster preparedness.

6.3 Implications for researchers and future work

For researchers, this paper opens a new research area intersecting software engineering and crisis, disaster and emergency management. Although many studies explore remote work and distributed teams, we still need a better understanding of how stress, distraction and family commitments affect developers working from home during crises, bioevents and disasters. More research is needed on how these events affect team dynamics, cohesion and performance.

More specifically, the dataset we publish alongside this paper can be significantly extended. An enormous amount of quantitative data is available regarding different countries, and how those countries reacted to the COVID-19 pandemic. Country data could be merged with our dataset to investigate how different contexts, cultures and political actions affect developers.

Moreover, the crisis continues. More longitudinal research is needed to understand its long-term effects on software professionals, projects and communities.

6.4 Lessons learned

This study taught us two valuable lessons about research methodology. First, collaborating with a large, diverse, international research team and releasing a questionnaire in multiple languages with location-specific advertising can generate a large, diverse, international sample of participants.

Second, Google Forms should not be used to conduct scientific questionnaire surveys. It is blocked in some countries. It does not record partial responses or bounce rates, hindering analysis of response bias. URL parameter passing, which is typically used to determine how the respondent found out about the survey, is difficult. Exporting the data in different ways gives different variable orders, encouraging mistakes. Responses are recorded as (sometimes long) strings instead of numbers, overcomplicating data analysis. We should have used a research focused survey tool such as Qualtrics777qualtrics.com.

7 Conclusion

The COVID-19 pandemic has created unique conditions for many software developers. Stress, isolation, travel restrictions, business closures and the absence of educational, child care and fitness facilities are all taking a toll. Working from home under these conditions is fundamentally different from normal working from home. This paper reports the first large-scale study of how working from home during a pandemic affects software developers. It makes several key contributions:

  • Evidence that productivity and wellbeing are suffering;

  • Evidence that productivity and wellbeing are closely related;

  • A model that explains and predicts the effects of the pandemic on productivity and wellbeing;

  • A ranked list of suggestions for supporting developers working from home;

  • Some indication that the pandemic may disproportionately affect women, parents and people with disabilities.

Furthermore, this study is exceptional in several ways: (1) the questionnaire used previously validated scales, which we re-validated using both principal components analysis and confirmatory factor analysis; (2) the questionnaire attracted an unusually large sample of 2225 responses; (3) the questionnaire ran in 12 languages, mitigating cultural biases; (4) the data was analyzed using highly sophisticated methods (i.e. structural equation modelling), which rarely have been utilized in software engineering research; (5) the study investigates an emerging phenomenon, providing timely advice for organizations and professionals; (6) the study incorporates research on emergency and disaster management, which is rarely considered in software engineering studies.

We hope that this study inspires more research on how software development is affected by crises, pandemics, lockdowns and other adverse conditions. As the climate crisis unfolds, research intersecting crisis management and software engineering will be increasingly needed.

Acknowledgements.
This project was supported by Dalhousie University. Thanks to Brett Cannon, Alexander Serebrenik, Klaas Stol and all of our pilot participants for their support. Thanks also to all of media outlets who provided complementary advertising, including DNU.nl, eksisozluk, InfoQ and Heise Online. Finaly, thanks to everyone at Empirical Software Engineering for fast-tracking COVID-related research.

References

  • B. Amir and P. Ralph (2018) There is no random sampling in software engineering research. In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings, pp. 344–345. Cited by: §6.2.
  • R. M. Anderson, H. Heesterbeek, D. Klinkenberg, and T. D. Hollingsworth (2020) How will country-based mitigation measures influence the course of the covid-19 epidemic?. The Lancet 395 (10228), pp. 931–934. Cited by: §2.1.
  • ARI (2020) Worry, gratitude & boredom: as covid-19 affects mental, financial health, who fares better; who is worse?. Angus Reid Institute. External Links: Link Cited by: §2.1, §2.2.
  • D. E. Bailey and N. B. Kurland (2002) A review of telework research: findings, new directions, and lessons for the study of modern work. Journal of Organizational Behavior: The International Journal of Industrial, Occupational and Organizational Psychology and Behavior 23 (4), pp. 383–400. Cited by: §2.2.
  • E. Baker, G. C. Avery, and J. Crawford (2007) Satisfaction and perceived productivity when professionals work from home. Research & Practice in Human Resource Management. Cited by: §2.2.
  • S. Baltes and S. Diehl (2018) Towards a theory of software development expertise. In Proceedings of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT FSE 2018, Lake Buena Vista, FL, USA, November 04-09, 2018, G. T. Leavens, A. Garcia, and C. S. Pasareanu (Eds.), pp. 187–200. External Links: Link, Document Cited by: §2.3.
  • S. Baltes and P. Ralph (2020) Sampling in software engineering research: a critical review and guidelines. arXiv 2002.07764. Cited by: §6.2.
  • Y. Baruch (1996) Self performance appraisal vs direct-manager appraisal: a case of congruence. Journal of Managerial Psychology 11 (6), pp. 50–65. External Links: Document Cited by: §2.2.
  • Y. Baruch (2000) Teleworking: benefits and pitfalls as perceived by professionals and managers. New technology, work and employment 15 (1), pp. 34–49. Cited by: §2.2.
  • K. D. Blake, R. J. Blendon, and K. Viswanath (2010) Employment and compliance with pandemic influenza mitigation recommendations. Emerging infectious diseases 16 (2), pp. 212. Cited by: §2.1, §2.1, §2.1, §2.2.
  • H. S. Bracha and F. M. Burkle (2006) Utility of fear severity and individual resilience scoring as a surge capacity, triage management tool during large-scale, bio-event disasters. Prehospital and disaster medicine 21 (5), pp. 290–296. Cited by: §4.3, item 10.
  • T. A. Brown (2006) Confirmatory factor analysis for applied research. Methodology in the social sciences, Guilford Publications. External Links: ISBN 9781593852740 Cited by: §5.5.
  • J. Brug, A. R. Aro, A. Oenema, O. De Zwart, J. H. Richardus, and G. D. Bishop (2004) SARS risk perception, knowledge, precautions, and information sources, the netherlands. Emerging infectious diseases 10 (8), pp. 1486. Cited by: 2nd item.
  • W. F. Cascio (2000) Managing a virtual workplace. Academy of Management Perspectives 14 (3), pp. 81–90. Cited by: §2.2.
  • K. A. Chrystal and P. D. Mizen (2003) Goodhart’s law: Its origins, meaning and implications for monetary policy. Central banking, monetary theory and practice: Essays in honour of Charles Goodhart 1, pp. 221–243. Cited by: §2.3.
  • C. Dall’Ora, J. Ball, A. Recio-Saucedo, and P. Griffiths (2016) Characteristics of shift work and their impact on employee performance and wellbeing: a literature review. International journal of nursing studies 57, pp. 12–27. Cited by: §3.
  • T. H. Davenport and K. Pearlson (1998) Two cheers for the virtual office. MIT Sloan Management Review 39 (4), pp. 51. Cited by: §2.2.
  • C. DiGiovanni, J. Conley, D. Chiu, and J. Zaborski (2004) Factors influencing compliance with quarantine in toronto during the 2003 sars outbreak. Biosecurity and bioterrorism: biodefense strategy, practice, and science 2 (4), pp. 265–272. Cited by: §2.1, §3.
  • N. Donnelly and S. B. Proctor-Thomson (2015) Disrupted work: home-based teleworking (hbtw) in the aftermath of a natural disaster. New Technology, Work and Employment 30 (1), pp. 47–61. Cited by: §1, §2.1, §2.1, §2.2, §3, §3, §3.
  • C. Duffy (2020) Big tech firms ramp up remote working orders to prevent coronavirus spread. CNN Business. External Links: Link Cited by: §1.
  • L. Duxbury, C. Higgins, and D. Neufeld (1998) Telework and the balance between work and family: is telework part of the problem or part of the solution? the virtual workplace. eds. m. igbaria & m. tan. hershey. PA: Idea Group Publishing. Cited by: §2.2.
  • K. E. Evers, P. H. Castle, J. O. Prochaska, and J. M. Prochaska (2014) Examining relationships between multiple health risk behaviors, well-being, and productivity. Psychological reports 114 (3), pp. 843–853. Cited by: §3.
  • D. B. Flora and P. J. Curran (2004) An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data.. Psychological methods 9 (4), pp. 466. Cited by: §6.2.
  • C. A. Furia, R. Feldt, and R. Torkar (2019) Bayesian data analysis in empirical software engineering research. IEEE Transactions on Software Engineering. Cited by: §6.2.
  • C. A. E. Goodhart (1984) Monetary Theory and Practice: The UK Experience. Macmillan Press. Cited by: §2.3.
  • D. Graziotin and F. Fagerholm (2019) Happiness and the productivity of software engineers. In Rethinking Productivity in Software Engineering, C. Sadowski and T. Zimmermann (Eds.), pp. 109–124. External Links: Link, Document Cited by: §2.3.
  • S. Hanselman (2020) Tweet. External Links: Link Cited by: §1.
  • C. Harper, L. Satchell, D. Fido, and R. Latzman (2020) Functional fear predicts public health compliance in the covid-19 pandemic. International Journal of Mental Health and Addiction, pp. 1–21. Cited by: §3.
  • L. Hu and P. M. Bentler (1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6 (1), pp. 1–55. External Links: Document Cited by: §5.5.
  • G. Inalhan and C. F. Ng (2010) Teleworker’s home office: an extension of corporate office?. Facilities 28 (3/4), pp. 137–155. Cited by: §4.3.
  • C. Jaspan and C. Sadowski (2019) No single metric captures productivity. In Rethinking Productivity in Software Engineering, C. Sadowski and T. Zimmermann (Eds.), pp. 13–20. External Links: Link, Document Cited by: §2.3.
  • K. G. Jöreskog (1999) How large can a standardized coefficient be. External Links: Link Cited by: §5.5.
  • J. Kaplan, L. Frias, and M. McFall-Johnsen (2020) A third of the global population is on coronavirus lockdown — here’s our constantly updated list of countries and restrictions. Business Insider. External Links: Link Cited by: §1.
  • R. C. Kessler, C. Barber, A. Beck, P. Berglund, P. D. Cleary, D. McKenas, N. Pronk, G. Simon, P. Stang, T. B. Ustun, et al. (2003) The world health organization health and work performance questionnaire (hpq). Journal of Occupational and Environmental Medicine 45 (2), pp. 156–174. Cited by: §4.3, §6.2.
  • Y. Kim, W. Zhong, M. Jehn, and L. Walsh (2015) Public risk perceptions and preventive behaviors during the 2009 h1n1 influenza pandemic. Disaster medicine and public health preparedness 9 (2), pp. 145–154. Cited by: §2.1.
  • A. J. Ko (2019) Why we should not measure productivity. In Rethinking Productivity in Software Engineering, C. Sadowski and T. Zimmermann (Eds.), pp. 21–26. External Links: Link, Document Cited by: §2.3.
  • J. Lavelle (2020) Gartner cfo survey reveals 74% intend to shift some employees to remote work permanently. Gartner. Note: https://www.gartner.com/en/newsroom/press-releases/2020-04-03-gartner-cfo-surey-reveals-74-percent-of-organizations-to-shift-some-employees-to-remote-work-permanently2 Cited by: §2.2.
  • V. W. Loose, V. N. Vargas, D. E. Warren, S. J. Starks, T. J. Brown, and B. J. Smith (2010) Economic and policy implications of pandemic influenza. Report SAND2010-1910, Sandia National Laboratories, Albuquerque, New Mexico March. Cited by: §2.1.
  • N. Madhav, B. Oppenheim, M. Gallivan, P. Mulembakani, E. Rubin, and N. Wolfe (2017) Chapter 17. pandemics: risks, impacts, and mitigation. Disease control priorities, 9. Cited by: §2.1.
  • C. R. McInerney (1999) Working in the virtual office: providing information and knowledge to remote workers. Library & information science research 21 (1), pp. 69–89. Cited by: §2.2.
  • A. N. Meyer, G. C. Murphy, T. Zimmermann, and T. Fritz (2017) Retrospecting on work and productivity: A study on self-monitoring software developers’ work. PACMHCI 1 (CSCW), pp. 79:1–79:24. External Links: Link, Document Cited by: §2.3.
  • C. S. North, B. Pfefferbaum, B. A. Hong, M. R. Gordon, Y. Kim, L. Lind, and D. E. Pollio (2010) The business of healing: focus group discussions of readjustment to the post-9/11 work environment among employees of affected agencies. Journal of occupational and environmental medicine 52 (7), pp. 713–718. Cited by: §2.1.
  • D. Paton (2008) Community resilience: integrating individual, community and societal perspectives. In The phoenix of natural disasters: Community resilience, K. Gow and D. Paton (Eds.), pp. 13–31. Cited by: §3.
  • M. P. Pérez, A. M. Sánchez, P. de Luis Carnicer, and M. J. V. Jiménez (2004) A technology acceptance model of innovation adoption: the case of teleworking. European Journal of innovation management. Cited by: §2.2.
  • S. J. Perry, C. Rubino, and E. M. Hunter (2018) Stress in remote work: two studies testing the demand-control-person model. European Journal of Work and Organizational Psychology 27 (5), pp. 577–593. Cited by: §2.2.
  • G. Prati, L. Pietrantoni, and B. Zani (2011) A social-cognitive model of pandemic influenza h1n1 risk perception and recommended behaviors in italy. Risk Analysis: An International Journal 31 (4), pp. 645–656. Cited by: §2.1, §2.1.
  • P. Ralph and E. Tempero (2018) Construct validity in software engineering research and software metrics. In Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018, pp. 13–23. Cited by: §2.2, §2.3, §4.3, §5.2, §5.5.
  • T. Rebmann (2009) Infectious disease disasters: bioterrorism, emerging infections, and pandemics. In APIC text of infection control and epidemiology, pp. 120.1–120.22. Cited by: §2.1.
  • A. J. Roberto, C. E. Goodall, and K. Witte (2009) Raising the alarm and calming fears: perceived threat and efficacy during risk and crisis. In Handbook of risk and crisis communication, pp. 287–303. Cited by: §3.
  • K. R. Ronan, E. Alisic, B. Towers, V. A. Johnson, and D. M. Johnston (2015) Disaster preparedness for children and families: a critical review. Current psychiatry reports 17 (7), pp. 58. Cited by: §3.
  • M. Rönkkö and J. Evermann (2013) A critical examination of common beliefs about partial least squares path modeling. Organizational Research Methods 16 (3), pp. 425–448. Cited by: §6.2.
  • J. Saldaña (2015) The coding manual for qualitative researchers. Sage. Cited by: §4.3.
  • J. M. Shultz, J. L. Cooper, F. Baingana, M. A. Oquendo, Z. Espinel, B. M. Althouse, L. H. Marcelin, S. Towers, M. Espinola, C. B. McCoy, et al. (2016) The role of fear-related behaviors in the 2013–2016 west africa ebola virus disease outbreak. Current psychiatry reports 18 (11), pp. 104. Cited by: §3.
  • J. Spataro (2020) Helping our developers stay productive while working remotely. Microsoft. External Links: Link Cited by: §2.3.
  • S. Taha, K. Matheson, T. Cronin, and H. Anisman (2014) Intolerance of uncertainty, appraisals, coping, and anxiety: the case of the 2009 h 1 n 1 pandemic. British journal of health psychology 19 (3), pp. 592–605. Cited by: §3.
  • E. Teasdale, L. Yardley, W. Schlotz, and S. Michie (2012) The importance of coping appraisal in behavioural responses to pandemic flu. British journal of health psychology 17 (1), pp. 44–59. Cited by: §2.1, §2.1, §3.
  • E. Thommes, M. Cojocaru, and S. Athar (2016) Absenteeism impact on local economy during a pandemic via hybrid sir dynamics. In International Conference on Dynamics of Disasters, pp. 309–328. Cited by: §3.
  • C. W. Topp, S. D. Østergaard, S. Søndergaard, and P. Bech (2015) The who-5 well-being index: a systematic review of the literature. Psychotherapy and psychosomatics 84 (3), pp. 167–176. Cited by: §4.3.
  • WHO (2020) Statement on the second meeting of the international health regulations (2005) emergency committee regarding the outbreak of novel coronavirus (2019-ncov). World Health Organization. External Links: Link Cited by: §1.
  • WHO (2020) Coronavirus disease 2019 (covid-19): situation report 70. World Health Organization. External Links: Link Cited by: §1.
  • Y. Xiang, Y. Zhao, Z. Liu, X. Li, N. Zhao, T. Cheung, and C. H. Ng (2020) The covid-19 outbreak and psychiatric hospitals in china: managing challenges through mental health service reform. International journal of biological sciences 16 (10), pp. 1741. Cited by: §3.
  • A. G. Yong, L. Lemyre, C. Pinsent, and D. Krewski (2017) Risk perception and disaster preparedness in immigrants and canadian-born adults: analysis of a national survey on similarities and differences. Risk analysis 37 (12), pp. 2321–2333. Cited by: §2.1, §4.3.
  • A. G. Yong and L. Lemyre (2019) Getting canadians prepared for natural disasters: a multi-method analysis of risk perception, behaviors, and the social environment. Natural Hazards 98 (1), pp. 319–341. Cited by: §2.1.