Understanding the Motivations, Challenges and Needs of Blockchain Software Developers: A Survey

The blockchain technology has potential applications in various areas such as smart-contracts, Internet of Things (IoT), land registry, supply chain management, storing medical data, and identity management. Although the Github currently hosts more than six thousand active Blockchain software (BCS) projects, few software engineering research has investigated these projects and its' contributors. Although the number of BCS projects is growing rapidly, the motivations, challenges, and needs of BCS developers remain a puzzle. Therefore, the primary objective of this study is to understand the motivations, challenges, and needs of BCS developers and analyze the differences between BCS and non-BCS development. On this goal, we sent an online survey to 1,604 active BCS developers identified via mining the Github repositories of 145 popular BCS projects. The survey received 156 responses that met our criteria for analysis. The results suggest that the majority of the BCS developers are experienced in non-BCS development and are primarily motivated by the ideology of creating a decentralized financial system. Although most of the BCS projects are Open Source Software (OSS) projects by nature, more than 93 different from a non-BCS development. The aspects of BCS development that differ from a non-BCS one are also the primary sources of challenges for them. Current BCS development ecosystem is immature and needs an array of tools to be developed or improved.

READ FULL TEXT VIEW PDF
03/20/2021

Tubu-io Decentralized Application Development Test Workbench

Decentralized services are increasingly being developed and their proper...
05/05/2022

Shortcomings of Class-level Documentation: A Survey

To better understand the shortcomings of class-level documentation, we c...
04/26/2021

Leaving My Fingerprints: Motivations and Challenges of Contributing to OSS for Social Good

When inspiring software developers to contribute to open source software...
03/06/2021

Blueprint: Cyberinfrastructure Center of Excellence

In 2018, NSF funded an effort to pilot a Cyberinfrastructure Center of E...
11/24/2019

A survey of of blockchain-based solutions for Energy Industry

The energy industry needs to shift to a new paradigm from its classical ...
09/16/2021

Towards a Provenance Management System for Astronomical Observatories

We present here a provenance management system adapted to astronomical p...
02/19/2021

Exploring Factors and Measures to Select Open Source Software

[Context] Open Source Software (OSS) is nowadays used and integrated in ...

1 Introduction

In 2008, a person or entity under the pseudonym Satoshi Nakamoto, published a whitepaper to introduce ‘Bitcoin,’ a digital currency (aka cryptocurrency) based on an immutable and decentralized public ledger known as blockchain nakamoto2008bitcoin . Currently, ‘Bitcoin’ is the leading cryptocurrency with a market cap of over 100 billion USD. On the other hand, the blockchain technology that runs Bitcoin could prove to be much more significant swan2015blockchain , as a blockchain can record transactions between two parties efficiently and in a verifiable and permanent way; thus eliminating the need for the third party in the middle. Moreover, the availability of all the transactions ever completed to all nodes makes a blockchain-based system more transparent than centralized solutions. Therefore, apart from its application in cryptocurrency, the blockchain technology has potential applications in various other domains such as smart-contracts delmolino2016step , Internet of Things (IoT) huh2017managing , land registry  underwood2016blockchain , supply chain management korpela2017digital , storing medical data azaria2016medrec , and identity management jacobovitz2016blockchain .

The technological innovation and fundamental changes required in the design, development, and deployment of blockchain have also attracted tremendous interests from the software development community. For example, a recent study partha2018 from March 2018 reported 3,000 Blockchain software (BCS) projects hosted on Github111https://github.com/topics/blockchain. In October 2018, within seven months, that number stands at 6,800 (i.e., more than doubled). Unlike traditional development, blockchain developers need to be cautious about malicious actors, secure an immutable and distributed database, and design efficient and reliable protocols withstanding the scarcity of tools and resources which is unavoidable for a new technology. The unalterable nature of a blockchain makes the recovery of an error prohibitively difficult or practically next to impossible if the vulnerability is detected after the deployment. Although some other large-scale software such as financial applications requires similar robustness, the rapidly changing blockchain ecosystem adds significant new challenges porru2017blockchain . The significant differences between traditional software development (i.e., non-BCS) and blockchain oriented development motivated Destefanis et al.  Destefanis2018BOSE even to propose a new development paradigm named Blockchain-Oriented Software Engineering (BOSE).

Although several characteristics of the BCS technology suggest BCS development to be different from a non-BCS development, very few software engineering (SE) research has focused on the former. Therefore, several questions regarding BCS development still remain a puzzle. For example, i) Who are the BCS developers and what are their motivations behind joining BCS development? ii) Is BCS development indeed different from a non-BCS development? iii) If different, what are the areas that mark those differences? iv) Do current development tools and practices that are tuned for non-BCS development satisfy the needs of BCS developers? v) If not, what are the tools and techniques that BCS developers are in need?

Answering these questions would enable the research community and the providers of technical tools to build necessary support to design, develop, test, and deploy BCS applications. Moreover, developers interested to join BCS projects are likely to have valuable guidance for their preparation. Therefore, this study aims to understand the motivations, challenges, and needs of BCS developers. We also aim to conduct a comparative analysis of BCS with non-BCS development to pinpoint specialized tasks that may benefit the former.

On these goals, we designed and sent an online survey to 1,604 BCS developers gathered via mining the Github repositories of 145 BCS projects. A survey is an ideal instrument for this study as current BCS developers have first-hand experiences of their challenges and needs. The survey received 156 responses from BCS developers that met our criteria for analysis. We adopted a systematic qualitative analysis approach to build a coding scheme for the open-ended responses. Using a qualitative analysis software, multiple coders independently assigned codes to each response and achieved a ‘substantial’ inter-rater reliability Cohen1960 . We reported partial results of the survey in a recent publication partha2018 , which explored only the software development practices (i.e., verification and validation, task assignment, requirement analysis, and communication and collaboration) of BCS projects. On the other hand, this publication focuses on a different set of questions none of which was included in our prior article.

The primary contributions of this study include:

  • A better understanding of BCS developers’ motivations, challenges, and needs;

  • A comparative analysis of BCS with non-BCS development;

  • Potential directions for SE researchers to build supports for BCS development; and

  • A characterization of the contemporary BCS development community.

The remainder of the paper is organized as follows. Section 2 provides a background on blockchain. Section 3 introduces the research questions of this study. Section 4 describes our research methodology. Section 5 describes the demographics of our respondents. Section 6 presents the results of this study. Section 7 discusses the implications of our findings. Section 8 describes the threats to the validity of our results. Finally, Section 9 concludes the paper.

2 Background

Figure 1: Simplified diagram of a blockchain

Blockchain is a decentralized, peer-to-peer, transparent, immutable, and append-only data storage. It keeps a permanent record of writes called transactions. Multiple transactions are grouped in blocks. Each block in a blockchain contains its hash computed using a well-known hashing or proof-of-work jakobsson1999proofs algorithm (e.g., SHA256, ethash, and equihash) and the hash of the previous block called parent block (Figure 1). The first block in a chain is called the genesis block, which does not have any parent. Each block’s hash is calculated based on its data, current timestamp and the hash of its parent block. Any change in a block’s data causes alteration of its hash and invalidates all the subsequent blocks, and the tampering becomes immediately evident to every member node of the chain. Hence, to compromise a blockchain, collusion of the majority of the network is required which is impractical in case of a large blockchain narayanan2016bitcoin . Therefore, blockchain is a chain of blocks where the blocks are irreversible and immutable.

All the nodes in a blockchain network participate simultaneously in finding the next block to write. This process is called mining, where the nodes calculate a hash value by adding a nonce (i.e., a random value) to a list of transactions waiting to be added to the blockchain. To be eligible as the next block, the hash must be smaller than an agreed-upon value (known as difficulty), and the nodes continue calculations using different nonces until they find a nonce that generates a hash satisfying the ‘difficulty.’ The node finding a new block will broadcast it to all other nodes in the network to confirm the correctness of this new block. Once confirmed, the new block is added to the blockchain, and each of the transactions contained in the block is considered verified chuen2015handbook . The finder is usually rewarded with a pre-defined number of tokens, known as block reward. The difficulty of the next block is determined by the network with a pre-defined algorithm.

There is no central control over the operation of a blockchain. The underlying philosophy is that no single participant or group of participants can control the infrastructure and all the participants in the network have an equal role to play. In the absence of a central controller, the transactions are mediated by the member nodes using a consensus protocol, which ensures that all the nodes have an identical copy of the blockchain. A new block is considered verified only after the majority of the member nodes vote it as true and trustworthy using the consensus protocol. A blockchain’s security is based on the assumption that tampering would have to happen across the majority of the nodes (aka 51% attack) of a network in the same way simultaneously. So once a blockchain network achieves critical mass, altering a blockchain posthoc becomes infeasible.

In the context of the blockchain, public key cryptography garfinkel1996public ensures the integrity and authenticity of any message/transaction. Each node owns a pair of asymmetric encryption keys simmons1979symmetric , where the public key is broadcast to all relevant nodes but the private key is kept secret. A sender signs messages with its own private key and a receiver verifies the integrity of the message by decrypting it with the sender’s public key. In cryptocurrency applications, the public key of a user also acts as his/her account address. Therefore, a user must sign outgoing transactions using his/her private key. A miner node would verify an outgoing transaction from an account only when it can authenticate the transaction using the owner’s public key.

One of the recent innovative applications of blockchain is Smart- contracts, which are self-executing contracts with the terms of the agreement between buyer(s) and seller(s) of transactions written using lines of code instead of a legal language. Smart contracts permit trusted transactions and agreements to be carried out among different anonymous parties without the need for a central authority, legal system, or external enforcement mechanism. Since a smart-contract, once deployed, lives on a distributed and decentralized blockchain network, it remains traceable, transparent, and irreversible.

3 Research Questions

This study primarily aims to understand the motivations, challenges, and needs of BCS developers and analyze the differences between BCS and non-BCS development. We aim to achieve this goal based on four specific research questions. We also explore one additional research question to characterize the contemporary BCS development community. Following subsections introduce the five research questions with a brief motivation behind each question.

3.1 Personal Characteristics of the BCS Developers

Characterizing the Open Source Software (OSS) developers has drawn interests from the researchers DAVID2008364 ; linus2005 ; hars2001working ; lakhani2003hackers ; Mockus2002 to understand the distribution of expertise, the strength (and durations) of their attachments to particular projects, and the recruitment and retention of newcomers. However, such a characterization for BCS developers is currently missing. Since understanding the demographics of the BCS developers regarding their age, gender, education, and general software development experience has fundamental importance, our first research question is:

RQ1: Who are contributing to BCS projects?

3.2 Motivations of BCS Developers

Much research has focused on understanding the motivations of OSS developers hars2001working ; hertel2003motivation ; Lee2017 ; roberts2006understanding ; lakhani2003hackers . Results of those studies indicate that most of the OSS developers are motivated by intrinsic factors (e.g., fun, feeling competent, or community recognition), while potential external rewards (e.g., financial gains, career growth) were secondary for the most. Although most of the BCS projects are also OSS projects, large inflows of cash through ICO (Initial Coin Offerings) or token sales ADHAMI2018 , which are very common for BCS projects are rare for non-BCS OSS projects. Many of the early BCS developers / investors have garnered significant financial rewards from the recent boom of the cryptocurrency market. Therefore, it won’t be surprising if external rewards are the primary motives for many of the BCS developers. Our next research question tries to find out whether BCS developers’ motivations are similar to non-BCS OSS developers or they are attracted by potential financial gains. Understanding the motivations of BCS developers is important since it will help to identify prospective joiners, which may form synergies with a BCS community. Hence, our next research question is:

RQ2: What are the primary motivations of BCS developers?

3.3 Differences Between BCS and non-BCS development

Although BCS projects possess many traits of traditional OSS projects, we expect some differences between BCS and non-BCS development due to several characteristics (e.g., the immutability of data, hostile environment, and difficulty of upgrading the software after deployment) that distinguish the BCS domain from the non-BCS one. Since identifications of the differences between BCS and non-BCS development will allow assessing the applicability of various traditional SE tools and techniques for BCS projects, we seek to find out:

RQ3: What are the differences between BCS and non-BCS development?

3.4 Challenges of BCS Development

Most of the innovative applications of the blockchain technology such as smart-contracts and distributed applications (aka dapps) are at nascent stages. The BCS development landscape is also changing at a rapid pace with projects fiercely competing with each other to emerge as the market leader Krafft-2018 . Although the high costs of bugs mandate high reliability from a BCS, developers face tremendous pressures from the investors to release the product. These scenarios coupled with the unique characteristics of the BCS domain pose several challenges to the BCS developers. We believe identifications of the primary challenges of BCS development will provide prospective joiners with guidance for their preparation and encourage research to mitigate those challenges. Hence our next research question is:

RQ4: What are the primary challenges of BCS development?

3.5 Tools that BCS Developers Need

The distributed and hostile execution environment of a BCS software makes many of the non-BCS development and testing tools inadequate for BCS development. The requirement of tools, once clearly understood, will lead to the development of supporting tools to design, develop, test, and deploy BCS applications. Since contemporary BCS developers have the first-hand knowledge of their needs for supporting tools, we inquire:

RQ5: What are the tools that BCS developers currently need?

4 Research Methodology

Since the five research questions of this study are geared towards gathering the opinions of BCS developers, we chose a survey as our research instrument. The remainder of this section describes the survey design, our participant selection criteria, pilot testing, data collection, and qualitative data analysis.

4.1 Survey

Our goal in designing the survey was to keep it as short as possible, while still gathering all of the relevant information. For the current paper, we only consider a subset of the survey questions, while question focusing on software development practices of BCS projects were reported in a recent publication partha2018 . Table 1 lists each survey question included in this paper, the research question that motivated its inclusion, and the answer choices provided. Questions indicated with a ‘D,’ rather than a ‘RQ#’ were included to gather demographics about the respondents. For the questions that were open-ended, there are no specified answer choices. Several of the open-ended and demographics questions were inspired by previous surveys murphy2014cowboys ; hertel2003motivation ; von2012carrots ; hars2001working ; lakhani2003hackers ; west2006challenges .

To compare BCS with non-BCS development, the survey asked the respondents (Q13 in Table 1) to rate their agreements for 16 statements on a 5-point Likert scale from ‘Strongly disagree’ to ‘Strongly agree.’ These statements (‘Statement’ column in Table 3) were adopted from a prior SE survey at Microsoft (referred as ‘MS survey’ hereinafter) to assess the differences between the game and non-game development murphy2014cowboys . While the MS survey had 28 statements, our survey did not include the statements regarding the company or manager as BCS projects are mostly community driven.

# RQ* Question Text Answer Choices
Q1 RQ1 How old are you? [#]
Q2 RQ1 Which gender do you identify yourself with? [Male, Female, Prefer not to disclose]
Q3 RQ1 What is your highest level of education? [High school, Bachelors, Masters, Ph.D.]
Q4 RQ1 How many years of software development experiences do you have? [Less than a year, between one to five years, between six to ten years, more than ten years]
Q5 D How many years have you been developing blockchain software? [Less than a year, between one to two years, between three to five years, more than five years]
Q6 D What is your primary Blockchain software project (i.e. the project that you have spent most of your time) [#]
Q7 D Do you receive any direct compensation (i.e, salary or cryptocurrency) from your primary project? [I get directly paid with a FIAT currency, I get compensated with shares, tokens, or cryptocurrency, I receive both FIAT salary and tokens/shares, No, I do not receive any direct compensation]
Q8 D Approximately, how many pull requests have you submitted to your primary project? [Less than 10, Between 11 to 30, More than 30]
Q9 D Approximately, how many hours on average do you spend per week on your primary project? [Less than 5, between 6 to 10, Between 11 to 20, Between 21 to 35, I work full time]
Q10 RQ2 What are your motivations to contribute to your primary project?
Q11 RQ3 Based on your experiences, what are the primary differences between blockchain and non-blockchain software development ? [#]
Q12 RQ3 The following statements aim to compare blockchain and non-blockchain software development. For each of the following items please rate how you agree or disagree with that statement. [Strongly disagree, Disagree, Neutral, Agree, Strongly Agree]
Total 16 statements ( shown in Table 3)
Q13 RQ4 What are the most challenging aspects of blockchain software development?
Q14 RQ5 Please describe the type of tools that you currently do not have, but if implemented, can greatly help your blockchain software development activities. [#]
*numbers refer to the research question that motivated the inclusion of the survey question
Table 1: Survey Questions

4.2 Participant Selection

To ensure valid results, we only surveyed BCS developers with sufficient experience. We identified 145 BCS projects based on following four criteria:

  • Tagged under at least one of the following six ‘topics’222https://blog.github.com/2017-01-31-introducing-topics/: blockchain, cryptocurrency, altcoin, ethereum, bitcoin, and smart-contracts.

  • ‘Starred’ by at least ten users.

  • Have at least five distinct contributors.

  • A manual verification of the repository confirmed it as a BCS project.

We used Github API333https://developer.github.com/v3/ to identify 1,604 contributors, each of whom had submitted at least five changes to one of those 145 projects. We mine the Git commit logs of the identified 145 projects to gather the email addresses of those 1,604 active contributors. We also got the survey questions, consent form, participant selection strategy, solicitation email, and data management reviewed and approved by our university’s Institutional Review Board (IRB).

4.3 Pilot Survey

To help ensure the understandability of the survey, we asked Computer Science professors and graduate students with experience in SE and experience in survey design to review the survey to ensure the questions were clear and complete. The feedback only suggested minor edits. The changes we made include: adding more answer choices to several questions and adding clarifying examples to three questions.

4.4 Data Collection

On December 13, 2017, we sent each of the 1,604 BCS developers in our list a personalized email mentioning the BCS repository that we mined to obtain his/her email address with a link to the survey hosted on Qualtrics snow2013qualtrics . We also asked the respondents through both the solicitation emails and a reminder in the survey to answer our questions based on his/her personal experiences with the BCS project where we obtained his/her email address. Since 62 of our solicitation emails bounced, we were left with at most 1,542 potential participants, assuming all other emails actually reached their intended recipient. On December 21, 2017, we sent a reminder email. We closed the survey on January 5, 2018; after the response rate slowed to almost no response each day.

Data from the survey link created with Google’s URL shortener showed a total 358 clicks on the survey URL (23% of the invitations). Out of those clicks, 200 people took the survey with a response rate of 13% (200/1542). As most of the questions were optional, many respondents skipped some of the questions. Only 115 respondents answered all the questions. After the exclusion of the 44 responses that did not answer either at least 75% of the questions or at least one open-ended question, we were left with 156 responses for analysis.

After contacting the authors of the MS survey, we were able to obtain their dataset, which allows us to compare and contrast BCS development with three non-BCS domains (i.e., Games, Office, and Other) from Microsoft (MS).

4.5 Qualitative Analysis Process

For the open-ended questions, we followed a systematic qualitative data analysis process. First, two of the authors independently extracted the general themes from the first 75 responses to each question. Using those themes, the authors had discussion sessions to develop an agreed-upon coding scheme for each question. Using this coding scheme, another author went through the remaining answers to determine any additional codes that need to be added.

With this scheme, two of the authors independently coded each response using the Coding Analysis Toolkit (CAT) lu2008rigor software. The coders could also add new codes, if necessary. We computed the level of inter-rater reliability of the manual coding process using Cohen’s kappa Cohen1960 , which was measured as 0.62. While there is no universally accepted ‘good’ kappa, values between 0.61 to 0.80 are generally recognized as ‘substantial agreements’ Landis-Koch:1977 . We used CAT to identify the discrepancies in coding and had discussion sessions to resolve all conflicts. Once we completed the coding process, we transferred the data into SPSS spss for further analysis along with the quantitative data.

5 Demographics

To provide a proper context for the results, this section describes the demographics of the projects represented by the respondents and of the respondents themselves.

5.1 Projects Represented

Table 2 provides the results to Q6 (Table 1) about respondents’ primary projects. The number in parenthesis represents the number of respondents who listed that project. Our respondents represent 61 different BCS projects. The Coin Development Index coin-dev-index , which tracks the top BCS projects, indicates our respondents representing 18 out of the top 25 projects. Also, 37% of our respondents coming from the top ten projects indicates the participation of the top BCS developers in our survey.

Multiple occurences Single occurences
Ethereum (22) Bitcoin (9) Ambisafe Basic Identity Token Bytom
Bitshares (7) Monero (6) Cpuminer Dash Distense
Sia (6) Waves (5) DNSChain Ebets ESKU
Solidity (5) Lbry (4) Etherbet Etherplay Fabric Labs
Ripple (4) Nem (3) Golem Haskoin ZeroLink
Cardano (3) Decred (3) Icofunding Ind Iroha
EOS (3) Hyperledger (3) JS Miner Keyrun Libsnark
IOTA (3) Factom (2) LiteCoin Payroll System PHP-Mpos
Feather coin (2) Lisk (2) Populus Progmathon Pycoin
Metamask (2) Namecoin (2) Shapeshift Snapcoin Status
Neo (2) Remix IDE (2) Steller Storj Swifty
Stratis (2) Trezor (2) Vandal Vcash Viper
Zcash (2) Undisclosed /Private (14)
Table 2: Projects represented

5.2 Respondents’ Demographics

(a) BCS development experience
(b) Type of financial compensation
(c) Total number of code commits in BCS projects
(d) Average number of hours per week spent on BCS development
Figure 2: Demographics of the respondents

In response to Q5, 81.4% of our respondents indicated having less than 2 years of BCS development experiences, while 37.8% had less than a year (Figure 2(a)). In terms of compensation (Figure 2(b)), 43.1% of our respondents do not receive any direct financial benefits. The ratio of volunteers among our respondents is similar to the ratios reported in prior studies bosu2017process ; Bosu-etal:JSS . Among the respondents receiving direct financial compensations, 37.3% reported receiving shares, tokens or cryptocurrencies that may further motivate them to spend efforts to make their projects successful.

In terms of the number of contributions to a BCS project (Figure 2(c)) 57.6% of our respondents have made more than 10, while 42.9% had submitted more than 30. On the other hand, 42.7% of our respondents spend at least 20 hours a week on a BCS project, and 32.7% were working full time (Figure 2(d)). Combining our respondents’ number of commits and number of hours per week spent in BCS projects, we conclude that our respondents include a sample of active BCS developers from the top BCS projects who are qualified to provide valuable insights for the goals of this study.

6 Results

The following subsections describe the results of our survey by answering the five research questions introduced in Section 3. To help clarify the results, we also include excerpts from the qualitative responses to the open-ended questions. Each of the excerpts is followed by a number representing a unique identifier for the respondent who expressed that opinion. For example, [#5] indicates a response from respondent number 5.

As a result of the coding process (Section 4.5), each of the open-ended questions had a large number of detailed categories. For this presentation of the results, we abstracted the detailed categories into a smaller number of high-level categories. In a qualitative analysis, each open-ended response could match multiple codes. Therefore, the sum of the percentages can be greater than 100%.

6.1 RQ1: Who are Contribution to BCS?

Figure 3: Personal characteristics of the BCS developers

Figure 3 shows the personal characteristics of the BCS developers in terms of their age (Q1), gender (Q2), education (Q3), and software development experience (Q5). We also compare these characteristics with the results from the 2018 Stack Overflow Annual Developer Survey (referred as the ‘SO Survey’ hereinafter) so-survey that includes the demographics of 98,855 developers around the globe. Due to its large number of respondents coming from 183 different countries, the SO Survey provides an accurate overview of the active software developers worldwide.

Around 95% of our respondents are males compared to only 3% females (Figure 3(a)). These numbers suggest that the ratio of females in BCS development may be lower compared to the the software development community as reported in the SO survey (6.7%).

In terms age (Figure 3(b)), our respondents are younger compared to the general software developer population. The distributions are noticeably different for two age groups. First, developers, who are aged between 26 to 30 years represent 31% of the BCS developers, compared to 25.4% general software developer population coming from the same age group. On the other hand, although 15.9% general software developer population are aged 40 years and higher, only 11.7% of the BCS developers belong to that age group. Therefore, BCS development may be attracting more young developers than the veterans.

In terms of the highest level of education (Figure 3(c)), our respondents are more qualified with 15.4% high school graduate, 46.8% with a bachelors degree, 32.7% with a masters degree, and 5.1% with a Ph.D. The corresponding numbers in the SO survey are 23.4%, 47.7%, 23.2%, and 2.2% respectively. The higher educational qualifications of our respondents may not be surprising as BCS development requires more in-depth knowledge of computing than non-BCS development zheng2016blockchain .

In terms of software development experiences (Figure 3(d)), 70.5% of our respondents have more than five years of development experiences and 42.3% have more than 10 years. The corresponding numbers in the SO survey are 62.3% and 34.1% respectively. It indicates that the BCS developers are likely to be more experienced in software development than their non-BCS counterparts.

6.2 RQ2: Motivations of BCS Developers

Figure 4(a) shows the primary motivations of BCS developers, which emerged from the answers to Q10 (Table 1) of our survey. Besides technical attraction, the other four categories of motivations that BCS developers have, are similar to the motivations of Open Source Software (OSS) developers hars2001working ; lakhani2003hackers ; ye2003toward . Since blockchain is a new technology, many BCS developers indicate their attractions to this innovative technology as one of their primary motives. While ideology did not top the list of OSS developers’ motives hars2001working ; lakhani2003hackers ; ye2003toward , it ranks top for the BCS developers.

Since results from the psychology domain suggest that a person’s motivation may vary based on his/her age, education level hitka2015impact or compensation hennessey1998reality , we investigated whether those factors had any impact on a BCS developer’s motives. Although, motivations of a person may also depend on his/her gender meece2006gender , we did not conduct a similar investigation based on gender due to the small number of females among our respondents. Our results suggest significant differences (Chi Square, ) in motivations based on the age or compensation of a respondent but no significant difference was found based on the level of education. While most of the BCS developers between 31 to 40 years of age have ideological motives, developers aged between 26 to 30 years are more likely to be motivated by external rewards and developers aged 25 years and younger are more likely to be motivated by the prospects of learning (Figure 4(b)). On the other hand, developers receiving some types of direct compensations are more likely motivated by external rewards. The following subsections examine these motivations in more detail.

(a) Motivations
(b) Age vs. Motivation (percentage computed based on motivations within the same age group)
Figure 4: Motivations of BCS developers

6.2.1 Ideology

The primary motivation behind Bitcoin, the first blockchain based cryptocurrency was to create a decentralized currency that cannot be manipulated by a central authority. A large number of BCS developers are motivated by a similar ideology.

[1em] 0emI truly believe in the right to have a private way to send and store money. Also removing power from banks and governments. [#195]

6.2.2 External rewards

Some developers contribute to BCS projects to earn money either by working part-time or by accepting bounty offers. Developers, who hold cryptocurrency, are naturally motivated to increase its value. [1em] 0em.. earn money; I get salary, and also I bought coins so their growth will give me money. [#147]

Many developers are full-time employees of the organization that manages his/her project. [1em] 0emI am paid to work full-time contributing to blockchain projects. [#75]

6.2.3 Intrinsic

Due to the various programming challenges that BCS development offers, many developers, who enjoy writing code to solve problems, contribute to BCS projects.

[1em] 0emI love coding. I love to solve problems and support people. [#127] Some contribute to BCS projects due to their passions for a particular area.

[1em] 0emIt’s an opportunity to work on a programming language, which I’ve always wanted to do. [#142]

Some want to improve their portfolio through their involvement and recognition in the BCS community. [1em] 0em.. become more famous in the community (get good reputation). [#63]

6.2.4 Technical attraction

Attraction to the blockchain technology is one of the motivations for many BCS developers. [1em] 0emCuriosity mainly. I was studying cryptography by myself before that and saw a chance to see it applied in unbelievable ways. [#31]

6.2.5 Learning

Developers considering blockchain as a promising technology for the future want to learn and add it to their portfolio. [1em] 0em… to develop a better understanding of making and working of a blockchain. [#103]

6.3 RQ3: Differences Between BCS and non-BCS Development

Our survey included two questions to find the differences between BCS and non-BCS development. Figure 5 shows the primary differences between BCS and non-BCS development as indicated by the respondents of our survey in response to Q11. Although BCS development has many similarities to traditional OSS development, due to some unique characteristics of the BCS domain, 93% of our respondents reported BCS development as somewhat different from non-BCS development. Majority of our respondents did not encounter such strict security or reliability requirements while developing a non-BCS. The unique characteristics of the BCS domain (e.g., immutability, difficulty in upgrading the software) was also a differentiating factor for more than one-third respondents.

The following six subsections detail those differences. We conclude this section with a comparison of BCS development with three software development domains from Microsoft (Section 6.3.6).

Figure 5: Differences between BCS and non-BCS development

6.3.1 Security/reliability

A higher emphasis on security and reliability is the primary factor that differentiates BCS from most of the non-BCS. Since the primary applications of the blockchain technology are maintaining ledgers of financial transactions, ensuring the security and reliability of a BCS application is the highest priority. [1em] 0emSecurity and backward compatibility are held with utmost importance here unlike some other FOSS projects. [#3]

A single defect in a BCS application can cost millions of dollars. For example, recently a bug in the parity wallet code enabled hackers to steal $30M worth Ethereum tokens santiagopalladino2017 . [1em] 0em..you should be more careful because there is too much at stake (nowadays a lot of money are invested in cryptocurrencies). In most of the projects (non-blockchain) when a bug appears, it will be fixed and soon forgotten. But in blockchain projects some bugs can be very costly and never forgotten. [#147]

6.3.2 Domain characteristics

Several characteristics of the BCS domain also differentiate BCS development from a non-BCS development. First, data stored in a blockchain is immutable. In other domains, there are several mechanisms to fix errors later by altering data. However, altering a blockchain ledger is almost impossible. [1em] 0em… also a disadvantage that you can not fix problems due to human errors or bugs in that transactions can not be changed. [#97]

Second, compared to the most non-BCS applications that operate on centralized and/or hosted environments, BCS applications operate on a complex, secured, distributed and decentralized network. [1em] 0emThe distributed nature of blockchain software development makes it difficult to build robust software. Unreliable connections, unexpected latency, and malicious nodes create a hostile production environment. [#135]

Third, blockchains use public key cryptography and cryptographic hash functions to store and verify transactions. Cryptography is difficult to master and very few other domains require similar in-depth knowledge of cryptography as the BCS domain. Moreover, the knowledge of networking and networking security is a must for BCS development. The daunting requirement of having knowledge about diverse technological areas is another differentiating factor. [1em] 0emAll kinds of knowledge are involved. It’s hard to comprehend the whole project, including cryptography, network programming, economy policy etc.[#166]

Finally, the blockchain technology is changing rapidly with new protocols, innovations, and possibilities emerging everyday. BCS projects that are not able to evolve rapidly are at the risk of loosing their market capitalization. [1em] 0emTechnology moves fast and blockchain software development (in general, not just Ethereum) is moving at an extremely fast pace. Need to keep up and adjust to meet whatever new requirements arise.[#98]

6.3.3 Immature ecosystem

Many of the innovative aspects of the blockchain technology (e.g., smart-contract, privacy) are relatively new. Although the number of BCS projects have grown exponentially during the last couple of years, many tools and libraries that may support BCS development are still missing. Even the tools that currently exist are not stable. Therefore, the BCS development ecosystem is immature compared to most of the non-BCS counterparts. [1em] 0emWe are in 1986 on a web development timeline, to my mind. Like everything is in C can hit memory leaks, crashes, etc. Everything very raw and dangerous. We still have most of the stack to build. [#137]

Since Blockchain is a new technology, there is scarcity of enough domain-experienced developers compared to most non-BCS domains. The demographics of our survey also shows more than 83% respondents with less than two years of experience in BCS development. [1em] 0emPeople have more experience in other areas. [#48]

6.3.4 Maintainability

Maintaining a BCS application is difficult compared to a non-BCS application. Blockchains usually incorporate new functionality through hard forks hardfork , which is usually scheduled way ahead of time to ensure that all the nodes are running the same version of the software. Due to the difficulty in upgrade, a BCS application needs to be well tested before a release. [1em] 0em.. non-blockchain software can easily upgrade features but blockchain software needs to wait for the preparation of nodes all over the world to upgrade functionality. [#22]

BCS applications run on public blockchains that have thousands of blocks created over the years. Therefore, a BCS application must be backward compatible and capable of validating earlier transactions or executing smart-contracts deployed through earlier versions. [1em] 0emEthereum is a public chain. So we have to consider backward compatibility all the time. [#28]

6.3.5 No difference

Some BCS developers do not find much difference between BCS and non-BCS development. [1em] 0emNot that different from other performance-sensitive, high reliability software. [#9]

Statement Rating
distribution
BCS Vs. Games BCS Vs. Office BCS Vs. Other
          A. Quality assurance
QA1. My software is well tested manually (e.g., paid testers thoroughly use the software).

0.82 0.06 0.02 0.14 0.04 0.12
QA2. My software is well tested by automated testing (e.g., scripts that thoroughly use the software).

0.32 0.04 0.12 0.18
QA3. My software is well tested by unit tests.

0.34 0.31 0.45 0.01
QA4. Bugs in my software are hard to diagnose.

0.23 0.12 0.08 0.2 0.06
QA5. It’s difficult to write thorough automated tests for my software because it’s so complex.

0.01 0.15 0.08 0.1 0.93 0.1
         B. Development process
DP1. My team has flexible release deadlines.

0.46 0.53 0.44
DP2. My software has clear functional requirements.

0.01 0.14 0.62 0.02 0.91 0.09
DP3. My team uses a waterfall process, rather than an agile process.

0.03 0.12 0.65 0.03 0.07 0.08
DP4. My team adheres strictly to a process (for example, scrum or waterfall).

0.2 0.37 0.29
         C. Software maintenance
SM1. My software has high technical debt (for example, a lot of hacks).

0.27 0.04 0.08 0.1 0.39 0.02
SM2. The technical debt is likely to be paid down in the future (for example, through refactoring).

0.18 0.23 0.01 0.17
SM3. My software’s architecture evolves significantly as the software gets more mature.

0.23 0.05 0.06 0.09 0.32 0.03
         D. Ideology / morale
DM1. My software creates value for society.

0.26 0.15 0.07 0.01 0.17
DM2. Creativity is highly valued on my team.

0.02 0.14 0.4 0.02 0.36 0.02
DM3. Creating my software is challenging.

0.19 0.06 0.5 0 0.66 0.03
DM4. When I tell people outside of my company about the software I work on, they are impressed.

0.03 0.13 0.07 0.1 0.05 0.11
Table 3: Comparison of BCS development with three non-BCS domains at Microsoft. Greybackground indicates a statistically significant difference. For the effect sizes (), a value in red indicates BCS developers agreeing less with that statement, while a value in green indicates more agreement from them.

6.3.6 How does BCS development differ from software development at Microsoft?

The ‘Rating distribution’ column in Table 3 shows the distributions of the ratings by our respondents for 16 statements ( The ‘Statement’ column in Table 3). The leftmost bar indicates ‘Strongly disagree,’ the middle bar indicates ‘Neutral,’ and the rightmost bar indicates ‘Strongly agree.’ We do not plot the ratings from the Microsoft developers since a prior publication reports those murphy2014cowboys .

Since the Shapiro-Wilk shapiro1965

test indicates that those ratings significantly differ from a normal distribution, we use non-parametric statistics. For each of the statements, we compare the responses from BCS developers with the responses of Microsoft developers from three domains (i.e., Games, Office, and Others). The three ‘

’ columns show statistical significance of the differences with the three domains based on the Wilcoxon rank sum test. The three ‘

’ columns show the effect sizes estimated using Rosenthal’s formula 

rosenthal1994parametric . Rosenthal rosenthal1996qualitative also recommends interpreting those effect sizes as: large, medium, and small.

Posthoc, we grouped the 16 statements into following four categories based on the theme of each statement.

A. Quality Assurance: BCS developers agree more than both Games and Office developers that their software is well tested using unit tests (QA2) and automated tests (QA3). However, for manual testing (QA1) the result is mixed. They agree more than Other but less than Office. From the remaining two statements, BCS developers agree more than the Games developers on bug diagnosis difficulty (QA4) but disagree with them on the difficulty of writing unit tests (QA5). These results indicate BCS developers’ higher emphasis on automated testing to ensure the quality of their software.

B. Development process: Among the four groups, BCS developers have the most flexible deadlines (DP1). Since the costs of defects are very high and patching a defect after release is very difficult, BCS developers are more flexible in postponing a release. On the other hand, among the four groups they are the least strict in following a process (DP4).

C. Software maintenance: Among the four groups, BCS developers are the most focused on paying technical debts (SM2), which further emphasizes the importance of long term maintainability of a BCS application. For the remaining two statements the differences were not significant.

D. Ideology/morale: In terms of ideology/morale, BCS has no significant difference with Office. The results of comparisons with the remaining two groups are mixed. On taking outside pride of their work (DM4), they agree more than Other but less than Games; while on creating value for the society (DM1), BCS tops both. BCS developers also agree less than the Games developer on the valuation of creativity (DM2) within their team.

In summary, BCS has the most differences with Games (i.e., 12), followed by Other (i.e., 7) and the least with Office (i.e., 6). BCS development has higher emphasis on security, reliability, and maintainability but lower emphasis on following a process or deadline than the other three domains. Apart from those differences, BCS development may not be much different from traditional development domains such as Office or Other.

6.4 RQ4: Challenges of BCS Development

Figure 6 shows the primary types of challenges of BCS development according to the respondents of our survey (Q13: Table 1). Since prior results from the Psychology domain suggest that a person’s ability to acquire knowledge of a new domain depends on his/her age and education level beier2005age , we investigated whether those factors have any impact on the challenges reported by the BCS developers from our survey. Although we did not find any differences based on the education level, we found that the developers aged 40 years and older, who are likely to have significant experiences in non-BCS domains, were more likely () to be challenged by the characteristics of the BCS domain (Section 6.4.1) than the other age groups.

Figure 6: Primary challenges of BCS development

6.4.1 Domain characteristics

Some of the characteristics of the BCS domain are the primary sources of challenges for more than 60% BCS developers. Around 70% respondents of our survey have more than five years of software development experience. Therefore, the aspects of BCS development that differ from a non-BCS are primary sources of challenges for them. While discussing BCS development challenges, our respondents reiterated following differences already mentioned earlier (section 6.3):

  • high costs of defects (section 6.3.1);

  • technological complexity (section 6.3.2);

  • distributed, decentralized, and hostile
    environment (section 6.3.2);

  • rapidly changing ecosystem (section 6.3.2); and

  • difficulty in maintenance (section 6.3.4).

Steep learning curves to get familiar with a BCS project is another challenge for many BCS developers. The codebase of a BCS project is not only complex but also requires a sound understanding of cryptography, networking, distributed systems as well as project specific protocols.

[1em] 0emThe bitcoin core codebase is an incredibly complicated system, so the hardest part is building up a good enough understanding of it such that it’s safe to make changes. [#156]

6.4.2 Development

BCS development challenges are related to testing, ensuring security, and reviewing code. Testing blockchain software is challenging as the software executes on a distributed and potentially hostile environment that currently cannot be adequately simulated on a development machine. [1em] 0em.. to test that bugs are not included in important parts such as consensus. [#22]

The security of a BCS software is the highest priority as it handles financial data that can be exploited for financial gains. Therefore, BCS developers must consider security aspects when writing code.

[1em] 0em… thinking with a security mindset. the software is literally money, so when writing it you have to be wary of any way in which things could go wrong.[#143]

Due to the lack of quality reviewers, open source software (OSS) developers often have to wait for a long time to get their code reviewed bosu2014impact . BCS developers also report similar challenges. [1em] 0em… it is often difficult to get code reviews on open source projects even with a history of contributions. [#75]

6.4.3 Non-technical

The non-technical challenges of BCS development are due to collaboration issues, difficulties in reaching agreement among the community members, and the ethical aspects of BCS. Most of the BCS projects are run by communities. However, many projects have very high valuations and a large sum of money is at stake. Therefore, community members often engage in arguments to decide a project roadmap bitcoin-poitics . [1em] 0emGovernance, deciding on changes, and set a future roadmap. [#35]

The development team comprising volunteers with diverse personalities, background, experience, and motivation often encounter issues to collaborate. [1em] 0emnothing particular to the blockchain, prickly personalities! [#51]

Some BCS developers are primarily motivated by the prospects of financial gains and will not hesitate to adopt an unethical approach if presented with an opportunity. Without a central monitoring authority, preventing developers with malicious intents is a challenge. [1em] 0emhuman greed, and cleaning up the mess after bounties and scams on forum.[#44]

6.4.4 Lack of supporting materials

The lack of supporting tools and documentation is a source of challenges for the BCS developers. Many of the supporting tools that can help their development tasks are yet to be developed. Moreover, the tools that are currently available are immature and unreliable. [1em] 0emReliability of and the lack of good development tools. Like testing frameworks, and the difficulty of debugging.[#16] To understand a complicated domain such as BCS, developers are looking for good learning materials and tutorials, but such materials are currently rarities. Even the documentation that they currently have are not user friendly.

[1em] 0emBlockchain is totally a new technology so there is few information we can find.

6.5 RQ5: Tools that BCS Developers Need

We asked the respondent of our survey to describe the type of tools that they currently do not have but if implemented can significantly improve their development productivity (Q14: Table 1). In response, our respondents indicated their needs for four categories of tools (Figure 7). We also hypothesized the the requirement for supporting tool may vary based on the software development experience or BCS development experience of a respondent. However, the results of our analysis did not find any statistically significant difference.

Figure 7: Categories of tools that BCS developers need

6.5.1 Testing support

Testing is the area of BCS development that currently needs supporting tools the most. BCS developers use ‘testnets’ testnet to experiment their code before deploying it to the ‘mainnet’ mainnet . Many developers experience difficulties in setting up a testnet. An easy to setup, ‘one-click testnet’ may be a solution. [1em] 0emEasy way of forking mainnets for testing purposes, a way to deploy a test net in one click would be nice.[#150]

The simulators that BCS developers currently have are limited and are unable to simulate a complex and hostile real world environment. While they have testnets, but the scales and complexities of testnets are no where closer to a mainnet. Few recent works stoykov2017vibes ; chen2017using have attempted to build such simulators, but no satisfactory solution exists yet. [1em] 0emEasier ways to simulate complex network topologies on one single machine to simulate the network. [#195]

Formal verification clarke1996formal techniques have been useful to secure BCS projects. However, developers find current formal specification languages (e.g., TLA+, VDM, and Z-notation) very complex to learn and use. They wish for user friendly alternatives. [1em] 0emEnd-to-end formal specification and verification tools with notations that mere programmers can understand. I sometimes think our formalists actually like to make their work obscure. [#1]

Static analysis and penetration testing tools have been useful in non-BCS domains for security testing chess2004static ; arkin2005software . Since, those tools do not work well on BCS codebase, developers wish for automated security testing tools designed specifically for the BCS domain. [1em] 0emFuzz testing, something like linting for security best practices. [#196]

6.5.2 Development support

The IDEs, designed for the non-BCS domains, lack adequate supports for testing and debugging a BCS codebase. Therefore, BCS developers use an array of tools for various development activities. An IDE designed specifically for BCS development would help them. [1em] 0em… they are all mostly disconnected, I have to switch from one tool/platform to another for my regular development activities many times every day. It would help if there were better integration between them all. [#66]

Even the tools that exist are not reliable, and BCS developers wish them to be more stable. [1em] 0emTools exist that are in their infancy and just need to get better. [#13]

6.5.3 Smart contract development

Smart-contracts are written using contract-oriented programming language such as Solidity or Vyper and then compiled into bytecode for a platform (e.g., Ethereum Virual Machine aka EVM). Remix, an IDE for solidity, currently lacks many features such as: error highlighting and line by line debugging, that BCS developers wish for. [1em] 0emBetter tool support for smart contract development. IDE integration with interactive debugging. [#137]

Before interacting with a smart-contract, a developer might want to verify its security properties by decompiling its bytecode. While few solutions exist suiche2017porosity , smart-contract developers wish for a reliable and user-friendly decompiler.

[1em] 0em… high-level Solidity decompiler that works (the current EVM-to-Solidity decompilers are horrible). [#113]

6.5.4 Others

The other tools wished by the respondents include UML/design notations for the BCS domain, containers for deployment, and automated performance analysis tools.

7 Discussion

In the following subsections, we discuss our findings and provide potential future research directions.

7.1 Contemporary BCS Developers

Due to the significant financial gains by the early cryptocurrency investors as well as a large influx of cash through ICOs, we hypothesized that the majority of the BCS developers might be motivated by external rewards. However, the ratio of our respondents reporting external motives (36%) were similar to the number from prior OSS studies hars2001working ; lakhani2003hackers . Moreover, the ratio of volunteers (43%) is also similar to what was reported in prior studies lakhani2003hackers ; Bosu-etal:JSS . On the other hand, ideological motives were more frequent among BCS developers (37%) than among the OSS developers hars2001working ; lakhani2003hackers . Therefore, ideology rather than financial rewards may be driving the top BCS projects.

Although we noticed that more than 81% of our respondents have less than 2 years of BCS development experience (Section 5.2), more than 70% developers from the same group were found to have more than five years of development experiences (Section 6.2). These numbers indicate that a large number of software developers, who are experienced in non-BCS development, have recently joined BCS projects potentially due to the recent hypes generated by the blockchain technology.

7.2 Is BCS development Really Different?

The answer to this question will depend on whom we ask. It’s true that BCS development has a very high emphasis on security and reliability, but many of the existing software development domains (e.g., financial transaction, air traffic controller, and nuclear power plant management) have a similar emphasis on security and reliability. If a developer’s non-BCS experience is in high assurance software (Section 6.3.5), then he/she might find little differences. However, over 93% of our respondents’ non-BCS experiences significantly differed from their experiences in BCS (Figure 5). Our survey received responses from 10% of the most active developers from the top BCS projects, and 70% of our respondents have more than 5 years of development experiences (Section 6.2). Yet, they found BCS development different and more challenging than non-BCS. Since most of the non-BCS domains do not have similar high reliability and security requirements as the BCS domain, developers coming from other domains (except high assurance software) will encounter challenges due to those differences (Section 6.4). Although the immaturity of the ecosystem will resolve with time, immutability of data as well as difficulty in upgrading the software after deployment, which is rare among the non-BCS domains, will remain as a differentiating factor.

7.3 Notes for Prospective Joiners

BCS Developers must be careful in writing code due to high costs of defects as well as difficulty in upgrading the software. Yet, they are under constant pressure due to a rapidly changing ecosystem as well as high expectations from the stakeholders to release new versions. Moreover, as the blockchains become more popular the scalability of BCS software has become an area of concern porru2017blockchain . As a result BCS development is challenging even for developers with considerable non-BCS experiences (Section 6.4.1).

BCS development also requires an in-depth knowledge of cryptography, networking, security, and distributed systems (Section 6.4.1). Yet, good tutorials and learning materials are hard to find (Section 6.4.4). Therefore, developers interested in contributing to BCS projects should be prepared to take considerable challenges in gaining the required skills.

7.4 Tool development

Based on the personal experiences of our respondents, they found some widely used tools tuned for non-BCS development, lacking required support for BCS development. While some of the needs expressed by our respondents (e.g., easy to write formal specification) may be a wishful thinking and difficult to achieve, most of those tools are feasible. Potentially implementable tools for BCS development include: testing environment, automatic security testing, static analysis for smart-contrats, and easy to deploy testnets. Since an array of research predict tremendous impacts of the blockchain technology and smartcontracts in future iansiti2017truth ; peters2016understanding ; fanning2016blockchain , the number of BCS projects and developers contributing to those projects will grow significantly over the next decade. Therefore, research and development efforts should focus on implementing those tools to build a mature BCS development ecosystem.

7.5 Research Directions

Our study identified several fertile SE research areas for developing an array of tools and techniques for BCS development. First, since several characteristics of the BCS domain is different from a non-BCS domain, quite a few categories of tools (e.g., IDE, simulator, and static analysis tools), that are stable and mature for non-BCS development, either do not exist or lacks important support for BCS (Section 6.5.1). Due to the lack of appropriate testing supports, BCS developers primarily depend on manual code reviews to ensure the security of their software partha2018 . As a result expensive bugs and hacks are very frequent in the BCS ecosystem Parity ; wan2017bug ; porru2017blockchain .

Second, smart-contracts, which open a new type of programming paradigm, are gaining popularities luu2016making . Yet, the smart-contract programming domain lacks basic development supports, such as an IDE with error highlighting, line by line debugging, and decompilers (Section 6.5.3). As a result, many of the smart-contracts deployed on public blockchains are vulnerable nikolic2018finding .

Finally, due to a higher emphasis on security and reliability, BCS developers are highly interested in writing formal specifications for their software. However, current formal verification languages and tools are difficult to learn and use for BCS (Section 6.5.1). Therefore, another potential direction could be building customized formal specifications for the BCS domain.

8 Threats to validity

The primary threats to validity is related to sample selection. The respondents of our survey may not adequately represent all BCS developers. While our respondents come from 61 different BCS projects, they primarily represent the top ones. Therefore, some of the opinions, especially, the challenges encountered by BCS developers in smaller projects may be different from those included in this study. However, our results also indicate that BCS developers’ challenges are primarily due to the differences between BCS and non-BCS development, which may be similar across all projects.

We selected five commits as a threshold for an invitation to this survey. A higher or lower threshold may have altered the results. However, none of the results of our research questions significantly differed based on the number of pull requests of a respondent (Q8). Therefore, this threat may be minimal.

Although the response rate of our survey is similar to prior SE surveys bosu2017process ; kononenko2016code , the response rate is only (13%). Therefore our results could suffer from a potential ‘non-response bias’ (i.e., the opinions of the respondents who chose to participate may be different from who did not) armstrong1977estimating . Even so, the 156 responses that we analyzed provide a rich source of data to reveal the insights described in this paper.

Section 6.3.6 compares BCS development with three non-BCS development domains from the same organization (i.e., Microsoft). Therefore, differences identified in that section may not apply to a different domain or a different organization. However, MS developers are eligible to provide us opinions on non-BCS development, since Microsoft is a well-recognized and mature software development organization.

9 Conclusion

Despite the existence of a large number of active BCS projects as well as tremendous developer interests in the blockchain technology, there have been few empirical SE research exploring this area. In an attempt to bridge this gap, we studied the motivations, challenges, and needs of BCS developers. Our results suggest that BCS development has several differences from non-BCS development that pose challenges to the developers. As a young and immature ecosystem, BCS domain needs an array of tools and resources that SE research can contribute to developing or improving.

Acknowledgements.
We are grateful to Emerson Murphy-Hill from NC State University; Tom Zimmermann, and Nachi Nagappan from Microsoft Research for providing us the dataset of their Survey at Microsoft. We thank the anonymous respondents of our survey.

References

  • (1) Adhami, S., Giudici, G., Martinazzi, S.: Why do businesses go crypto? an empirical analysis of initial coin offerings. Journal of Economics and Business (2018)
  • (2) Arkin, B., Stender, S., McGraw, G.: Software penetration testing. IEEE Security & Privacy 3(1), 84–87 (2005)
  • (3) Armstrong, J.S., Overton, T.S.: Estimating nonresponse bias in mail surveys. Journal of marketing research pp. 396–402 (1977)
  • (4) Azaria, A., Ekblaw, A., Vieira, T., Lippman, A.: Medrec: Using blockchain for medical data access and permission management. In: Open and Big Data (OBD), International Conference on, pp. 25–30. IEEE (2016)
  • (5) Beier, M.E., Ackerman, P.L.: Age, ability, and the role of prior knowledge on the acquisition of new domain knowledge: promising results in a real-world learning environment. Psychology and aging 20(2), 341 (2005)
  • (6) Bosu, A., Carver, J., Guadagno, R., Bassett, B., McCallum, D., Hochstein, L.: Peer impressions in open source organizations: A survey. Journal of Systems and Software 94(0), 4 – 15 (2014)
  • (7) Bosu, A., Carver, J.C.: Impact of developer reputation on code review outcomes in oss projects: An empirical investigation. In: Proceedings of the 8th ACM/IEEE international symposium on empirical software engineering and measurement, p. 33. ACM (2014)
  • (8) Bosu, A., Carver, J.C., Bird, C., Orbeck, J., Chockley, C.: Process aspects and social dynamics of contemporary code review: insights from open source development and industrial practice at microsoft. IEEE Transactions on Software Engineering 43(1), 56–75 (2017)
  • (9) Chakraborty, P., Shahriyar, R., Iqbal, A., Bosu, A.: Understanding the software development practices of blockchain projects: A survey. In: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM ’18, pp. 28:1–28:10 (2018)
  • (10) Chen, C., Qi, Z., Liu, Y., Lei, K.: Using virtualization for blockchain testing. In: International Conference on Smart Computing and Communication, pp. 289–299. Springer (2017)
  • (11) Chess, B., McGraw, G.: Static analysis for security. IEEE Security & Privacy 2(6), 76–79 (2004)
  • (12) Chuen, D.L.K.: Handbook of digital currency: Bitcoin, innovation, financial instruments, and big data. Academic Press (2015)
  • (13) Clarke, E.M., Wing, J.M.: Formal methods: State of the art and future directions. ACM Computing Surveys (CSUR) 28(4), 626–643 (1996)
  • (14) Cohen, J.: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1), 37–46 (1960)
  • (15) Dahlander, L., Mckelvey, M.: Who is not developing open source software? non-users, users, and developers. Economics of Innovation and New Technology 14(7), 617–635 (2005)
  • (16) David, P.A., Shapiro, J.S.: Community-based production of open-source software: What do we know about the developers who participate? Information Economics and Policy 20(4), 364 – 398 (2008). Empirical Issues in Open Source Software
  • (17) De Filippi, P., Loveluck, B.: The invisible politics of bitcoin: Governance crisis of a decentralized infrastructure. Internet Policy Review 5(4), 529–546 (2016)
  • (18) Delmolino, K., Arnett, M., Kosba, A., Miller, A., Shi, E.: Step by step towards creating a safe smart contract: Lessons and insights from a cryptocurrency lab. In: International Conference on Financial Cryptography and Data Security, pp. 79–94. Springer (2016)
  • (19) Destefanis, G., Marchesi, M., Ortu, M., Tonelli, R., Bracciali, A., Hierons, R.M.: Smart contracts vulnerabilities: a call for blockchain software engineering? In: Proceedings of the 2018 International Workshop on Blockchain Oriented Software Engineering, pp. 19–25. IEEE, Campobasso, Italy (2018)
  • (20) Dev, J.A.: Bitcoin mining acceleration and performance quantification. In: Electrical and Computer Engineering (CCECE), 2014 IEEE 27th Canadian Conference on, pp. 1–6. IEEE (2014)
  • (21) Documentation, B.D.: Mainnet, bicoin main network. https://bitcoin.org/en/glossary/mainnet. Accessed: 2018-01-04
  • (22) Fanning, K., Centers, D.P.: Blockchain and its coming impact on financial services. Journal of Corporate Accounting & Finance 27(5), 53–57 (2016)
  • (23) Gandal, N., Halaburda, H.: Competition in the cryptocurrency market (2014)
  • (24) Gandal, N., Halaburda, H.: Can we predict the winner in a market with network effects? competition in cryptocurrency market. Games 7(3), 16 (2016)
  • (25) Garfinkel, S.L.: Public key cryptography. Computer 29(6), 101–104 (1996)
  • (26) Gilbertson, T., Vroegindewey, L.: Larry’s cryptocoin risk (2017). URL https://www.coindevelopmentindex.com/
  • (27) Hars, A., Ou, S.: Working for free? motivations of participating in open source projects. In: System Sciences, 2001. Proceedings of the 34th Annual Hawaii International Conference on, pp. 9–pp. IEEE (2001)
  • (28) Hayes, A.S.: Cryptocurrency value formation: An empirical study leading to a cost of production model for valuing bitcoin. Telematics and Informatics 34(7), 1308–1321 (2017)
  • (29) Hennessey, B.A., Amabile, T.M.: Reality, intrinsic motivation, and creativity. American Psychologist 51, 1153–1166 (1998)
  • (30) Hertel, G., Niedner, S., Herrmann, S.: Motivation of software developers in open source projects: an internet-based survey of contributors to the linux kernel. Research policy 32(7), 1159–1177 (2003)
  • (31) Hitka, M., Balážová, Ž.: The impact of age, education and seniority on motivation of employees. Business: Theory and practice 16, 113 (2015)
  • (32) Huh, S., Cho, S., Kim, S.: Managing iot devices using blockchain platform. In: Advanced Communication Technology (ICACT), 2017 19th International Conference on, pp. 464–467. IEEE (2017)
  • (33) Iansiti, M., Lakhani, K.R.: The truth about blockchain. Harvard Business Review 95(1), 118–127 (2017)
  • (34) J. Richard Landis, G.G.K.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977). URL http://www.jstor.org/stable/2529310
  • (35) Jacobovitz, O.: Blockchain for identity management (2016)
  • (36) Jakobsson, M., Juels, A.: Proofs of work and bread pudding protocols. In: Secure Information Networks, pp. 258–272. Springer (1999)
  • (37) Kononenko, O., Baysal, O., Godfrey, M.W.: Code review quality: how developers see it. In: Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on, pp. 1028–1038. IEEE (2016)
  • (38) Korpela, K., Hallikas, J., Dahlberg, T.: Digital supply chain transformation toward blockchain integration. In: Proceedings of the 50th Hawaii international conference on system sciences (2017)
  • (39) Krafft, P.M., Della Penna, N., Pentland, A.S.: An experimental study of cryptocurrency market dynamics. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, pp. 605:1–605:13 (2018)
  • (40) Lakhani, K.R., Wolf, R.G.: Why hackers do what they do: Understanding motivation and effort in free/open source software projects. MIT Sloan Working Paper 4425-03 (September 2003)
  • (41) Lee, A., Carver, J.C., Bosu, A.: Understanding the impressions, motivations, and barriers of one time code contributors to floss projects: A survey. In: Proceedings of the 39th International Conference on Software Engineering, ICSE ’17, pp. 187–197. IEEE Press, Piscataway, NJ, USA (2017). DOI 10.1109/ICSE.2017.25. URL https://doi.org/10.1109/ICSE.2017.25
  • (42) Lu, C.J., Shulman, S.W.: Rigor and flexibility in computer-based qualitative research: Introducing the coding analysis toolkit. International Journal of Multiple Research Approaches 2(1), 105–117 (2008)
  • (43) Luu, L., Chu, D.H., Olickel, H., Saxena, P., Hobor, A.: Making smart contracts smarter. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 254–269. ACM (2016)
  • (44) Meece, J.L., Glienke, B.B., Burg, S.: Gender and motivation. Journal of school psychology 44(5), 351–373 (2006)
  • (45) Mockus, A., Fielding, R.T., Herbsleb, J.D.: Two case studies of open source software development: Apache and mozilla. ACM Trans. Softw. Eng. Methodol. 11(3), 309–346 (2002)
  • (46) Murphy-Hill, E., Zimmermann, T., Nagappan, N.: Cowboys, ankle sprains, and keepers of quality: How is video game development different from software development? In: Proceedings of the 36th International Conference on Software Engineering, pp. 1–11. ACM (2014)
  • (47) Nakamoto, S.: Bitcoin: A peer-to-peer electronic cash system (2008)
  • (48) Narayanan, A., Bonneau, J., Felten, E., Miller, A., Goldfeder, S.: Bitcoin and Cryptocurrency Technologies: A Comprehensive Introduction. Princeton University Press (2016)
  • (49) Nikolic, I., Kolluri, A., Sergey, I., Saxena, P., Hobor, A.: Finding the greedy, prodigal, and suicidal contracts at scale. arXiv preprint arXiv:1802.06038 (2018)
  • (50) Overflow, S.: Stack overflow annual developer survey (2017). URL https://insights.stackoverflow.com/survey/
  • (51) Palladino, S.: The parity wallet hack explained (2017). URL https://blog.zeppelin.solutions/on-the-parity-wallet-multisig-hack-405a8c12e8f7
  • (52) Peters, G.W., Panayi, E.: Understanding modern banking ledgers through blockchain technologies: Future of transaction processing and smart contracts on the internet of money. In: Banking Beyond Banks and Money, pp. 239–278. Springer (2016)
  • (53) Porru, S., Pinna, A., Marchesi, M., Tonelli, R.: Blockchain-oriented software engineering: challenges and new directions. In: Proceedings of the 39th International Conference on Software Engineering Companion, pp. 169–171. IEEE Press, Buenos Aires, Argentina (2017)
  • (54)

    Roberts, J.A., Hann, I.H., Slaughter, S.A.: Understanding the motivations, participation, and performance of open source software developers: A longitudinal study of the apache projects.

    Management science 52(7), 984–999 (2006)
  • (55) Rosenthal, J.A.: Qualitative descriptors of strength of association and effect size. Journal of social service Research 21(4), 37–59 (1996)
  • (56) Rosenthal, R.: Parametric measures of effect size. The handbook of research synthesis pp. 231–244 (1994)
  • (57)

    Shapiro, S.S., Wilk, M.B.: An analysis of variance test for normality (complete samples).

    Biometrika 52(3/4), 591–611 (1965)
  • (58) Simmons, G.J.: Symmetric and asymmetric encryption. ACM Computing Surveys (CSUR) 11(4), 305–330 (1979)
  • (59) Snow, J., Mann, M.: Qualtrics survey software: handbook for research professionals. Qualtrics Labs, Inc (2013)
  • (60) SPSS, I.: Spss statistical software. IBM Corporation 24 (2017)
  • (61) Stoykov, L., Zhang, K., Jacobsen, H.A.: Vibes: fast blockchain simulations for large-scale peer-to-peer networks. In: Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference: Posters and Demos, pp. 19–20. ACM (2017)
  • (62) Suiche, M.: Porosity: A decompiler for blockchain-based smart contracts bytecode. DEF CON 25 (2017)
  • (63) Swan, M.: Blockchain: Blueprint for a new economy. ” O’Reilly Media, Inc.” (2015)
  • (64) Technologies, P.: A postmortem on the parity multi-sig library self-destruct. https://paritytech.io/a-postmortem-on-the-parity-multi-sig-library-self-destruct/ (2017)
  • (65) technology, S.: Blockchain software security best practices. https://www.synopsys.com/blogs/software-security/blockchain-software-security-best-practices/ (2018)
  • (66) Underwood, S.: Blockchain beyond bitcoin. Communications of the ACM 59(11), 15–17 (2016)
  • (67) Von Krogh, G., Haefliger, S., Spaeth, S., Wallin, M.W.: Carrots and rainbows: Motivation and social practice in open source software development. MIS quarterly pp. 649–676 (2012)
  • (68) Wan, Z., Lo, D., Xia, X., Cai, L.: Bug characteristics in blockchain systems: a large-scale empirical study. In: Mining Software Repositories (MSR), 2017 IEEE/ACM 14th International Conference on, pp. 413–424. IEEE (2017)
  • (69) West, J., Gallagher, S.: Challenges of open innovation: the paradox of firm investment in open-source software. R&d Management 36(3), 319–331 (2006)
  • (70) Wiki, B.: Hardfork. https://bitcoin.org/en/glossary/hard-fork. Accessed: 2018-01-04
  • (71) Wiki, B.: Testnet. https://en.bitcoin.it/wiki/Testnet. Accessed: 2018-01-04
  • (72) Ye, Y., Kishida, K.: Toward an understanding of the motivation open source software developers. In: Proceedings of the 25th international conference on software engineering, pp. 419–429. IEEE Computer Society (2003)
  • (73) Ye, Y., Kishida, K.: Toward an understanding of the motivation open source software developers. In: Proceedings of the 25th International Conference on Software Engineering, ICSE ’03, pp. 419–429. IEEE Computer Society, Washington, DC, USA (2003). URL http://dl.acm.org/citation.cfm?id=776816.776867
  • (74) Zheng, Z., Xie, S., Dai, H., Chen, X., Wang, H.: An overview of blockchain technology: Architecture, consensus, and future trends. In: Big Data (BigData Congress), 2017 IEEE International Congress on, pp. 557–564. IEEE (2017)
  • (75) Zheng, Z., Xie, S., Dai, H.N., Wang, H.: Blockchain challenges and opportunities: A survey. Work Pap.–2016 (2016)