The inclusion of human-centered approaches in software engineering has received a lot of attention in recent years (Da Silva et al., 2011; Salah et al., 2014; Brhel et al., 2015; Schön et al., 2017). Human-centered design (HCD) (DIS, 2009), design thinking (DT) (Brown, 2009) and participatory design (PD) (Spinuzzi, 2005) have been shown to be beneficial for the software design and development process (Da Silva et al., 2011), especially if the designer is aware of their challenges and limitations (Salah et al., 2014; Bordin and De Angeli, 2016).
While there is a growing body of research in exploring user research and agile methods, very little has been done in the area of designing software for vulnerable populations. The exact definition of the term ”vulnerable populations” is the focus of many discussions (Ruof, 2004). Sometimes it is defined vaguely, other times by extension (listing the conditions of users). In this paper we use it to refer to people potentially exposed to harm or not capable of protecting their own interests.
We argue that software engineering methods - and software engineers - are ill-prepared for addressing this type of projects. An obvious first consideration is that reasoning on ethics and values tends to be more complex and that each user study requires a very careful design - as well as the need to follow specific guidelines and undergo reviews by an Institutional Review Board (IRB) (for the Protection of Human Subjects of Biomedical and Research, 1978). AS we will see the issues go much beyond the incorporation of an ethical approval process (which we found beneficial, besides being appropriate and required).
In this short paper we join the thread of work on the interplay between values and software engineering methods (Ferrario et al., 2014, 2016). We report on our experiences in developing applications for institutionalised older adults over the past years and summarise the lessons we have learned, especially in terms of how we adapted the agile processes we used to follow to cope with the scenarios at hand, and translate the lessons into a corresponding set of recommendations. We hope and trust that this will help teams be more effective when dealing with this scenario and avoid mistakes that can be very costly and hard to recover from.
2.1. Human-centered agile development
A vast literature has investigated how to combine human-centered approaches with agile methodologies. Systematic literature reviews have focused on the principles user-centered agile development (Brhel et al., 2015), recurring patterns in the integration (Da Silva et al., 2011) and characteristics of stakeholder involvement (Schön et al., 2017). The most relevant to our discussion is that of Salah et. al (Salah et al., 2014), which summarises the challenges of integrating agile methodologies with HCD. This review analysed 71 articles on the topic and derived the following challenges: i) lack of time for upfront activities, due to the nature of agile development to encourage responsiveness to changes instead of upfront planning, ii) conflicts in prioritising UCD and development activities, given the different views on what constitutes progress, iii) negative work dynamics arising from potentially competing goals and different communication practices, iv) difficulty in organising usability testing and incorporating feedback, due to the time restrictions in agile, v) lack of documentation, which creates confusion to UCD practitioners, who are used to record the trace of the design and rationale.
These reviews, and the works they are based on, provide an insightful perspective on the practices and challenges surrounding human-centered approaches in agile development. However, when vulnerable settings are involved, the challenges become amplified because, as we will see, the timing of access to users, the kind of users we can involve, and the nature and number of iterations is subject to constraints, both self-imposed and imposed by the environment.
2.2. Engineering for vulnerable populations
Efforts have been made in incorporating human-centered approaches in sensitive contexts, and especially in the development of healthcare systems. Carroll and Richardson (Carroll and Richardson, 2016) make a case for the lack of a established framework to guide software developers in identifying requirements in healthcare, and propose integrating design thinking as an entire pre-requirements phase. Another interesting take by Texeira et. al (Teixeira et al., 2011) combines more traditional system analysis techniques with UCD and PD, which required a facilitator to translate requirements back and forth between stakeholders. The scenario addressed by both works is certainly sensitive, though no actual emphasis is given in dealing with vulnerable populations.
In a similar setting, Kieffer et al. (Kieffer et al., 2017) applied agile methods in combination to formative usability to the development of an application for patients with diabetes. In reporting the challenges, the authors mention i) the access to users in the medical context, ii) the recruitment process that took about six months in total, iii) the time to get the study protocol validated by an ethical committee, which was four months. On these challenges, the authors reflect that the medical expert should have been involved much earlier in the process. These insights give us a dimension of the practical difficulties in involving vulnerable populations.
Knowledge transfer and communication among multidisciplinary teams is another topic investigated. Weber and Price (Weber and Price, 2016) propose a knowledge transfer model between clinicians and software engineers to facilitate the development of healthcare systems. The model is comprised of a knowledge tailoring loop with three main phases: i) monitoring and evaluation of software, in order to collect observation in ”real settings” right from the start, ii) identification of problems, involving a qualitative understanding of previous results, and iii) adaptation and tailoring of software, which involves design sessions with lead users and synthesis of results in multidisciplinary teams. While valid in the setting described, using software upfront to gain insights might produce undesired effects, such as the loss of interest by users and stakeholders to continue the collaboration, which is why we believe its applicability with vulnerable populations can be considered limited.
A more extreme case of team communication was studied by Leonardi et al. (Leonardi et al., 2011), reporting on the experience of team members with background in HCD and semi-formal requirement engineering. The challenge was framed as an inter-cultural dialogue between professionals from different disciplines. The authors stress the importance of mutual learning, especially via the definition of a shared dictionary to bridge the gap between the disciplines.
Speedplay (Ferrario et al., 2014) is a software project management framework that integrates action research, participatory design and agile development, to approach relatively small projects targeting specific community needs. It is particularly targeted at multi-disciplinary projects seeking social innovation, where the community, researchers and engineers work actively together. The process model is comprised of four main steps: prepare, design, build and sustain, and it is characterised by slower cycles at the beginning followed by faster paced cycles by the end of the project. The model also promotes mutual learning while assigning responsibilities based on skills. An application of Speedplay is presented by Simm et al. (Simm et al., 2014) in the context of a tool for anxiety management for adults with high functioning autism. In working with this vulnerable population, the authors mention participants reacting poorly to changes and fluctuating participation as some of the challenges to an agile research and development.
While we found many of the guidelines from the literature quite useful and we recognised the same challenges, in our experience we have stumbled upon difficulties and derived insights that we have not seen discussed deeply and uncovered several aspects that have not emerged yet. We discuss them in the following.
3. The case of residential care
We describe our findings based on a joint university-industry project aiming at designing a set of innovative solutions for the residential care scenario focused at increasing the emotional well-being of residents, staff, and family members and at facilitating interactions. We also build on previous experience (Fiore et al., 2017) studying analogous issues in the pediatric palliative care case. In this section we describe the context and give an overview of the project setup.
Transitioning to long-term residential care is one of the more difficult moments in the life of an older adult and their family(Lee et al., 2002). This is complicated by the perceived negative social view on residential care that sees institutionalisation as a failure by family members (Ryan and Scullion, 2000), due to cultural stereotypes about care systems, resulting in a sense of guilt, loss and abandonment in the family, as well as a challenging work environment for care professionals.
The relationship between family members and professional caregivers is not without tensions. From the family members side, communication is challenged by a perceived lack of meaningful, timely and understandable information (Fiore et al., 2017). Professional caregivers instead consider ”dealing” with family members as part of the job, but can see communications as potentially problematic. This can lead to professionals avoiding family members and vague communications (Hertzberg et al., 2003). Another critical point is the collaboration between these actors. Though collaboration from family members is in principle welcomed by professionals, this is not always translated into practice as traditional care models are not designed for full collaboration (Haesler et al., 2004). Family members can also be overly demanding, having unrealistic expectations about what the professionals have to do, and even taking issue in care practices (Vinton et al., 1998). Finally, as we experienced, NH staff members work at full capacity and in rather stressful conditions, both in terms of helping residents and in terms of managing family members demands and expectations. This is the scenario we were set to work with: very frail participants, emotionally demanding, and possibly conflictive.
We followed a mix of agile and human-centered approaches, iterating on the following three main phases: product discovery, development and validation. Agile methodologies do not address the product discovery phase (Brhel et al., 2015) but the need for a dedicated phase is evident in human-centered approaches and the software processes that incorporate them (e.g. (Ferrario et al., 2014)).
As a result of the process we identified and developed three IT-based solutions: i) a reminiscence-based tool to stimulate social interactions between family members and residents, ii) a personalised magazine to build a sense of community and stimulate conversations, and iii) a communication and collaboration tool to facilitate information sharing and family involvement. Describing the tools its outside the scope of this paper, and the interested reader is referred to (Fiore et al., 2017; Ibarra et al., 2017; Caforio et al., 2017).
4. Issues and Recommendations
We now list challenges we had to face due to working with vulnerable subjects and our recommendations.
Iterations are limited, errors are costly. Agile processes allow us to iterate often with users and to correct course of actions as needed. This is often done by pushing ”software probes” at early phases (e.g., (Weber and Price, 2016)). In our scenario, we found that the number of possible design iterations are limited: There is a relatively small number of institutions willing at the start to go through an adventure with you, the personnel is often under stress, residents and family members face loads of challenges. Even those who are enthusiastic at the beginning can become less cooperative (or simply less available) as time goes by. Furthermore, while learning from errors is a positive aspect in agile methods, continuous changes can be disruptive to a population not used to change (Simm et al., 2014) and can harm their interest to be involved and participate. More importantly, errors in dealing with vulnerable populations can damage the trust, and that is something very difficult to recover from (Mara et al., 2013). Here an ”error” can be simply giving a hint of suspicion that the system you are building goes in a direction that does not fit the needs of the individual you are speaking with, even if that is not the case. Once this happens, even reassurances that their feedback will be taken into account might not have the hoped effect.
Something we found useful and highly recommend here is to observe users and analyse the relation between stakeholders and technology through the lenses of appropriation (Dourish, 2003), which is often revealing desired technology features that are satisfied via other means today. Furthermore, we recommend spending more time at the start assessing assess to participants, and specifically identifying participants for which we can have an ”agile” style of access and interactions, and participants for which i) we have reduced access and ii) errors are sensitive. For the latter categories, we recommend deeper studies before introducing software probes or mockups.
The need for going through ethical approval processes impacts the process in many ways (including positive ones). Obtaining an approval requires time, both to write the necessary documentation and to go through the approval process, which may involve one or more entities. In our case we went through both University and NH committees. Timescales here are typically of 1-3 months, depending on frequency of committee meetings and on whether clarifications are requested. This timescale is already beyond any modern agile standard. However the process is also an opportunity to carefully think and design the study protocol and receive suggestions. Since we have to be very conservative with iterations and user access, this step is actually helpful especially for a team with an agile mentality which might be tempted to reduce planning and have a bias towards action. A related aspect is the need to re-assess ethical considerations in an ”agile” settings. This not only requires continuous adjustments in the research practice (Rashid et al., 2015), but can also result in further changes to initial study protocols.
Therefore, our recommendation here is to include in the approval process a structured plan of actions, carefully considering and anticipating possible outcomes and designing subsequent study steps accordingly, as opposed to iterating in an agile way. This is both to solicit more informed feedback, but also to limit further requests to the ethical committee to hopefully minor modifications to a plan (if at all), which typically result in faster feedback.
Related to appropriation and ethics is the issue of workarounds that people (and specifically staff members) take, very often with the intent to help people in need. We found this to happen in all scenarios involving vulnerable subjects, beyond the case of institutionalised older adults. Observing workarounds is very useful in design, but in sensitive settings such as the one investigated, the ”creativity”, ingenuity, or simply the commitment and dedication of participants might be perceived as deviations from procedures and sometimes even from the law. In this sense, ethical considerations of reporting or incorporating such learnings in the design arise. What we recommend here is to ensure that the team is aware and follows recommended ethical guidelines and practices for these cases (Mara et al., 2013). This is something that even people trained in user studies may not be familiar with, and an error here may cost people their job.
Participant involvement. From the perspective of participant involvement (Radermacher, 2006), our approach qualifies as research-initiated, with shared decisions with the industry partner, and nursing homes actors being consulted and informed. This puts us halfway between traditional and more participatory approaches (Ferrario et al., 2014). Incorporating a human-centered approach proved to be successful in identifying needs and materialising technology concepts, but limited in testing more forward thinking features. In a scenario where technology should also follow regulations and (sometimes anticipate) changes in policies, following a full community-driven or human-centered approach is not always feasible (Norman, 2005). Thus, rather than taking a decision a priori, we recommend to base the decision on the level of involvement according to the design goals, potential bias, conflictive views, and ethical considerations.
Personas are designed to evoke emotional responses, creating empathy and keeping the team focused on the target users. In our work we had two challenges related to personas: one is access. Even if it is rather easy to identify personas and in general to cluster users into characteristics and needs, access to personas in different groups is hard to organise for many reasons. The other is that sometimes people had strong feelings about the party they interact, which can result in colorful or emotionally provoking personas. We therefore recommend i) to tone down description of personas when discussing with end users and ii) carefully assess at the start of the project (or after the personas have been identified) the access to different personas and prioritise requests: do not expect to manage to have access to all personas.
Varying feedback. Response bias is a widely studied behavior. With vulnerable subjects, we found the problem to be exacerbated. It manifested itself particularly when staff members, in high level discussions, reported an open attitude towards technology supporting staff-FM interactions (in accordance with the management and the political decision makers). However, when we drilled down into understanding how the communication should take place in very specific scenarios, we observed a certain resistance in by staff members related to some potential features. This occurred despite the research team being competent and trained in how to run studies with vulnerable subjects. In retrospect, drilling down to details earlier would have avoided us to work on features that are unlikely to make it in the system. We recommend therefore to identify features that might be contested and to drill down on them early, by providing concrete and realistic examples to validate acceptance.
Multidisciplinary teams are essential but difficult to manage. Our core team was comprised of a multidisciplinary group of: i) sociologists with background in participatory action research and qualitative research methods, ii) researchers with background in software engineering and human-computer interaction, iii) product managers with experience in the healthcare sector, and iv) cognitive scientists and psychologists with competence on interactions and stress. Interactions with vulnerable populations require empathy, soft skills, experiences in designing studies in a way that is mindful of biases, and the ability to avoid putting the participants in an uncomfortable situation. This is hardly something that can be learned with a crash course. Nonetheless, involving software engineers in informal visits – and user studies when possible – has proven useful in our experience in creating empathy and having a more realistic view of the context, which was later useful when incorporating and discussing the lessons learned.
The communication and collaboration in the team was facilitated by the cross-functional team members with experience in software engineering and human factors. This setup is known to facilitate the integration of design and development (Brhel et al., 2015), minimising the need for resorting to, for example, translations between team members (Leonardi et al., 2011). This was further facilitated by the use of scenarios, personas and mockups that were concrete materialisations of lessons learned. However, coordinating efforts towards activities that would maximise requirement elicitation has been more challenging. We see this as a consequence of competing views on what qualifies as useful insights among sociologists and engineers. Cross-functional members were fundamental in mediating these differences. Their importance cannot be overestimated and we feel that the lack of such competences can jeopardise the design effort.
In summary, what we take home is the need to mix agile approaches with waterfall concepts. With some participants we can follow and iterate with short-lived design or development sprints, while with vulnerable populations we execute much longer sprints, characterised by a thorough design process that anticipates possible alternatives as opposed to designing them iteratively. Participants involvement needs the same flexibility: with some users observation only is appropriate, with others we can leverage PD, with others again we can follow traditional user research. We find that the challenge of design lies therefore not in the choice of a specific process and model for the project, but in identifying which participants and which tasks are suited for a given process and design approach.
Acknowledgements.This project has received funding from the EU Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 690962, for the studies run outside the EU. This work was also supported by the “Collegamenti” project funded by the Province of Trento (l.p. n.6-December 13rd 1999), for the studies run in Italy.
- Bordin and De Angeli (2016) Silvia Bordin and Antonella De Angeli. 2016. Focal Points for a More User-Centred Agile Development. Springer International Publishing, Cham, 3–15.
- Brhel et al. (2015) Manuel Brhel, Hendrik Meth, Alexander Maedche, and Karl Werder. 2015. Exploring principles of user-centered agile software development: A literature review. Information and Software Technology 61 (2015), 163–181.
- Brown (2009) Tim Brown. 2009. Change by design. (2009).
- Caforio et al. (2017) Valentina Caforio, Marcos Baez, and Fabio Casati. 2017. Viability of Magazines for Stimulating Social Interactions in Nursing Homes. In IFIP Conference on Human-Computer Interaction. Springer, 72–81.
- Carroll and Richardson (2016) Noel Carroll and Ita Richardson. 2016. Aligning healthcare innovation and software requirements through design thinking. In SEHS, IEEE/ACM. IEEE, 1–7.
- Da Silva et al. (2011) Tiago Silva Da Silva, Angela Martin, Frank Maurer, and Milene Silveira. 2011. User-centered design and agile methods: a systematic review. In Agile Conference (AGILE), 2011. IEEE, 77–86.
- DIS (2009) ISO DIS. 2009. 9241-210: 2010. Ergonomics of human system interaction-Part 210: Human-centred design for interactive systems. ISO. Switzerland (2009).
- Dourish (2003) Paul Dourish. 2003. The appropriation of interactive technologies: Some lessons from placeless documents. CSCW 12, 4 (2003), 465–490.
- Ferrario et al. (2016) Maria Angela Ferrario, Will Simm, Stephen Forshaw, Adrian Gradinar, Marcia Tavares Smith, and Ian Smith. 2016. Values-first SE: research principles in practice. In ICSE-C, IEEE/ACM. IEEE, 553–562.
- Ferrario et al. (2014) Maria Angela Ferrario, Will Simm, Peter Newman, Stephen Forshaw, and Jon Whittle. 2014. Software engineering for’social good’: integrating action research, participatory design, and agile development. In ICSE 2014. ACM, 520–523.
- Fiore et al. (2017) Angela di Fiore, Francesco Ceschel, Francesca Fiore, Marcos Baez, Fabio Casati, and Giampaolo Armellin. 2017. Understanding how software can support the needs of family caregivers for patients with severe conditions. In Proceedings of the 39th International Conference on Software Engineering. IEEE Press, 33–36.
- for the Protection of Human Subjects of Biomedical and Research (1978) National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. 1978. The Belmont Report.
- Haesler et al. (2004) Emily Haesler, Michael Bauer, and Rhonda Nay. 2004. Constructive staff-family relationships in the care of older adults in the institutional setting: A systematic review. JBI Database of S. Reviews and Implementation Reports 2, 10 (2004), 1–76.
- Hertzberg et al. (2003) Annika Hertzberg, Sirkka-Liisa Ekman, and Karin Axelsson. 2003. ‘Relatives are a resource, but…’: registered nurses’ views and experiences of relatives of residents in nursing homes. Journal of clinical nursing 12, 3 (2003), 431–441.
- Ibarra et al. (2017) Francisco Ibarra, Marcos Baez, Francesca Fiore, and Fabio Casati. 2017. Stimulating Conversations in Residential Care Through Technology-Mediated Reminiscence. In IFIP Conference on Human-Computer Interaction. Springer, 62–71.
- Kieffer et al. (2017) Suzanne Kieffer, Aissa Ghouti, and Benoit Macq. 2017. The Agile UX Development Lifecycle: Combining Formative Usability and Agile Methods. (2017).
- Lee et al. (2002) Diana TF Lee, Jean Woo, and Ann E Mackenzie. 2002. A review of older people’s experiences with residential care placement. JAN 37, 1 (2002), 19–27.
- Leonardi et al. (2011) Chiara Leonardi, Luca Sabatucci, Angelo Susi, and Massimo Zancanaro. 2011. Design as intercultural dialogue: coupling human-centered design with requirement engineering methods. In INTERACT 2011. Springer, 485–502.
- Mara et al. (2013) Andrew Flood Mara, Liza Potts, and Gerianne Bartocci. 2013. The ethics of agile ethnography. In Proc. conference on Design of communication. ACM, 101–106.
- Norman (2005) Donald A Norman. 2005. Human-centered design considered harmful. interactions 12, 4 (2005), 14–19.
- Radermacher (2006) Harriet L Radermacher. 2006. Participatory action research with people with disabilities: Exploring experiences of participation. Ph.D. Dissertation.
- Rashid et al. (2015) Awais Rashid, Karenza Moore, Corinne May-Chahal, and Ruzanna Chitchyan. 2015. Managing emergent ethical concerns for software engineering in society. In ICSE 2015, Vol. 2. IEEE, 523–526.
- Ruof (2004) Mary C Ruof. 2004. Vulnerability, vulnerable populations, and policy. Kennedy Institute of Ethics Journal 14, 4 (2004), 411–425.
- Ryan and Scullion (2000) Assumpta Ann Ryan and Hugh F Scullion. 2000. Nursing home placement: an exploration of the experiences of family carers. Journal of advanced nursing 32, 5 (2000), 1187–1195.
- Salah et al. (2014) Dina Salah, Richard F Paige, and Paul Cairns. 2014. A systematic literature review for agile development processes and user centred design integration. In 18th Intl. conference on evaluation and assessment in software engineering. ACM, 5.
- Schön et al. (2017) Eva-Maria Schön, Jörg Thomaschewski, and María José Escalona. 2017. Agile requirements engineering: a systematic literature review. Computer Standards & Interfaces 49 (2017), 79–91.
- Simm et al. (2014) Will Simm, Maria Angela Ferrario, Adrian Gradinar, and Jon Whittle. 2014. Prototyping’clasp’: implications for designing digital technology for and with adults with autism. In Proceedings of DIS 2014. ACM, 345–354.
- Spinuzzi (2005) Clay Spinuzzi. 2005. The methodology of participatory design. Technical communication 52, 2 (2005), 163–174.
- Teixeira et al. (2011) Leonor Teixeira, Vasco Saavedra, Carlos Ferreira, and Beatriz Sousa Santos. 2011. Using participatory design in a health information system. In EMBC, 2011 Annual International Conference of the IEEE. IEEE, 5339–5342.
- Vinton et al. (1998) Linda Vinton, Nicholas Mazza, and Yu-Soon Kim. 1998. Intervening in family-staff conflicts in nursing homes. Clinical Gerontologist 19, 3 (1998), 45–68.
- Weber and Price (2016) Jens H Weber and Morgan Price. 2016. Closing the gap: enacting knowledge transfer between engineering and use of healthcare software. In Proc. of the Intl. Workshop on Software Engineering in Healthcare Systems. ACM, 19–25.