Critical Requirements Engineering in Practice

10/03/2019 ∙ by Leticia Duboc, et al. ∙ UNIVERSITY OF TORONTO U. Ramon Llull 0

The design of software systems inevitably enacts normative boundaries around the site of intervention. These boundaries are, in part, a reflection of the values, ethics, power, and politics of the situation and the process of design itself. This paper argues that Requirements Engineering (RE) require more robust frameworks and techniques to navigate the values implicit in systems design work. To this end, we present the findings from a case of action research where we employed Critical Systems Heuristics (CSH), a framework from Critical Systems Thinking (CST) during requirements gathering for Homesound, a system to safeguard elderly people living alone while protecting their autonomy. We use categories from CSH to inform expert interviews and reflection, showing how CSH can be simply combined with RE techniques (such as the Volere template) to explore and reveal the value-judgements underlying requirements.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Today’s societies run on software. Software Engineering (SE) is rightfully expected to address social and ethical concerns of software systems in society. Software systems have enormous potential for human improvement, but are implicated in all sorts of maladies. From radicalizing voters to the erosion of privacy, from mechanisms to support emission test cheating to the energy impact of bitcoin mining, software is never a neutral element of society [7] [14]. This raises serious ethical issues, including responsibility and power relationships in systems design, the politics of stakeholder engagement, and the role of human and social values in engineering [8]. Within SE, Requirements Engineering (RE) is arguably the key leverage point for social change and sustainability [1]. By focusing on whom to involve as stakeholders, how to elicit their perspectives, how to consider these in architectural design choices, and how to define acceptance criteria, RE frames the design scope and establishes the success conditions for systems development. It is no surprise then that it is increasingly called upon to address social and ethical concerns of software systems in society [16]. In practice, RE and SE professionals must engage a broad range of stakeholders, facilitate the emergence of a shared understanding of what the systems development effort should achieve, and represent the outcomes of that understanding. In this process, they carry an ethical and moral responsibility to all those affected, and in particular, to those affected but not involved in systems design. This means that they must engage with the normative frameworks and rules of ethics; pay attention to how human values – what people regard as important – constitute the ‘facts of the future’[8]; understand how their own work is subject to social relationships of stakeholders (politics); and be sensitive to issues of power: who it is held by, how it is wielded and in which form it influences choices and technological trajectories. This is challenging, because software systems are fully intertwined with physical and natural environments, economic processes, and with social and cultural life [12]. However, as many Requirements Engineering frameworks are ultimately grounded in the scientific method, they lack the concepts to address politics, morality, aesthetics, and beliefs [4, 21]. This makes it so difficult to move between what Goguen called the “wet” and the “dry” – between rich human and social worlds and the formalized technical models and methods used in software engineering [9].

To address the social construction of requirements, RE often suggests the use of interpretive systems thinking frameworks such as Soft Systems Methodology, which focuses on facilitating the emergence of shared understanding [3]. But these frameworks are not able to address the marginalization that inevitably arises out of power dynamics [11]. The need to make visible and reflect on such concerns [21] led to the development of Critical Systems Thinking (CST) frameworks, but only a few have considered them in RE [22] [6]. In part, this can be attributed to their focus on philosophical theory, social critique, and epistemology.

As a result, practitioners can feel rather helpless. Even if they have the best intentions, how are they to “rationally justify the normative implications of systems design”, as Ulrich framed it [21]

? In other words, how can they justify their work, their design decisions, their actions, when it is not considered feasible to estimate or predict possible effects spread in time and space; when they have no foundational education in social sciences, policy, or ethics; when they are embedded in industry projects with tight time lines, profit expectations and dispersed networks of potential stakeholders?

Fig. 1: Overview of CSH and Key Concerns, adapted from [20].

In this article, we describe a collaborative Action Research project[2] that demonstrates how Critical Systems Heuristics (CSH), the best known CST framework (see sidebar II), can be used in the context of RE to gain a critical awareness of power and politics; to support critical reflection on the human values and boundary judgments that guide and frame a project; and to support requirements professionals in understanding and questioning the selectivity of their choices. Intertwining CSH with standard RE practice helped us to make visible the risk of marginalization faced by the beneficiaries in a software-driven technology development project, and to adjust early requirements activities to more fairly represent the concerns of those at risk of marginalization. As a result, the project created two types of artifacts: A requirements document following the often-used Volere template [15], and a set of what CSH calls ‘ideal maps’– accounts of the values, knowledge, politics and perspectives forming the basis of the requirements statements. Fig. 1 summarizes the kinds of issues normally addressed by these two frameworks as well as the CSH questions guiding the creation of these maps. Our findings show that CSH is invaluable in supporting RE. CSH lacks the content and structure of RE, while RE practice lacks an approach to critical reflection on the values driving systems development. When combined effectively, the result is a critically appreciative RE practice that can be adopted by software professionals.

Ii Combining RE and CSH in software system development

Ii-a The problem situation

The world’s population is aging. In Spain, where this project takes place, 34,6% of the population is expected to be above 65 years by 2066. Many will live alone and may require support. Clues that they need help can be subtle, and may pass unnoticed by their families and caretakers. For example, a change in routine could indicate the beginning of certain diseases like dementia[10]. These clues would either have to be observed by visitors, who are only present for a small portion of the day, or communicated by the elderly people themselves, who may not have the cognitive capacity to notice the cues or realize when they indicate a problem.

Ii-B The Project

The HomeSound project focuses on a vulnerable population of fragile elderly people living at home. It aims to design a Wireless Acoustic Sensor Network and algorithms to support their lives by detecting unusual sounds that might indicate possible changes in routine, accidents, or mishaps (see sidebar III). The project is in the early stages of RE, focused on exploring the potential for technology transfer.

Ideal Map Description
1 Ideal map 1 was created by the requirements engineer and one of the CSH experts, as the latter walked the former through the CSH process. It contained the requirements engineer’s observations from their previous discussions with the HomeSound development team, third-sector organizations, and the families and caretakers of elderly people. Among other things, the map identified the elderly as the main beneficiary, and the stated purpose as increasing their independence, to allow them to stay home longer. It also set the elderly’s ‘independence and well-being levels’ and the ‘number of years living alone at home’ as measures of improvement. We also assumed that if their well-being increased, this would reduce the ‘number of distressed calls from elderly people’ to their caretakers, which could therefore be taken as a possible guarantor of success. Reflection revealed, among other things, that this first map was built from the selective memory of the requirement engineer and most likely represented the privileged views of those developing the system. We therefore decided to consult the notes of previous conversations with stakeholders to more accurately reflect their needs. We also felt the need to involve a third researcher on the critique of the maps, and to help reconcile and synthesize the dialogue in lieu of more thorough participant involvement.
2 Ideal map 2 was created after the consultation of the notes from previous meetings and interviews with stakeholders. The new map added the families as a primary beneficiary and the society and the healthcare system as secondary beneficiaries, extending the boundaries of the system. The purpose now included peace of mind for families. Ideal map 2 also reviewed the concept of well-being and independence, among other things. Reflection on the validity of our measurements of success made us realize that we needed to get input from professionals with expertise relevant to the situations of elderly people. For Ulrich, relying on incomplete or dogmatic perspectives is a major ‘source of deception’ and can be a false guarantor that harms our understanding of the situation and our systems design [21]. Reflection also raised a number of questions, including: What are the boundaries of the system? Is ‘years at home’ a suitable measure of well-being? What about uninvolved family members? Should the purpose be to increase independence or rather self-determination? Do people really understand the implications of the technology? Why have we observed a blind trust in technology in our conversations with stakeholders? What if the elderly cannot be the decision makers? etc. This last question in particular raises important issues of fairness in representing the concerns of those at risk of marginalization.
3 Ideal map 3 integrated the views of a social worker and a psychologist, both specialized in the isues of the elderly. The interviews highlighted several issues, including that the system did not increase independence, but rather security (as it cannot meet their physical and emotional needs) and that the number of calls from the elderly were a false guarantor of success. We also learned about common behaviors, coping mechanisms, the importance of a trusted person, and several scales for measuring well-being of elderly people and their caretakers. Ideal map 3 included new measures of success (e.g. social support, anxiety of the caretaker, early signs of dementia) and professional scales used in psychology and social care to measure them, and an extended list of decision makers and sources of knowledge. Reflection raised doubts regarding the measurement of self-determination and early signs of dementia.
4 Ideal map 4 was developed after an interview with a practical philosopher specialised on the impact of technological projects on ethics and privacy. We discussed issues such as: How do you frame care and well-being? What is the amount of trust required from the user? Can over-reliance of families on this technology lead to a loss of ‘human touch’ and thus reduce well-being rather than support it? Can the technology reduce the autonomy of the elderly, who should have the right to decide when to get help? Will third parties be interested in this data? etc. Each of these questions bring power imbalance and the politics of stakeholders to the forefront of RE activities. Finally, we recognized the importance of public debate on such technologies, and we identified techniques from practical philosophy for uncovering stakeholder ethics, morals and values. The new map included autonomy as a primary aim, a better definition of self-determination, the general public as a desired expert, institutions that are interested in data as a commodity as an undesired expert, and possible worldviews about being old, supporting the elderly, living the good life, and surveillance technology. Reflection led us to recognize we had been more concerned with people’s perception than with security, assuming this was something more easily solvable. Hence, we chose to interview a security expert.
5 Ideal map 5 integrated the opinions of an IT security specialist who performs security audits of technological solutions. They highlighted that sound monitoring may not be the best solution to the problem due to the intrinsic risks of having microphones in the house. We learned about the different privacy and security risks of these devices, and were recommended a security audit when the system is under development. The new map included the desired skills that a security expert could bring to the project, a new guarantor of success (the number of vulnerabilities discovered in recurrent security audits), and several risks and possible measures. Reflection led us to a discussion on the role of risk assessment in CSH and whether it could lead to overlooking the important issue of moral justification for the project. We also reflected on how the insights from CSH were different from our previous experiences with RE techniques, and on how we could integrate these insights into RE. This led us to attempt to map the CSH findings to the Volere framework.

Ideal map 6 incorporated the insights derived from translating the findings of the CSH to the Volere specification. These included classifying previous secondary aims as

unrealistic aims (as the technical system could only address specific aspects of this larger aim) and improving rationale, relationship, and consistency for items across different sections of the ideal map. Reflection showed us that there is value in iterating between standard RE and CSH, as discussed in Section III.
TABLE I: Iterations of the ideal map

Ii-C The Process

In this Action Research, we have combined standard RE practice with CSH through iterative cycles of critical RE practice followed by reflection. The research team was composed of a researcher at La Salle and two from the University of Toronto. The former had been involved in conversations with third-sector companies and other stakeholders and has close contact with the technical and business developers of HomeSound, but had no previous knowledge of CSH. The other two were knowledgeable in CSH. They took a mentoring role and helped to critically reflect on the models created by the first researcher. We will refer to these as requirements engineer and CSH experts, respectively. To make the activities and findings recoverable[2] within the space constraints of the article, we provide a high-level overview of the iterations and have made the template materials available (see below).

The requirements engineer created several versions of the ideal map for the HomeSound project, guided by the questions of Table I. As in any iterative process, ideal maps are incomplete. The first two maps, for example, reflect the privileged views of those building the systems. These views are slowly extended to incorporate the viewpoint of different stakeholders and the results of the team’s critical reflection on that version of the map. Table I summarizes this process, highlighting in italics the topics that refer to the CSH questions (see Fig. 1), and in bold the kind of awareness commonly brought up by CSH111The CSH process still continues, now to integrate the views of elderly people. But at the time of writing no new version of the ideal map had been generated from the later inputs..

Iii Discussion and Conclusion

Requirements engineering must address issues of power, politics and human social values. Critical Systems Thinking frameworks have been developed to make visible and reflect on such concerns, but have not been taken up in RE. This article asked: What is the role of Critical Systems Heuristics in Requirements Engineering?

The process described in Table I illustrates the challenges that the engineering perspectives of SE face. The systems that matter when software is developed are inevitably socio-technical systems. Abdicating the responsibility to account for that is not an ethical option, but it is difficult to do justice to the implications that result. As others have argued, RE is in a unique position to address the social responsibility that derives from software systems’ central role in society [1, 16]. Living up to that role requires integrating social theory and critical perspectives into the core of what RE does, and how it accounts for what it does.

In this article, we showed how Critical Systems Heuristics can be used for structuring early explorations of requirements in a project designing an embedded device that supports elderly people at home. We proceeded in iterative progressions through consecutive versions of a map of critical categories, using interviews, reflection, and internal critique. The role of the CSH framework was to provide an effective framing for developing a reflexive understanding of stakeholders and concerns; revising high-level purposes, goals and measures of success to represent the interests of those affected; probing from multiple perspectives how different project and system scopes can and should be justified; and exploring how human, social and economic values should drive the project. Some of the issues raised during the process were highlighted in bold and italics in Table I. For example, we learned that, if not carefully designed, the system could reduce the autonomy of the elderly– the very opposite of our intended purpose. We also revisited our initial understanding of well-being and elderly support, leading us to balance “safety” and “self-determination”. These issues had not been challenged by the privileged view of the system’s developers prior to this exercise.

This type of critical reflection complements recent work on values and politics in RE that explicitly considered human values in the RE process [19], and modelled stakeholder interactions [13] in a way similar to the political analyses of Soft Systems Methodology [3]. Rather than focus on a descriptive analysis of power and politics in systems design, as do Milne and Maiden [13], CST and CSH commit practitioners to revealing, and even avoiding, situations where power can marginalize perspectives of presumed beneficiaries. Following this, we focus on discursive acts, and supporting critical reflection that makes visible the implicit boundary judgments of all involved. In attitude, this is closer to critical design [5] and similar approaches in HCI. But because CSH, despite its specific language and terminology, is a small-scale heuristic framework, it is highly suitable for being adopted quickly and without extensive study of social theory.

Beyond a requirements specification, using CSH in the project also created ideal maps. Translating the issues captured by the ideal maps to the Volere specification forced us to think in more concrete terms about the system. We realized, for example, that the project’s secondary purposes of “Better serving the society by allowing elderly people to stay longer at their own home” and “Reducing the financial burden of the social security system” really fall outside of a reasonable and justifiable design scope, being in fact an unrealistic aim, though the technical contribution can and should address specific aspects of this larger aim.

Although we can easily map the CSH questions to the Volere template, the type of issues that the process revealed are of a very different nature from those typically found in requirements documents, including educational sample specifications such as the Volere package. The latter are more focused on the system features than the values of the stakeholders involved and affected. While we are not claiming that the analysis of real system is a fair comparison to a sample specification, it does makes us wonder if the RE community should deliberately create didactic materials that highlight issues of ethics, power, politics, and human and social values.

Finally, creating the CSH ideal map and the Volere specification enabled us to incorporate the critical reflection that RE frameworks on their own do not provide. In order to facilitate the integration of CSH and RE, we created an ideal map template complementing Ulrich´s questions, a mapping from this template to the Volere template, and an annotated version of the latter.222Refer to: The latter cannot be made available due to copyright issues.

RE is a key leverage point for addressing social and ethical concerns of software systems in society. We have demonstrated the value of Critical Systems Heuristics in early-phase RE. While this cannot guarantee ethical and fair software systems design - nothing can - it provides a crucial stepping stone for the different kind of SE that is called for in the 21st century.

Critical Systems Heuristics (CSH) is a methodology and approach to decision-making and evaluation within systems design. CSH emphasizes that the boundaries constituting a system are not objective necessities, but enacted through decisions. Claims about scope, measurement, or stakeholders are normative, value-laden claims that must be not (just) optimal, but also legitimate. At stake in any situation are the conditions by which these decisions are made. Beliefs in the objectivity of expertise or the exigencies of powerful stakeholders can lead to the marginalization of lay voices in decision-making. CSH is an ethical commitment to participatory design, and to creating conditions where those affected by a system can be involved in planning it. When experts and lay-participants interact, there are asymmetries in knowledge and power, but the burden of explanation lies with the experts involved. The situated knowledge and values of those affected are in-themselves qualification to speak freely and critically about the assumptions and judgments of those with power in decision making and design. CSH focuses on revealing the values underlying design by iteratively applying 12 questions (see Fig. 1) covering dimensions of motivation, control, knowledge, and legitimization. Iterations alternate between descriptive (‘is’) and ideal (‘ought to be’) modes. The goals of CSH are to make the values shaping the scope of the system visible; to allow those involved in design to reflect on beliefs supporting purpose, practise and improvement; and to create a space where those affected by design and implementation are on equal footing with the knowledge of experts.
TABLE II: SIDE BAR: Critical System Heuristics.
Acoustic Event Detection automatically identifies events of interest from within continuous audio streams [18]. This technology has several applications, like home security (e.g. CO2 alarm, broken window), people and pet care (e.g. routine change, barking), transportation (e.g. traffic monitoring), entertainment (e.g. adapt sound spectrum based on surrounding noise), wellness (e.g. baby crying, snoring), social (e.g. pause music when having conversation). The SmartSound [17]

is an Anomalous Noise Event (ANE) Detector designed to work in real-time on low-cost acoustic sensors of a Wireless Acoustic Sensor Network (WASN). It uses Gaussian Mixture Modelling to distinguish anomalous noise events from background noise. Its most mature application is in the real-time detection and representation of the acoustic impact of road infrastructure. It has also been used to monitor endangered bird species in a national park, and on an Ambient Assisted Living platform to assist medical staff to track the status of patients in real-time.

TABLE III: SIDE BAR: Acoustic Event Detection.


The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 712949 (TECNIOspringPLUS), from the Agency for Business Competitiveness of the Government of Catalonia, and from NSERC under RGPIN-2016-06640.


  • [1] C. Becker, S. Betz, R. Chitchyan, L. Duboc, S. M. Easterbrook, B. Penzenstadler, N. Seyff, and C. C. Venters (2016-01) Requirements: The Key to Sustainability. IEEE Software 33 (1), pp. 56–65. External Links: ISSN 0740-7459, Document Cited by: §I, §III.
  • [2] P. Checkland and S. Holwell (1998) Action research: its nature and validity. Systemic practice and action research 11 (1), pp. 9–21. Cited by: §I, §II-C.
  • [3] P. Checkland (1999) Systems Thinking, Systems Practice: Includes a 30-Year Retrospective. Wiley (English). Cited by: §I, §III.
  • [4] C. W. Churchman (1979-05) The systems approach and its enemies. Basic Books (en). External Links: ISBN 978-0-465-08342-8 Cited by: §I.
  • [5] A. Dunne and F. Raby (2013) Speculative everything: design, fiction, and social dreaming. MIT press. Cited by: §III.
  • [6] S. Elsawah, A. McLucas, and M. Ryan (2015) Beyond why to what and how: the use of systems thinking to support problem formulation in systems engineering applications. In 21st International Congress on Modelling and Simulation, Cited by: §I.
  • [7] V. Eubanks (2018) Automating inequality. St. Martin’s Press. Cited by: §I.
  • [8] A. Feenberg (2010) Ten paradoxes of technology. Techné: Research in Philosophy and Technology 14 (1), pp. 3–15. Cited by: §I.
  • [9] J. Goguen (1994) Requirements Engineering as the Reconciliation of Technical and Social Issues. In Requirements Engineering: Social and Technical Issues, pp. 165–199. Cited by: §I.
  • [10] T. L. Hayes, F. Abendroth, A. Adami, M. Pavel, T. A. Zitzelberger, and J. A. Kaye (2008) Unobtrusive assessment of activity patterns associated with mild cognitive impairment. Alzheimer’s & Dementia 4 (6), pp. 395–405. External Links: Document Cited by: §II-A.
  • [11] M. C. Jackson (2007) Systems approaches to management. Springer Science & Business Media. Cited by: §I.
  • [12] M. Jarke, P. Loucopoulos, K. Lyytinen, J. Mylopoulos, and W. Robinson (2011-11) The Brave New World of Design Requirements. Information Systems 36 (7), pp. 992–1008. External Links: ISSN 0306-4379, Link, Document Cited by: §I.
  • [13] A. Milne and N. Maiden (2012) Power and politics in requirements engineering: embracing the dark side?. Requirements Engineering 17 (2), pp. 83–98. Cited by: §III.
  • [14] S. U. Noble (2018) Algorithms of oppression. NYU Press. Cited by: §I.
  • [15] J. Robertson and S. Robertson (2000-01) Volere requirements specification template. pp. . Note: Available in: Cited by: §I.
  • [16] G. Ruhe, M. Nayebi, and C. Ebert (2017-09) The Vision: Requirements Engineering in Society. In 2017 IEEE 25th International Requirements Engineering Conference (RE), pp. 478–479. External Links: Document Cited by: §I, §III.
  • [17] J. Socoró, F. Alías, and R. Alsina-Pagès (2017) An anomalous noise events detector for dynamic road traffic noise mapping in real-life urban and suburban environments. Sensors 17 (10), pp. 2323. Cited by: TABLE III.
  • [18] A. Temko (2007) Acoustic event detection and classification. Ph.D. Thesis, Department of Signal Theory and Communications, Universitat Politecnica de Catalunya, Barcelona, Spain. Cited by: TABLE III.
  • [19] S. Thew and A. Sutcliffe (2018) Value-based requirements engineering: method and experience. Requirements Engineering 23 (4), pp. 443–464. Cited by: §III.
  • [20] W. Ulrich and M. Reynolds (2010) Critical systems heuristics. In Systems approaches to managing change: A practical guide, pp. 243–292. Cited by: Fig. 1.
  • [21] W. Ulrich (1983) Critical heuristics of social planning: a new approach to practical philosophy. J. Wiley and Sons. Cited by: §I, §I, §I, TABLE I.
  • [22] J. W. Wing, T. N. Andrew, and D. Petkov (2015) A systemic framework for improving clients’ understanding of software requirements. In ECIS 2015 Proceedings at AIS Electronic Library (AISeL), Cited by: §I.