Agile software development processes, especially Scrum and Scrum-hybrids (i.e. Scrumban or Scrum/XP), are standard practices in modern software companies. Team meetings employed in these methods facilitate empirical process control, create regularity, and decrease the demand for additional unplanned or unstructured sessions Schwaber2017 ; rubin12 . The literature regarding Scrum provides detailed descriptions of each meeting and what should be accomplished in them Schwaber2017 . However, few prescriptions or best practices on how these goals should be achieved exist. Consequently, teams face several challenges concerning the facilitation and perceived effectiveness of meetings including:
Not all meetings are equally relevant for all participating team members, leading to a decrease in participant engagement Akif2012 .
Planning and review meetings may be perceived as a waste of development time and as too “simple” Cho2008 .
Regular meetings featuring identical structures create monotony which can negatively affect outcome quality kua2013 .
A lack of follow-through regarding meeting outcomes leads to a feeling that “the same things are discussed over and over” Przybyek2017 .
The transparency afforded by Agile approaches can be abused leading to feelings of being exposed or inadequate in team members Conboy2011 .
Knowledge of these common meeting issues allows addressing, resolving, and preventing problems before they become hindrances to team collaboration Cho2008 . Researchers and practitioners have proposed and collected various activities and best practices for different phases and contexts of the Scrum process. Much of these focus on Retrospective Baldauf2018 ; jovanovic2016 ; Ng2020 or Sprint Planning meetings, e.g., the common Planning Poker in Scrum rubin12 ; Haugen2006 , but other activities exist hohmann2006 ; Gray2010 .
Design thinking (DT) has emerged as an approach that can support software development by providing interactive and engaging techniques for collaborative problem analysis and solution development. It can improve creativity, product understanding, collaboration, and empathy towards customers and the team plattner_DT_2020 . Employing DT as a preceding phase to Agile software development is a popular scenario Rafael2019 . However, in line with previous studies Pereira2018 , our research has also shown that the application of DT in later stages of the development cycle is valuable for development teams dobrigkeit_design_2019 . Developers especially value the change in working style and the required customer-centered mindset. As such, DT techniques provide an opportunity to add further useful activities to the Scrum meeting toolbox. They can help address common challenges, achieve meeting requirements for attendees, or aid in reaching meeting outcomes that have the support of all team members. We, therefore, explore how techniques from DT can be applied to support Scrum meetings.
2 Background and Related Work
In this Section, we briefly describe the key concepts of Scrum and DT as well as existing research into their activities and techniques. Additionally, we describe existing approaches to combine Agile Software Development and DT.
Scrum is an Agile process framework for managing work on complex products in short iterations (Sprints), with an emphasis on software development Schwaber2017 . It defines the different roles of a Scrum team, the rules to follow, the artifacts to create and the meetings that steer and organize a Sprint Schwaber2017 ; rubin12 . A Scrum team consists of the Product Owner (PO), the Scrum Master (SM), and the Development Team. The PO is responsible for managing the product vision and the Product Backlog, which contains the required work items needed to achieve the vision. The SM facilitates the Scrum meetings and supports the team in overcoming issues. The development team develops the (software) product. Work happens in an iterative, incremental manner, carried out in cycles called Sprints ScrumPrimer2012 . There are five main Scrum meetings that facilitate team cooperation and collaboration rubin12 :
Backlog Refinement, where work items are prepared
Planning, where the work items for the next Sprint are decided
Daily Scrum, provides a regular team status update
Review, concerned with inspecting the product and collecting feedback
Retrospective, focuses on process improvement
We provide a detailed description of each meeting with its requirements and outcomes in Section 4.
Studies on the subject of the challenges involved in running Scrum meetings have identified multiple issues of relevance. The literature notes that Scrum events are not equally relevant for all participating team members, leading to a decrease in engagement Akif2012 . Similarly, meetings such as Sprint Planning and Review, which were perceived by participants as “simple”, have been described as a waste of valuable development time Cho2008 . If Scrum meetings are repeatedly held in the same fashion and with similar structures, apathy and monotony felt by attendees can negatively affect the quality of meeting outcomes kua2013 . In case studies of multiple teams, Retrospective meetings were judged to be ineffective by participants, as the same issues were repeatedly discussed in subsequent meetings Przybyek2017 . Furthermore, the downsides and drawbacks of the full transparency and visibility provided by Scrum events have been highlighted, such as exposing developer shortcomings publicly. This can prove to be counterproductive, as team members can be made to feel inadequate or can lead to unhealthy environments where this transparency is abused Conboy2011 .
The prescriptive or “dogmatic” nature of Scrum events has been criticized, e.g., regarding the Daily Scrum meeting, which is strictly time-boxed and should ideally always be performed every day at the same time with the entire team. These requirements can break down in real-world circumstances featuring distributed teams, meeting setup times, flexible working schedules, and varying team sizes Meyer14 .
Existing Activities for Scrum Meetings
Various activities and best practices have been proposed in the related literature for different phases and contexts of the Scrum process to address common Scrum meeting challenges. In particular, Scrum process facilitators have introduced tools and meeting agendas that help in achieving the main goals of Scrum meetings, e.g., for the Retrospective (focusing on creative prompts for feedback collection) and Sprint Planning (focusing on collecting effort estimations),
Much previous research is available on Retrospectives, and the activities that teams can employ during these meetings and studies continue to be published Ng2020 . Retrospective meetings have been in use by teams since before Agile methods became widespread. Team activities or “games” that participants play to keep sessions from becoming stale and repetitive have been used in Retrospectives since their inception kerth2000 . These interactive games are designed to encourage the required reflection and team collaboration needed for the meeting. Derby and Larsen describe the purpose of Retrospective activities as helping a team “think together” Derby2006 . Retrospective activities have been shown to positively impact the creativity, involvement, and communication of team members Przybyek2017 .
Sprint Planning and Backlog Refinement meetings are also closely associated with best practices in agile processes. These activities are concerned with the requirement of producing effort estimates for work items. Unstructured group estimation processes pose the risk of being highly influenced by company politics, group pressure, or dominant personalities, which may reduce estimation performance Haugen2006 . To allow teams to come up with shared estimates while reducing bias-related anti-patterns, tools such as Planning Poker were introduced Ralph2013 . Through the use of structured meeting agendas and categories of answers, these activities can help ensure that all team members participate and that their opinions are heard, regardless of group influence Haugen2006 .
Scrum practitioners have recommended different activities for use in Scrum meetings, but few of these have garnered widespread attention and adoption.
2.2 Design Thinking
The term Design Thinking is used ambiguously and can describe a) a cognitive style of working dominant with designers, b) the idea of including design practices into all parts of an organization or c) an approach to creating new and innovative products and services. In this article, we understand DT as an approach to new product and service development, as it is taught at Stanford or the HPI School of Design Thinking Potsdam. In this approach, the design process and methods from design and other disciplines have been remolded in a way that non-designers can apply them to successfully create new products and services. Within this understanding of DT, the DT process is described as an iterative and flexible multi-step process with recommended techniques for each step. While authors do not agree on the number of steps, the steps of different processes can be mapped to a) steps that focus on learning about the problem and the user, b) ideating on a solution, and c) prototyping and testing this solution with users. Detailed descriptions of these processes are available brown_design_2008 ; thoring_understanding_2011 ; wolbling_design_2012 ; brenner_design_2016 . The recommended techniques accompanying each process step, provide practitioners with multiple ways to facilitate each step (cmp. ideo_ideo_2003 ; hanington2012universal ; kumar_101_2012 ). We give a detailed overview of such techniques in Section 5.
Research on DT found that its advantages include increased collaboration, empathy, product differentiation, and cost savings due to reduced redesign work and shorter lead time to development carlgren_design_2014 .
The working style for most DT techniques is very interactive and research with development teams suggests that developers welcome this variety in work style when applying DT. Additionally, research suggests that developers value and implement DT techniques when they have the required knowledge dobrigkeit_design_2019 and that developers can increase empathy towards users and team members, as well as shared product understanding and collaboration plattner_DT_2020 .
2.3 Design Thinking and Agile Software Development
Research on how to integrate DT and software development suggests different approaches. Lindberg et al. present four models of how DT can support software development lindberg_perception_2012 . We previously suggested adding two more models and expanding the list to six dobrigkeit_design_2019 . Table 1 provides an overview of these six integration possibilities.
|Split project model||DT as separate project before development|
|Overlapping teams model||DT as initial process phase with one or more development team members participating|
|Unified project model||DT as initial process phase and a large overlap of DT team and development team|
|Continuous DT model||Smaller DT phases are regularly integrated into agile development cycles|
|Ad-hoc DT model||DT as supporting tool that development teams can apply in form of DT workshops or DT phases if required|
|Toolbox model||Methods developers can use to overcome problems they can’t solve with IT methods|
In line with these proposals, researchers have suggested different integration approaches covering the range of different models. Running DT as a preceding phase to agile development, as suggested in the first three models seems the most common form: the Integrated Design Thinking and Lean Development Approach hildenbrand_intertwining_2012 , the Nordstrom Innovation Lab Process grossman-kahn_skip_2012 , InnoDev plattner_innodev_2018 or Converge marcus_software_2015 .
Continuous approaches of applying DT and software development are proposed by the Integrated Design Thinking and Agile Framework for Digital Transformation marcus_integrated_2016 , the Human-centered Agile Workflow ahram_introducing_2018 and DT@XP sohaib_integrating_2018 , which suggest regular DT workshops or phases as part of each development cycle. Ad-hoc DT implementations in later stages of the development cycle are proposed by InnoDev plattner_innodev_2018 and Converge marcus_software_2015 . These approaches propose spontaneous DT workshops with low planning overhead, when new features are added to the product, when tackling identified blockers, or supporting agile meetings. Toolbox approaches that implement these concepts were previously suggested plattner_DT_2020 ; pedersen_ux_2016 .
Adding DT techniques to the toolbox of Scrum meeting facilitators would be in line with approaches that follow the Ad hoc DT Model or the Toolbox Model.
We performed a two-step study to explore how techniques from the DT toolkit can be applied to address the challenges of Scrum meetings. First, we compared the requirements, challenges, and existing method recommendations from the Scrum literature with various methods described in the DT literature. In a following step, we supported this theoretical comparison with empirical evidence and concrete examples collected from a meta-analysis. In this meta-analysis, we reviewed data collected in two former studies in which the topic of DT techniques in Scrum meetings had emerged.
The first original study (OS-1)
was a ten-month longitudinal study in a global software company with a team of three Scrum teams who were experienced in both Scrum and DT. In this study, we explored when, where, and how DT was used during agile development. We observed several of the team meetings during the ten month period and interviewed eleven team members regarding their experiences using DT in their team. Additionally, we interviewed five employees from two further teams in the same company. This study and its results are further described indobrigkeit_design_2019 .
In the second original study (OS-2), we facilitated workshops introducing DT tools in agile development teams from six different companies. In this study, we explored the comprehensibility and applicability of a DT toolbox to everyday agile development with DT beginners. To this end, we prepared a collection of worksheets for twelve different DT tools, which we validated with six different companies. We observed one company in detail over a period of twelve weeks, in which they were introduced to one method each week and were asked to apply that method at least twice during the week. At the end of each week we interviewed the team in a group interview and asked all team members to fill out a method survey.
The five other companies were introduced to our toolkit in a workshop format in which we introduced them to three of the methods from our toolkit and afterwards conducted a group interview. The interview focused on finding out the experiences made within the workshop and if, where, and when team members would make use of these methods again. This study and its results are described in detail in a previous publication plattner_DT_2020 .
3.1 Research Context
As the focus of OS-1 was on understanding where, when, and how DT supports agile development, we chose the global software company because it had embraced DT as well as agile development practices for several years. The three teams participating in OS-1 were chosen because they were reported to be using DT in their daily work and accordingly all interviewees in this study had prior experience with DT, albeit to a different extent.
The six companies chosen for OS-2 had little or no prior experience with DT, with the focus of this study being to explore the applicability of a toolbox for DT novices.
The participants of these two studies featured a broad range of DT experience. Table 2 provides an overview of the teams that we observed and interviewed for this study.
|high||German-based team, incl designers developers and managers of a national software company developing a software for real estate management.|
|OS-1||Team B||3||high||high||US-based team, 1 designer, 1 developer and 1 manager of a national software company that develops proof-of-concept prototypes for new technologies or unsolved customer problems.|
|OS-1||Team C||2||high||high||US-based team, 1 designer and 1 manager of a national software company that acts as DT consultants to company customers, supporting them during their own DT efforts through training, coaching, and co-creation.|
|high||Team of programmers and designers from a medium sized, German software development company.|
|high||Team incl. a sales-manager, a designer and a mixed group of front-end and back-end developers from a German company developing team collaboration and security solutions.|
|high||Software developers of different teams from a German company developing project management software.|
|high||Diverse team incl. UX designers, Agile coaches and developers of a German company developing business applications.|
3.2 Data Collection and Analysis
For the theoretical comparison, we extracted the requirements and outcomes of each of the Scrum meetings as well as existing activities recommended for these meetings from the seminal literature. We concentrated on the descriptions by the Scrum originators Schwaber2017 ; Sutherland2007 and highly-referenced introductory literature that presents and describes the basic Scrum events rubin12 ; ScrumPrimer2012 ; Kniberg2015 ; Meyer14 . We compared these requirements with techniques frequently mentioned in the DT literature to create a mapping of Scrum meetings and DT techniques that can support them.
For the meta-analysis we re-analyzed the data collected during OS-1 and OS-2. In OS-1, we wrote observation notes during various Scrum meetings of the observed Team A over a period of ten months and recorded and transcribed eleven interviews with members of that team as well as five additional interviews with members of two further teams (Team B and C) from the same company. In OS-2, we collected observation notes over a period of twelve weeks within Team D and during five workshops with teams E-H. Additionally, we recorded and transcribed twelve group interviews in Team D (one after each week) and one group interview with each of the other four teams following their workshops. In this meta-analysis we iteratively coded the available materials with a focus on the use of DT during Scrum meetings. We thus derived concrete examples to support our initial mapping. As a result we created initial guidelines on how to integrate DT techniques in each of the Scrum meetings for Scrum Masters and teams that want to improve their meetings’ level of interactivity and collaboration or simply desire more variety.
4 Goals and Requirements of Scrum Meetings
The Scrum meetings, sometimes referred to as ceremonies Meyer14 , conducted by teams during a development iteration form the core of the Scrum software development method. These meetings mark points in the process where team members regularly discuss iteration progress and process aspects as well as work on development artifacts. The Scrum Guide describes the purpose of these meetings as creating “regularity and to minimize the need for meetings not defined in Scrum” Schwaber2017 . Schwaber and Sutherland point out that all meetings are essential and that not including a prescribed Scrum meeting results in “reduced transparency” and a “lost opportunity to inspect and adapt” Schwaber2017 . In the following, we present and characterize the five central Scrum meetings, with a particular focus on their requirements and outcomes.
4.1 Product Backlog Refinement
In the Product Backlog Refinement meeting, also referred to as Backlog Grooming Kniberg2015 , the team focuses on preparing product backlog work items for the following one or two Sprints ScrumPrimer2012 . Refinement includes activities such as analysis of requirements, adding required details to work items, estimating implementation effort, or splitting large work items Schwaber2017 . Furthermore, outdated or redundant items should be removed from the product backlog to keep its components as clear and actionable as possible. While the refinement of work items can form part of a Planning meeting, depending on teams’ preferences and context, most of this work should be done before the Planning meeting Kniberg2015 . Separating these meetings allow the subsequent Planning to take advantage of well-analyzed and estimated items, making it more efficient ScrumPrimer2012 .
Many of the details of how Product Backlog Refinement is conducted are up to the team. Scrum does not feature a defined time box or even a frequency for the meeting. However, the Scrum Guide notes that the meeting should not consume “more than 10% of the capacity of the Development Team” Schwaber2017 .
Attendees: Development Team, Scrum Master, Product Owner.
Point in Process: Custom, depending on the Scrum implementation.
Requirements: A shared understanding of both the technical effort to implement a solution and the business value it delivers. This allows establishing cost estimates and enables discussions on priorities.
Outcome: A set of well-understood, clearly described, thoroughly analyzed, and estimated set of user stories in the Product Backlog ScrumPrimer2012 .
4.2 Sprint Planning
A Sprint Planning meeting marks the start of a new development iteration in Scrum. They are used to prepare for the upcoming Sprint and are typically divided into two parts: the first concerned with what to build, the second with the specifics of how to implement it ScrumPrimer2012 . The Product Owner communicates their vision of what is most critical regarding the project’s next steps. Together with the Development Team, the PO discusses the highest-priority items (from the Product Backlog), which represent those features that are most interesting for the near future. The contexts and details for these high-priority items are reviewed by team members, providing the rationale and a shared understanding of the value inherent in the planned features.
Based on these discussions and insights, the team may then define a shared, relatively stable Sprint Goal ScrumPrimer2012 and the next items to be worked on rubin12 ; Sutherland2007 . The Sprint Goal summarizes the objective that the team aims at achieving within the Sprint, which ideally has a cohesive theme. It is based on the latest product increment, the projected capacity, and the past performance of the team ScrumPrimer2012 . To forecast the functionality that can be developed during the Sprint, the entire Scrum team collaborates to understand the work needed to achieve the Sprint Goal Schwaber2017 . The Planning meeting provides an opportunity to flesh out further details of the work items planned for the first days of the Sprint, e.g., by creating additional subtasks for work items or bugs. When effort estimates are not yet available (e.g., as work items were recently updated), the team works together to provide these assessments. Considering the priority and effort estimates, the Development Team decides on the amount as well as the specific tasks it will complete in the upcoming Sprint. The Development Team has the ability and responsibility to additionally include relevant work items with potentially lower priority, e.g., in the case of dependencies ScrumPrimer2012 .
By the end of the Sprint Planning meeting, the Development Team should have formed a consensus on the plan of action and should be able to explain how it intends to accomplish the Sprint Goal and build the anticipated product increment Schwaber2017 .
Attendees: Development Team, Scrum Master, Product Owner (optional for breaking down items into tasks).
Point in Process: At the beginning of a Sprint.
Requirements: The Product Owner representing the customer and the development team must share a mental model and a deep (technical) understanding of the work ahead and the business value it presents Conboy2011 .
A coherent Sprint Goal, summarizing the development work to be done.
A list of items to be worked on during the next Sprint, ideally including the highest priority ones.
A plan of how to deliver the planned functionality in a product increment.
4.3 Daily Scrum
The Daily Scrum meeting is a short team meeting to enable collaboration, in which the Development Team inspects and synchronizes their work as well as outline the next aims on a daily basis Schwaber2017 . The primary justification for a daily meeting during a Sprint is the general Agile principle that direct contact is critical to project success Meyer14 . In the Daily Scrum, progress toward the specified Sprint Goal and the remaining work items of the Sprint Backlog are reviewed, with a particular focus on identifying impediments to the team’s productivity ScrumPrimer2012 . Necessary In-depth discussions of identified issues are postponed rubin12 and the Scrum Master helps team members resolve obstacles in the following steps ScrumPrimer2012 ; Sutherland2007 . Daily Scrum meetings are designed to be time-boxed to 15 minutes, to improve communication, and to possibly eliminate further meetings by facilitating quick decision-making Schwaber2017 .
The structure of the Daily Scrum meeting is decided by the Development Team with a focus on producing an actionable plan for the day. It represents an opportunity for the members of the Development Team to make realistic commitments to each other Meyer14 . The Scrum Guide proposes the following three questions for every participant to structure the meeting Schwaber2017 :
What did I complete yesterday that helped meet the Sprint Goal?
What will I work on today to help meet the Sprint Goal?
Do I see any impediments or blockers towards the Sprint Goal?
The Daily Scrum meeting is not intended to be used as a detailed status reporting meeting, instead it focuses on quick information sharing on what is happening across the team. Its tone should ideally be pleasant and enjoyable while staying informative. The meeting is also known as a Stand-up as one of the original ideas to ensure a short session was to require participants to stand Meyer14 .
Attendees: Development Team, (Product Owner and Scrum Master participate as developers if they are actively working on Sprint Backlog items).
Point in Process: Every day of a Sprint.
Outcome: An understanding between team members of the progress towards the Sprint Goal and the impediments that need to be tackled Meyer14 .
Requirements: As the meeting is strictly time-boxed, concise recollections and summaries of recent work by attendees as well as individual reflections on encountered impediments are essential.
4.4 Sprint Review
Held at the end of the Sprint, the Sprint Review meeting is used to reflect on the work of the completed Sprint Meyer14 . It focuses on inspecting the created product increment and eliciting feedback Sutherland2007 and is the time to showcase the team’s work. It is, therefore, also referred to as a Sprint Demo Kniberg2015 , where a hands-on inspection of the real software takes place, and the Development Team gets credit for their accomplishments Kniberg2015 . During the meeting, the team members present the Sprint results to key stakeholders and customers and discuss progress. The meeting provides an opportunity for the Product Owner to gain insight into the status of both the current product and the rest of the team. Other team members gain updates from the Product Owner and the market situation ScrumPrimer2012 .
To be demonstrable, the Sprint’s work must result in a working product increment and meet the team’s agreed quality bar. In the case of software development, this might refer to a system that is integrated, tested, user documented, and “potentially shippable” ScrumPrimer2012 . The Sprint Review is the second to last meeting of a development iteration. The entire group collaborates on the next steps so that the Sprint Review provides valuable input to the Sprint Planning meetings of the next sprint Schwaber2017 . The Product Backlog is updated based on the completed work items or shifts in business value or the market Schwaber2017 .
Attendees: Development Team, Scrum Master, Product Owner, other project stakeholders as appropriate.
Point in Process: After work on the Sprint has concluded.
Outcome: A revised Product Backlog based on collected feedback Schwaber2017 .
Requirements: Knowledge of the status of work items. An understanding of how the developed functionality integrates with the rest of the product. Strategies to collect actionable feedback regarding the software increment.
The Retrospective meeting is the last Scrum meeting of a Sprint, taking place after the Sprint Review and before the next Sprint Planning. It is focused on inspecting and improving the executed development process of teams and is concerned with creating a plan for improvements to be enacted in the future Schwaber2017 . The meeting helps identify and rank aspects of the completed Sprint that have proven themselves and should be continued and those that should be discontinued ScrumPrimer2012 ; Sutherland2007 . Assumptions that led the team astray should be identified and their origins examined. Improvement opportunities can be identified for all process aspects, including people, relationships, work process, and tools, with the overall goal of making the development process and the employed practices more effective and enjoyable Schwaber2017 .
The Scrum method itself does not prescribe the detailed contents of Retrospectives and many activities as well as agendas for conducting the meeting have been proposed Baldauf2018 . By the end of the Retrospective, the team should have found a consensus on the concrete improvement actions it will attempt to implement in the next Sprint. The most impactful improvements should be addressed as soon as possible and may be added to the next iteration’s Sprint Backlog.
Attendees: Development Team, Scrum Master, Product Owner.
Point in Process: At the very end of a Sprint.
Outcome: Set of ordered action items regarding the team’s work processes to be enacted in the next Sprint Meyer14 .
Requirements: Detailed recollections of the enacted process of the concluded Sprint, including relevant interactions, dependencies, highlights, and encountered issues. Reflection on the severity of these items.
5 Techniques in the Design Thinking Toolkit
As described before, the DT approach uses a multitude of techniques that support each step of the DT process. These techniques originate from diverse areas, like quality management, research in creativity and design, research in communication, ethnography, and informatics brenner_design_2016 . There is no complete list of DT techniques, and practitioners add new techniques to their toolkits from other disciplines. So while no exact definition of a DT method exists, some methods are frequently mentioned and described by researchers wolbling_design_2012 ; plattner_innodev_2018 ; carleton_playbook_2013 and practitioners ideo_ideo_2003 ; hanington2012universal ; kumar_101_2012 . Method collections, such as ideo_ideo_2003 ; hanington2012universal ; kumar_101_2012 ; carleton_playbook_2013 both from research and practice provide an overview of possible techniques and are often organized by steps in the design or DT process. For this section we propose a slightly different categorization, focusing on the tasks that can be achieved with a group of techniques. In the following,we describe the categories of techniques that are typically found in a design thinker’s toolkit and give a few concrete examples for each category. Table 3 provides an overview of the presented categories with some examples.
|Warm-up Techniques||Dancing in the Dark, Two Truth and One Lie, Paperclip Exercise, Marshmallow Challenge|
|Energizing group exercises designed to help participants ease into an atmosphere of teamwork.|
|Unpacking Techniques||Charetting, Semantic Analysis, Mind Mapping|
|Group exercises designed to share and discuss the existing understanding and assumptions.|
|Desk Research Techniques||News Search, Market Research, Technology Research|
|Research within books, published research literature or the internet,e.g. by searching for market trends, patents, news, etc.|
|Field Research Techniques||Extreme User or Expert Interviews, Fly on the Wall, Shadowing, Contextual Inquiry, Immersion|
|Techniques geared towards learning directly from the user or stakeholder.|
|Knowledge Sharing Techniques||Story Telling, Role Playing|
|Techniques geared towards sharing experiences, information and results from research or testing with the team.|
|Knowledge Organization Techniques||Clustering, Venn-Diagram, 2x2Matrix, Timeline|
|Techniques geared towards organizing and visualizing information from research and testing.|
|Knowledge Consolidation Techniques||Personas, PoV Statement, Journey Maps, Storyboards|
|Techniques geared towards condensing information from research or testing in a meaningful way usable for later process stages.|
|Idea Generation Techniques||How to and How Might we Question, Brainstorming|
|Techniques that prompt the generation and sharing of ideas towards a specific prompt or question.|
|Prototyping Techniques||Sketches, Role Plays, Storyboards, Wire Frames|
|Techniques that represent an aspect of an idea or a product in a meaningful way to gather feedback or allow for testing, e.g., by telling a story or presenting a schematic visual.|
|Testing Techniques||Think Aloud, Concept Testing, A/B Testing, Usability Testing|
|Techniques, in which feedback or experiences of users are gathered through direct engagement with a prototype or the product.|
|Feedback Techniques||I Like I wish, Feedback Capture Grid, Empathy Map|
|Techniques developed to elicit and collect structured feedback using different categories.|
|Facilitation Techniques||Check-in & Check-Out, Team Rules, Planning sessions, Reflective sessions, Voting techniques|
|Techniques usually used by coaches and facilitators to ensure effective team work.|
5.1 Warm-Up Techniques
Warm-ups also called ice-breakers or energizers, are a known concept for facilitators of all kinds of team-based workshops. Warm-ups can serve a variety of functions. In the very literal sense of warming-up, these methods can help to energize participants after a longer presentation or early in the morning by providing several minutes of physical activity, e.g. Dancing in the Dark. For this activity participants block their sight, e.g. with a post it or their hats. Then the moderator starts music that is suitable for dancing and asks the participants to perform different types of dancing and movements, for example Tango, Belly Dance or a simple spin. In a new team or at the beginning of a workshop, warm-ups can help participants get acquainted with each other by facilitating the sharing of specific knowledge. For example, the exercise Two Truths and One Lie asks each person in the group to state three things about themselves, two of which are true and one is a lie. Afterwards, the rest of the group has to guess which one is the lie. During a workshop warm-up techniques can help to get the group into the right mind-set for the following exercises. If for example the next exercise for the group is to brainstorm ideas, a warm-up that practices brainstorming can help to get the creativity flowing. The paperclip exercises is such a warm-up. It asks each participant to write down as many possible usage scenarios for a big paper clip as they can over a certain time frame, e.g. 5 minutes. Similarly, if building a prototype is the next step a warm-up that requires building something gets people into the activity. For example, in the marshmallow challenge teams are competing to build the highest possible structure out of tape, spaghetti and a marshmallow. By doing so they learn to build, fail and iterate together as a team. If at some point in time the team is in a negative mood, warm-ups can provide a fun activity that helps to create a positive atmosphere before carrying on. If a team is stuck, they provide distraction, making it possible to return to the task later.
5.2 Unpacking Techniques
Techniques used for unpacking are designed to share and discuss the existing understanding and assumptions on a topic. While doing so, teams can reach a shared understanding and uncover knowledge gaps or assumptions that have to be verified. Especially in the beginning of the process they can help to understand and analyze the design challenge. Common unpacking activities include charetting, semantic analysis or simple brainstorming techniques such as mind mapping. Charetting is an activity that can be used for unpacking but also for ideation. During unpacking the team first brainstorms relevant users or contexts for their design challenge. In a second step the team picks one of these users and brainstorms potential issues relevant to that user. In the third step the team picks the most interesting challenge and brainstorms potential solutions. Steps 2-3 are repeated for further users. Following these brainstorming tasks, the team discusses what they have discovered and rephrases the design challenge to reflect the new understanding. For a semantic analysis of the given design challenge the team highlights important passages or words from the design challenge and discusses experiences, thoughts, open questions, associations and assumptions in detail. The information discussed is collected on sticky notes or the whiteboard. Thus the team explores different aspects of the challenge and reaches a common understanding. At the end of the analysis the team often rephrases the design challenge to reflect their new understanding. A less structured approach to unpacking is mind mapping, whereby the team brainstorms whatever comes to mind when thinking of the design challenge and then organizes the information in a mind map with the challenge in the center.
5.3 Desk Research Techniques
Desk research techniques, in contrast to field research, include analysis activities that can be done from one’s desk. These include reviewing studies, reports or patents in published research literature as well as online sources and grey literature. Furthermore, market or technology trends can be explored usually through specific search sites on the internet.
5.4 Field Research Techniques
Field research techniques are designed to learn directly from users and stakeholders. Several of the techniques originate from qualitative and ethnographic research in social sciences. These techniques can be separated into interview techniques, observation techniques, and self-immersion. Interview techniques elicit information from users or stakeholders by asking them questions. Common interview techniques are extreme user or expert interviews; or group interviews. Observation techniques allow the team to learn from the users or stakeholders in their actual environment by observing (and maybe asking them questions), e.g. in their workplace. Observation techniques common in DT are Fly on the Wall, Shadowing or Contextual Inquiry. Immersion techniques aim for the team to make relevant experiences themselves if possible. These techniques include trying out a service or product by themselves, or simulating specific conditions, e.g. bad eyesight, for an extended period of time.
5.5 Knowledge Sharing Techniques
Knowledge sharing techniques support a DT team in distributing the information collected through research and testing activities. A common technique for knowledge sharing is story telling. In this activity, the involved researchers describe the details of their research to the rest of the team in the form of a story. The other participants are asked to listens attentively and to collect the important information on sticky notes. The form of a story should enable presenting the gathered information in a simple format and is supposed to provide reference points for anchoring key facts in the long-term memory of listeners.
5.6 Knowledge Organization Techniques
DT teams collect a vast amount of information that has to be organized in meaningful ways in order to learn from it. Common techniques to do so include clustering, Venn Diagrams, 2x2 Matrices, Timelines or process diagrams. With clustering, the team aims to group the information and looks for relationships or contradictions between clusters. Diagrams like matrices, Venn Diagrams or timelines help the team to organize their findings and highlight specific aspects, such as time or product categories and thereby uncover hidden needs and problems, or knowledge gaps.
5.7 Knowledge Condensation Techniques
In order to further work with the knowledge gained through research and testing, the team needs to condense it in such a way, that it can easily be accessed in later process stages. Personas, Point of View Statements, Storyboards, or Journey Maps are common techniques to condense knowledge. Personas represent a certain group of users with their needs, problems and wishes. The Point of View Statement condenses those problems from the problem domain the team currently wants to tackle. Storyboards or Journey Maps can be used to condense the experiences of many users into one typical experience.
5.8 Idea Generation Techniques
An integral part of every design process is to develop and discuss solution concepts. Many techniques from design or other creative disciplines can be used to develop ideas within a team. Common techniques include How to or How might we questions as brainstorming prompts, and a myriad of brainstorming techniques. The How to and How might we questions are often related to a Point of View Statement or a Persona derived from knowledge consolidation. They represent a specific problem to solve (How might we) or a specific aspect of the problem or the solution (How to). With regards to brainstorming techniques, different techniques work for different people and teams. Some brainstorming techniques let individuals brainstorm silently first and follow with a sharing and another group brainstorming session. Other brainstorming techniques use group brainstorming only. Some brainstorming techniques incorporate bodily activity. Some brainstorming techniques even use further prompts to generate ideas such as images or objects. Additionally DT prescribes a set of brainstorming rules.
5.9 Prototyping Techniques
In order to further develop an idea or test it with team members, users or stakeholders the idea needs to be prototyped. Prototypes can be a number of things, including simple sketches, product packages, roleplays, storyboards, wire frames, or functioning prototypes. Prototypes can have a different level of abstraction, ranging from low-fidelity prototypes designed to test basic interactions or ideas to high fidelity prototypes to test details of the interaction and functionality. Additionally, prototypes may cover the complete solution concept or single aspects of the concept.
5.10 Testing Techniques
Evaluating concepts with the actual user is an important aspect of DT. Several techniques to test concepts from areas such as UX Design, Human Computer Interaction, Design, and Software Development are part of the DT toolkit. Common techniques include the Think Aloud Method, Concept Testing, A/B Testing, or Usability testing. With the Think Aloud Method the user is asked to look at or try out the prototype and voice his thoughts out loud while doing so. The tester can ask clarifying questions. Concept Testing usually involves a low-fidelity prototype to explain the concept and an interview to gather feedback. A / B Testing is used to derive feedback for different alternatives be it features or concepts. Usability Testing aims to test how easy a product is to use.
5.11 Feedback Techniques
Feedback techniques can help to elicit and provide feedback, as well as document feedback. Most feedback techniques provide a specific set of categories in which to organize the feedback. For, example I like, I wish asks participants to provide feedback about what they liked and what they wish would have been different. Sometimes I like, I wish has an additional category called What if or How To in which participants can provide concrete examples to improve in the future. Other common feedback techniques include Five Finger Feedback, the Empathy Map, and the Feedback Capture Grid.
5.12 Facilitation Techniques
Facilitation techniques aim to ensure conditions in which the team can effectively work towards their objectives. These techniques include those that help a team, manage conflicts, reflect on their process or manage their resources, e.g.time or energy. Common techniques include: Team Rules, which a team agrees on, Check-Ins that allow for team members to transition into the group work and let the team know what might affect their working today, Check-Outs or reflective sessions similar to retrospectives that provide space for teams to reflect on their process and their team work. Voting techniques, such as dot voting or thermometer voting can help to arrive at a decision if no consensus can be reached. Planning sessions help a team to decide on how to divide their time and work force. And the famous Time Timer helps keeping things time boxed.
6 Mapping of Scrum Meetings and Techniques from the Design Thinking Toolkit
By linking the collected requirements for each of the Scrum meetings and the goals and effects of the different types of techniques in the DT toolkit we derived a mapping between the two. Table 4 presents our theoretical mapping and the following sections provide explanations and examples for each entry.
|Theoretical mapping of Scrum meeting and applicable Design Thinking techniques|
6.1 Techniques Generally Applicable to Scrum Meetings
DT warm-ups, short energizing activities, can be employed before longer meetings to motivate participants. If several Scrum meetings are facilitated on the same day warm-ups can help the team transition from one meeting to the next and provide a welcome break. On such meeting days facilitation techniques such as check-ins and check-outs and establishing meeting rules can help the team transition into and out of the meeting work mode. Additionally the Time Timer can help staying within planned time boxes.
6.2 Product Backlog Refinement
During Refinement, developers need to understand the users’ requirements, come up with solutions on how to implement these and document them. To support the understanding of the product vision as well as work items in general, artifacts from knowledge organization and consolidation techniques, which describe and document findings from user research, such as Personas, Journey Maps, Storyboards can be employed by the PO to help the team understand the underlying needs and business aspects. Similarly, artifacts from prototyping techniques such as wireframes and other low fidelity prototypes can be employed by the PO to help the team understand aspects of the functionality. Additionally, when discussing how to implement a feature, unpacking methods such as charetting or creating a mind-map can help to form a shared understanding of the feature. When discussing possible implementations, idea generation and prototyping techniques such as different types of brainstorming or sketching sessions can support the generation and discussion of possible solutions. The artifacts from these techniques, e.g. wire frames, diagrams, Journey Maps or Storyboards help to document the refined solution and can feed back to understanding the feature during planning or development. Finally, voting techniques can support estimation and prioritization of work items.
6.3 Sprint Planning
The main goal of this meeting is to make a plan and commit to it. As this is not a creative or problem-focused meeting, DT techniques are less helpful. However, some DT artifacts and techniques can support the meeting goals. As previously mentioned, several DT artifacts from knowledge organisation and condensation as well as from prototyping techniques can support the PO in explaining the necessary work and can help the team understand the needs, business aspects, and functionality behind the feature. Similarly, to the Product Backlog refinement, unpacking techniques help to achieve a shared understanding if the feature is not clearly enough defined yet. Additionally, voting techniques, such as Dot Voting or Thermometer Voting, can support the team in prioritizing and deciding which work items to include in the following sprint. And finally, the team can make use of idea generation techniques when formulating and discussing the sprint goal.
6.4 Daily Scrum
The daily Scrum, as the short status update is another meeting that is not creative and problem-focused. However, preparing the Daily Scrum meeting by quickly unpacking recent achievements and identified impediments as a group or as individuals can help stay in time. Establishing rules similar to DT’s brainstorming rules can ensure that discussions are deferred. A possible adaptation might be:
Defer judgment and discussions.
Stay focused on the topic.
One conversation at a time.
Observers defer talking.
Additionally, these rules can be visibly placed around the meeting area as commonly seen with brainstorming rules in DT workspaces.
6.5 Sprint Review
As this meeting focuses on inspecting results and collecting feedback, actual customer representatives or stakeholders are present at these meetings who might not be familiar with the details relevant to software development. DT testing techniques, such as the Think Aloud technique can support developers in eliciting targeted, actionable feedback. Additionally, feedback techniques such as the Feedback Capture Grid, the Empathy Map or the Five Finger Feedback can support customers in giving structured feedback, which can be used to adapt work items or gauge the business value of features.
6.6 Sprint Retrospective
Creative activities for Retrospectives are already commonplace jovanovic2016 and the toolbox of DT techniques can add to these. The Retrospective focuses on improving the work process and, as such, a large part of the DT process and its techniques can be applied to this meeting.
In fact, entire Retrospectives can be facilitated in the form of a DT workshop, including the understand or empathize step to collect attendees thoughts, a clustering step to identify major feedback items, a voting to decide which issues to work on for the next Sprint, an ideation phase to come up with solutions as well as a prototyping phase to sketch out the solution. A testing phase implements the solution in the following Sprint. During such a DT workshop as well as during ”normal” retrospectives unpacking, knowledge sharing, and feedback techniques can support the team in reflecting on the past sprint. Knowledge organisation and condensation techniques help organize the feedback and identify improvements for the upcoming sprint. Idea generation and prototyping techniques can support the generation and documentation of action items.
7 Concrete Examples from our Studies
After building a theoretical mapping between the Scrum meetings and their requirements and different categories of techniques from the DT toolkit we analyzed the data from our two original studies in order to derive concrete examples from practice and compare them to our mapping. For each of the Scrum meetings Table 5 provides an overview of the DT techniques observed and mentioned in our study. As some of these techniques were not mentioned before, Table 6 briefly describes the specific DT techniques and artefacts mentioned in this section.
|Practical experiences with Scrum meetings and applicable Design Thinking techniques|
|Product Backlog Refinement||
|30 Seconds Sketch||
7.1 Techniques generally applicable to Scrum Meetings
The adherence to predefined time boxes for Scrum meetings can be supported by a common tool from DT workshops: the Time Timer.
It is a large round analogue timer that visually displays the passage of time as well as the remaining time using a continuously decreasing red disk. After a developer experienced the Time Timer for the first time in a DT workshop he explained: ”That was a real aha moment. If we could have these for our meetings that would be great.“
”That was a real aha moment. If we could have these for our meetings that would be great.“
7.2 Product Backlog Refinement
One of the SMs from our study, who is also a developer, explained how he plans to introduce knowledge condensation techniques, to better describe features and achieve shared understanding when discussing backlog items: ”We will start using Storyboards for larger features, explaining why the user needs this feature and how it is connected to the solution. Something we usually ignore, but with such Storyboards it can be done. Storyboards can serve as the vision for this feature. Additionally, I would like to have Personas in the backlog for orientation and then sort the backlog around these Personas and stories for orientation instead of the 4 levels of items we have now. Like this, we can achieve shared understanding early on.” In a similar notion, another developer mentioned that he would like to have Personas around all the time to remember the users: ”We should print these Personas and have them always near, so we never forget who is the user of the things we do.” A developer explained how his team uses a quick form of brainstorming to discuss features and achieve a shared understanding: ”If a task still has aspects that need clarification it can be interesting to do a short design thinking session. In the simplest case we just do a short brainstorming to discuss what the feature means for each of us.”
Concerning idea generation techniques during refinement, developers saw potential in using ideation techniques, but not in all cases. One developer stated: ”With sketching, [he talked about the 30 seconds sketch technique] ]It is easier to communicate one’s ideas, and seeing them drawn makes it easier to modify and build upon them. However, another developer noted: It depends on the level of refinement. If all that remains are technical challenges, DT does not really help. If aspects of the feature are not clear yet then DT is a really interesting approach to refinement.”
7.3 Sprint Planning
In our study, the process of planning the next items to work on was rarely mentioned in conjunction with DT techniques. As described in 7.2, one interviewee wanted to use storyboards and personas whenever discussing backlog items. In contrast, other study participants pointed out that the Scrum planning meeting relied on facts, such as the velocity of the team, the amount of work they forecast could be completed in the upcoming development iteration and the priority of work items. As one developer explained: ”For planning it’s not a fit I think. It’s about prioritizing and the capacity of the team; it’s just not a creative process.“
7.4 Daily Scrum
The Daily Scrum meeting, as a regular check-in and team synchronization meeting, was not explicitly mentioned by interviewees as requiring the support of DT techniques. However, Daily Scrum meetings should ideally be short and rely on strict time-boxes. As described in Section 6.1 the Time Timer can help teams with time keeping especially during this meeting.
7.5 Sprint Review
The use of testing techniques during review meetings to collect feedback from customers and stakeholders were mentioned several times in our studies. For example, one interviewee noted: ”Evaluation techniques — not traditional ones, where you ask questions and they answer, but for example, you just give the system to the user and see how he interacts with it — that might help in getting better feedback.“
Furthermore, feedback techniques were mentioned as being useful. One developer explained how a feedback capture grid not only helped to collect feedback but also led to cooperative ideation with the customer: ”I think it was good, and it helps to share different point of views. You can see what is liked and disliked, and the criticisms makes you ask for questions and the questions gives you some ideas. So there is a cycle for generating ideas.“
7.6 Sprint Retrospective
One of the SMs from our studies facilitates his Retrospectives as DT workshops: ”Running a feedback session with 30 people is chaos, it would be a whole day retro with nothing achieved. For large groups more organization is needed so we collected feedback with the 3 Ls in the group, quickly clustered and then split up into smaller groups for the next steps. That’s where DT really makes sense.” As next steps, the smaller groups in this retrospective would brainstorm solutions and prototype them in a story board or sketch. A developer from his team describes his first experience with such a DT-Retrospective as follows: ”That big retro was a very nice experience. It was totally different from what we are used to. Normally, we just sit in a room, but here I find it was very interactive and I liked that people were open. I felt that most of the conclusions from that meeting are on their way now, so it really had a good impact on our team.”
In this chapter, we analyzed the integration of DT techniques into the meeting activities of the Scrum software development method from two perspectives.
First, we analyzed the five central Scrum meetings and reviewed their goals, requirements, and outcomes along with categories of DT techniques and the tasks they support. From the outcomes of this step, we derived a mapping of Scrum meetings and applicable DT techniques. In this theoretical mapping, we showed links between at least two categories of DT techniques with each of the five investigated Scrum meetings. We found the most matches between categories of DT techniques and the Retrospective, the Product Backlog Refinement, as well as the Sprint Planning meetings. This finding is in line with related research and practice for Scrum activities in meetings, which also concentrate on Retrospectives, Product Backlog Refinement and Sprint Planning.
In a second step, we analyzed data from two previous studies in which the topic of DT techniques supporting Scrum meetings emerged. This meta-analysis provided concrete examples from practice of how DT techniques support the facilitation of Scrum meetings. In our case study, we found examples of DT techniques supporting four out of the five Scrum meetings. For the remaining meeting, the Daily Scrum we could not observe any specific methods and the participants did not specifically mention the meeting in our interviews. For the Sprint Planning, some study participants specifically mentioned that they do not consider DT techniques useful for this event, while others considered artifacts from knowledge condensation within the backlog as useful to establish vision and shared understanding.
|Comparison of theory and practice of Scrum meetings and applicable Design Thinking techniques|
Comparing the two created mappings, we find several of our theoretical links confirmed by concrete examples from practice. Table 7 provides an overview of which entries could be confirmed and which entries could not be confirmed within this study. However, some of the categories of DT techniques were not mentioned or observed in our studies. For example, the anticipated general applicability of DT warm-up techniques for all Scrum meetings could not be confirmed as part of this study. This might be explainable by the perceptions of participants. Some of the better known warm ups and activities, especially those that include elements of playfulness, are easily perceived as childish games or “fluff” by professionals Verity2012 ; Kolko2018 .
A noticeable difference between the mappings can be seen for the Sprint Planning meeting. While the meeting requirements, goals, and outcomes match with several DT activities the data from our studies only supports one of these matches: the use of knowledge condensation techniques to explain the vision of a feature and support shared understanding. Additionally participants even mentioned that they don’t think DT activities are useful in this meeting. A possible explanation for this discrepancy might be found in the different ways planning meetings are facilitated. While in some teams Sprint Planning and Backlog Refinement are held as one meeting or as two meetings directly following each other, other teams separate the two meetings by a number of days. Thus, a Sprint Planning meeting which directly follows a Backlog Refinement probably focuses on planning and prioritizing action items and does not require further establishment of shared understanding. For a Sprint Planning that does not include or follow a Backlog Refinement the theoretically useful artifacts and techniques might be more relevant.
Another interesting finding was the mention of the 3Ls as a DT technique. This technique is commonly known as a retrospective activity for the data gathering stage, albeit as the 4Ls 1114Ls Retro activity https://retromat.org/de/?id=78. From our experience this activity is less common in the DT community. While it is hard to say in which area activities or techniques were first introduced and used, we believe this is an indicator that the toolkit of Scrum meetings and the DT toolkit are compatible and seem to have merged in the eyes of the interviewee.
This study is based on two previous studies investigating DT use in software development teams but did not specifically focus on DT techniques for Scrum meetings. Accordingly, these previous studies featured an adequate sample of participating teams but provided only a small selection of concrete examples for DT techniques used in Scrum meetings. Further studies explicitly focused on this topic, can confirm additional entries in our mapping or can add additional new entries and concrete examples. Additionally, in this study, we investigated the five central Scrum meetings, which are generally applied in professional software development contexts. Several additional meetings exist, especially in different scaled versions of Scrum. One such example is the Scrum of Scrums meeting, introduced by one of the originators of the Scrum method Sutherland2001 , which provides a meeting with members of multiple development teams. Future work should, therefore, investigate the use of DT techniques in these scaled Scrum meetings.
Despite these limitations, our results indicate that integrating DT methods into Scrum meetings is a promising area of research with practical implications. Several of the DT activities that can be added to the toolbox of Scrum activities promote collaboration and a shared understanding within the development team. They help achieve meeting goals and prevent common Scrum meeting issues. As added activities to the Scrum toolkit they provide a wider variety of methods for each Scrum meeting, promote diversity, and prevent monotonous meetings. Finally, development teams are confronted with additional use cases for DT techniques they might already be familiar with from DT workshops or DT project phases. Thus Scrum team members have the opportunity to further practice these DT techniques. However, not all Scrum meetings can or should be supported by DT in the same way, i.e., Sprint Planning and Daily Scrum can only marginally integrate DT techniques, while the Retrospective and the Product Backlog Refinement provide several opportunities to make use of DT techniques.
This research complements existing efforts to integrate DT and agile software development methods. Our results provide a starting point for practitioners to leverage DT’s advantages, not only in an initial DT phase but also during regular development activities, in a similar fashion as proposed in the Ad-hoc DT model or the Toolbox model. The mapping developed in this chapter presents an initial guideline on how to integrate individual DT techniques into existing Scrum meetings. It provides motivation and ideas on enhancing Scrum meeting routines with further techniques for meeting facilitators and teams: either identifying additional use cases for already familiar techniques or by pointing out new techniques worth investigating.
-  Ken Schwaber and Jeff Sutherland. The Scrum Guide - The Definitive Guide to Scrum: The Rules of the Game. Technical report, scrumguides.org, 2017.
-  Kenneth S Rubin. Essential Scrum: A practical guide to the most popular Agile process. Addison-Wesley, 2012.
-  R Akif and H Majeed. Issues and Challenges in Scrum Implementation. International Journal of Scientific & Engineering Research, 3(8):1–4, 2012.
-  Juyun Cho. Issues and Challenges of agile software development with SCRUM. Issues in Information System, IX(2):188–195, 2008.
-  Patrick Kua. The Retrospective Handbook: A guide for agile teams. Leanpub.com, 2013.
-  Adam Przybyłek and Dagmara Kotecka. Making agile retrospectives more awesome. In Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, volume 11, pages 1211–1216, 2017.
-  Kieran Conboy, Sharon Coyle, Xiaofeng Wang, and Minna Pikkarainen. People over Process: Key Challenges in Agile Development. IEEE Software, 28(4):48–57, jul 2011.
-  Corinna Baldauf. Retromat - Run great agile retrospectives! Leanpub.com, 2018.
-  Miloš Jovanović, Antoni-Lluis Mesquida, Nikola Radaković, and Antonia Mas. Agile retrospective games for different team development phases. Journal of Universal Computer Science, 22(12):1489–1508, 2016.
-  Yen Ying Ng, Jȩdrzej Skrodzki, and Maciej Wawryk. Playing the Sprint Retrospective: A Replication Study. In Lecture Notes in Business Information Processing, volume 376 LNBIP, pages 133–141. 2020.
-  N.C. Haugen. An Empirical Study of Using Planning Poker for User Story Estimation. In AGILE 2006 (AGILE’06), volume 2006, pages 23–34. IEEE, 2006.
-  Luke Hohmann. Innovation Games: Creating Breakthrough Products through Collaborative Play. Addison-Wesley, 2006.
-  Dave Gray, Sunni Brown, and James Macanufo. Gamestorming: A Playbook for Innovators, Rulebreakers, and Changemakers. O’Reilly Media, Inc., 1st edition, 2010.
-  Franziska Dobrigkeit, Philipp Pajak, Danielly de Paula, and Matthias Uflacker. Dt@it toolbox - design thinking tools to support everyday software development. In Design Thinking - Research Investigating Design Team Performance, Understanding Innovation. Springer, 2020.
-  Rafael dos Santos Braz, José Reinaldo Merlin, Daniela Freitas Guilhermino Trindade, Carlos Eduardo Ribeiro, Ederson Marcos Sgarbi, and Fabio de Sordi Junior. Design thinking and scrum in software requirements elicitation: A case study. In Design, User Experience, and Usability. Design Philosophy and Theory, pages 179–194. Springer International Publishing, 2019.
-  Julio Cesar Pereira and Rosaria de F.S.M. Russo. Design thinking integrated in agile software development: A systematic literature review. Procedia Computer Science, 138:775 – 782, 2018.
-  Franziska Dobrigkeit and Danielly de Paula. Design thinking in practice: understanding manifestations of design thinking in software engineering. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering - ESEC/FSE 2019, pages 1059–1069. ACM Press, 2019.
-  Pete Deemer, Gabrielle Benefield, Craig Larman, and Bas Vodde. The Scrum Primer. InfoQueue, version 2 edition, 2012.
-  Bertrand Meyer. Agile! The Good, the Hype and the Ugly. Springer International Publishing, Cham, 1 edition, 2014.
-  Norman L Kerth. The ritual of retrospectives: how to maximize group learning by understanding past projects. Software Testing & Quality Engineering, 2(5):53–57, 2000.
-  Derby Esther and Diana Larsen. Agile retrospectives: Making Good Teams Great. Pragmatic Bookshelf, 2006.
-  Paul Ralph. Possible core theories for software engineering. In 2013 2nd SEMAT Workshop on a General Theory of Software Engineering (GTSE), pages 35–38. IEEE, may 2013.
-  Tim Brown. Design thinking. Harvard Business Review, 86(6):84, 2008.
-  Katja Thoring and Roland M. Müller. Understanding design thinking: A process model based on method engineering. In DS 69: Proceedings of E&PDE 2011, the 13th International Conference on Engineering and Product Design Education, London, UK, 08.-09.09. 2011, 2011.
-  Anja Wölbling, Kira Krämer, Clemens N. Buss, Katrin Dribbisch, Peter LoBue, and Abraham Taherivand. Design Thinking: An Innovative Concept for Developing User-Centered Software. In Software for People: Fundamentals, Trends and Best Practices, Management for Professionals, pages 121–136. Springer Berlin Heidelberg, 2012.
-  Walter Brenner, Falk Uebernickel, and Thomas Abrell. Design thinking as mindset, process, and toolbox. In Walter Brenner and Falk Uebernickel, editors, Design Thinking for Innovation, pages 3–21. Springer International Publishing, 2016.
-  IDEO. IDEO Method Cards: 51 ways to inspire design. 2003.
-  Bella Martin and Bruce Hanington. Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers, 2012.
-  Vijay Kumar. 101 design methods: A structured approach for driving innovation in your organization. John Wiley & Sons, 2012.
-  Lisa Carlgren, Maria Elmquist, and Ingo Rauth. Design Thinking: Exploring Values and Effects from an Innovation Capability Perspective. The Design Journal, 17(3):403–423, September 2014.
-  Tilmann Lindberg, Eva Köppen, Ingo Rauth, and Christoph Meinel. On the perception, adoption and implementation of design thinking in the IT industry. In Design thinking research, pages 229–240. Springer, 2012.
-  Tobias Hildenbrand and Johannes Meyer. Intertwining Lean and Design Thinking: Software Product Development from Empathy to Shipment. In Alexander Maedche, Achim Botzenhardt, and Ludwig Neer, editors, Software for People: Fundamentals, Trends and Best Practices, Management for Professionals, pages 217–237. Springer, 2012. 10.1007/978-3-642-31371-4_13.
-  Benjamin Grossman-Kahn and Ryan Rosensweig. Skip The Silver Bullet: Driving Innovation Through Small Bets And Diverse Practices. LEADING THROUGH DESIGN, pages 815–829, 2012.
-  Franziska Dobrigkeit, Danielly de Paula, and Matthias Uflacker. InnoDev - A Software Development Methodology Integrating Design Thinking, Scrum and Lean Startup. In Design Thinking - Research Looking Further: Design Thinking Beyond Solution-Fixation, Understanding Innovation, pages 199–228. Springer, 2018.
-  Bianca H. Ximenes, Isadora N. Alves, and Cristiano C. Araújo. Software project management combining agile, lean startup and design thinking. In Aaron Marcus, editor, Design, User Experience, and Usability: Design Discourse, volume 9186, pages 356–367. Springer International Publishing, 2015.
-  Kavitha Gurusamy, Narayanan Srinivasaraghavan, and Sisira Adikari. An integrated framework for design thinking and agile methods for digital transformation. In Aaron Marcus, editor, Design, User Experience, and Usability: Design Thinking and Methods, volume 9746, pages 34–42. Springer International Publishing, 2016.
-  Leonhard Glomann. Introducing ‘human-centered agile workflow’ (HCAW) – an agile conception and development process model. In Tareq Ahram and Christianne Falcão, editors, Advances in Usability and User Experience, volume 607, pages 646–655. Springer International Publishing, 2018.
-  Osama Sohaib, Hiralkumari Solanki, Navkiran Dhaliwa, Walayat Hussain, and Muhammad Asif. Integrating design thinking into extreme programming. Journal of Ambient Intelligence and Humanized Computing, 2018.
-  Tina Øvad Pedersen. UX Toolbox for Software Developers: Methods and Training. PhD thesis, Aalborg University, 2016.
-  Jeff Sutherland and Ken Schwaber. The Scrum Papers: Nuts, Bolts, and Origins of an Agile Process. Technical report, Scrum, Inc, 2007.
-  Henrik Kniberg. Scrum and XP From the Trenches. C4Media, 2nd edition, 2015.
-  Tamara Carleton, William Cockayne, and Antti-Jussi Tahvanainen. Playbook for Strategic Foresight and Innovation -A hands-on guide for modeling, designing, and leading your company’s next radical innovation. Stanford, 2013.
-  Julie Verity and Kevin McCullagh. Design Thinking. In The New Strategic Landscape, volume 37, pages 52–68. Palgrave Macmillan UK, London, 2012.
-  Jon Kolko. The divisiveness of design thinking. Interactions, 25(3):28–34, apr 2018.
-  Jeff Sutherland. Inventing and reinventing scrum in five companies. Cutter IT journal, 14:5–11, 2001.