Agility is responsiveness to change: An essential definition

09/22/2019
by   Lucas Gren, et al.
Volvo
Saab
0

There is some ambiguity of what agile means in both research and practice. Authors have suggested a diversity of different definitions, and many practitioners have their own idea of what it is. It is, therefore, difficult to interpret what agile really is. The concept, however, exists in its implementation through agile practices. In this paper, we draw on our experience from several agile transformations and argue that adopting an agile approach boils down to being responsive to change. That is the core purpose of any agile transformation. To support this claim, we relate agile principles, practices, and the agile manifesto to this core definition. Drawing on our practices of organizational changes in software companies, we also show that many misunderstandings about agility can be avoided through this definition. That, in turn, increases the likelihood of successful agile transformations.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

10/17/2017

Beneficial and Harmful Agile Practices for Product Quality

There is the widespread belief that Agile neglects the product quality. ...
04/05/2019

Using the agile adoption framework to assess agility and guide improvements

Agility implies a set of principles that need to be followed in order to...
11/17/2021

The Integrated List of Agile Practices – A Tertiary Study

Context: Companies adapt agile methods, practices or artifacts for their...
07/24/2019

Agile Transformation: A Summary and Research Agenda from the First International Workshop

Organisations are upscaling their use of agile. Agile ways of working ar...
04/05/2019

Agile Process Consultation -- An Applied Psychology Approach to Agility

An agile change effort in an organization needs to be understood in rela...
11/10/2017

"Doing" Agile versus "Being" Agile. Empirical Results from 250+ Projects

In numerous occasions Agile practitioners have warned about the negative...
05/29/2020

Design of Transformation Initiatives Implementing Organisational Agility – An Empirical Study

This study uses 125 responses from companies of all sizes headquartered ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Why agile software development is misunderstood

We hardly need to provide an introduction to agility in the way we’ve done in all our research papers. Such an introduction would probably include the agile manifesto, its principles, and some descriptions of how the existing agile practices try to implement the principles. Agile has become a well-known concept. The concept, however, exists only in its implementation instances

(Laanti et al., 2013) and in success factors that vary (Chow & Cao, 2008).

In this paper, we aim to define the core of the agile approach drawing on our collective experiences of six agile transformations. These transformations were conducted in large (more than 5000 employees) organizations developing complex systems requiring multi-team collaboration (i.e. a scaled agile approach). The software systems developed by these companies were, however, not the sole delivery, but an essential component of a larger product. The experiences were collected during a period of over ten years.

Laanti et al. (2013) list all the definition of agility up until 2013 in research and they range from describing effectiveness, ability to steer, rule-base, people, communication, speed, flexibility, responsiveness, empowerment, change, feedback, value, delivery, innovations, adaptability, collaborative, iterative development, self-organization, light-weight process, cost-conscious, customer-driven, strategic, conceptual framework, and so on and so forth. Even the principles of the agile manifesto spans from customer satisfaction, continuous delivery, value, early delivery, adaptability, competitiveness, customer benefit, collaboration, motivated individuals, good environment, support, trust, efficiency, communication, progress measurement, sustainability, people, technical excellence, simplicity, optimization of work, self-organization, built in improvement of efficiency and behavior, and so on and so forth.

According to our observations, practitioners are confused about what agile is. The agile definitions comprise all the studies done in organizational and social psychology, management, and engineering research in the last century, but without any references to these results. This is a pity because the agile ways-of-working work, and when companies see other companies succeeding with agile transformations, they too want to get on that (agile release) train.

Some define agile by comparing it to waterfall-like and plan-driven development methods. In a rigid structure where one phase of the large project needs to be finished before the next phase can start, being agile seems difficult, which it also is. However, many companies, and especially start-ups, had projects that were different even before agile got famous, and it’s cumbersome to define a concept only by comparing to something else, we argue.

We also argue that most, if not all, organizations tailor the agile construct, meaning that they interpreted the definition so that it makes sense to their context. Still, this type of activity often focuses on process- and technology-related aspects, ignoring essential factors related to organizational values and social norms. So, what is at the core of agile and how is it different from how companies organized work before?

Other practitioners (in our experience, mainly managers) resort to Wikipedia to obtain a first idea of something new to them. Wikipedia cites Collier (2011) who defines agile software development as “an approach to software development under which requirements and solutions evolve through the collaborative effort of self-organizing and cross-functional teams and their customer(s)/end user(s).” It continues by citing the Agile Alliance web-page (https://www.agilealliance.org/agile101/) where agility is defined as “the ability to create and respond to change in order to succeed in an uncertain and turbulent environment.” We argue the latter is the best one so far, but that it could be shortened to “responsiveness to change.” We’ll now explain why.

2 Defining the core of agile

Without a common definition of what being agile means, their interpretation may vary between various organizational units, which could cause miscommunication, confusion, and even conflict. If the construct is defined too broad, the individual team and organizational units will create their own, more narrow, interpretations based on their own needs, goals, and experiences, which, in turn, creates a misaligned organization.

Moreover, the problem with dumping all aspects of modern organizational psychology into an approach is that too many employees in the transitioning organizations will be confused. They will not, even after a long time, grasp, what we consider to be, the true meaning of agility and how it differs from what they used to do, and how that should affect their working methods.

Furthermore, agile has become a hollow and general concept, and organizations affirm that they have a good (i.e. agile) enterprise or that they do good (i.e. agile) software development. We argue that this might cause organizations to adopt agile methods for the wrong reason, meaning that they do it because they have to, without in-depth insights into what it means. They want to be classified as an agile company, but they might not be fully committed to conducing the necessary changes.

Studies that look at practices have, for example, a hard time distinguishing between agile and lean (Petersen, 2011). There are a lot of methods and practices on top of these two paradigms. The core of lean relates efficiency (i.e. doing things right), which is about removing waste while sustaining the same productivity (Womack & Jones, 2003). Agile, however, relates to effectiveness – doing the right things. We, therefore, argue that agile is all about responsiveness to change. In the following section, we will relate most of the agile principles and methods to that core definition. We have observed that by focusing on this simplified core statement, companies are more likely to have a successful agile transformation. Such an organizational change requires much energy and can cause stress. Companies thus need to keep their focus on what matters, i.e. what they want to achieve.

3 How every other aspect of agility relates to the core

3.1 The agile values and principles

We will start with the agile manifesto and work our way through the agile principles and show how everything in the agile concept can be related to responsiveness to change. We’ll use the updated version of the manifesto created by Henrik Kniberg on his blog (http://blog.crisp .se/author/henrikkniberg) where he replaced some software-specific words with more general equivalents:

  • Individuals and interactions over processes and tools.

  • Working solutions over comprehensive documentation.

  • Customer collaboration over contract negotiation.

  • Responding to feedback over following a plan.

Processes and tools need to be developed, and rigid, if they are to be accepted by employees (Beer et al., 1990). According to our experience, the reason for focusing on individuals and interactions over processes and tools is, therfore, to increase their responsiveness to change.

Extensive and overly detailed documentation takes a long time to produce, thereby reducing the flexibility of the software by increasing the cost. Additionally, documentation is often more of a formal requirement that adds little of customer value. The reason for focusing on working solutions over comprehensive documentation is thus to elevate the organization’s ability to adapt.

Contract negotiations are often governed by a political agenda which makes contracts between parties static and inflexible. Organizations must thus focus on customer collaboration over contract negotiation to facilitate responsiveness to change. Following constant plans in a rapidly changing world are not in line with satisfying customers’ changing needs. Companies need to recognize that, the development of software is a knowledge-building activity in which customers and developers know more tomorrow than they do today. Creating plans and requirement specifications upfront and expecting them to remain constant throughout the development project is a utopia. Therefore, the reason for focusing on responding to feedback over following a plan is to increase the responsiveness to change.

In an attempt of making the manifesto more concrete, the authors connected a set of twelve principles to their manifesto that have been reviewed by Williams (2012). To align the agile principles with the changes to the manifesto above, we have replaced the word software with solutions.

  1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable solutions. — In order to create valuable solutions, we need to know what the customer values, and since that can change fast we need to deliver early and continuously.

  2. Welcome changing requirements at the start of each iteration, even late in development; agile processes harness change for the customer’s competitive advantage. — By welcoming changes in requirements we make sure that we know when the customer changed her mind.

  3. Deliver working solutions frequently, from a couple of weeks to a couple of months, with a preference for the shorter time-scale. — In order to check if the world has changed (i.e. the customer or end-user), we need to deliver frequently otherwise we will not know the most recent changes.

  4. The whole team, from business people through testers, must communicate and collaboratively work together throughout the project. — Values is not only the best technical solution, therefore we need to figure out what adds the most value since this could as well be something non-technical (like, for example, business-related).

  5. Build projects around empowered, motivated individuals with a shared vision of success; give them the environment and support they need, clear their external obstacles, and trust them to get the job done. — With a shared vision, the employees can work in the same direction towards the same goal, which is hopefully aligned with adding the most value. Empowered individuals also dare to respond to change, and all the unnecessary and slow hierarchies can then be avoided.

  6. The most efficient, effective method for conveying information to and within a development team is through synchronous communication; important decisions are documented so [they] are not forgotten. — Synchronous communication enables responsiveness to change since information then doesn’t get stuck somewhere.

  7. Valuable, high-quality solutions is the primary measure of progress at the end of each short time-boxed iteration. — Valuable solutions must be useful solutions, which implies that we know what adds values to the customer today.

  8. Agile processes promote sustainable development. The whole team should be able to maintain a reasonable work pace that includes dedicated time for exploration, visioning, re-factoring, and obtaining and responding to feedback. — It is not possible to figure out what adds values if people are not given the opportunity to reflect, listen to feedback, and explore.

  9. Continuous attention to technical excellence and good design enhances agility. — Good designs lessens the technical debt, which increases the ability to respond to change.

  10. Simplicity –the art of maximizing the amount of work not done– is essential. — We can only simplify our solutions if we know that they add value, otherwise we simplify waste.

  11. The best architectures, requirements, and designs emerge from self-organizing teams guided by a vision for product release. — We need self-organizing teams because they are the only ones that can respond to change fast enough. We cannot afford to have slow decision hierarchies, and the people working on that specific solution will be able to make the most informed decision. The trick is to set the decision-making frames for teams so that they know when and which mistakes are fine to make.

  12. With each iteration, the team candidly reflects on the success of the project, feedback, and how to be more effective, then tunes and adjusts its plans and behavior accordingly. — Reflecting on success will increase the likelihood of success in the proceeding projects. The lean aspect of efficiency also comes in here, because lean and agile are complementary concept and when we have figured out what adds value we want to deliver that efficiently. However, we want to check that continuously in order to respond to change. Since agile and lean go hand-in-hand large-scale agile frameworks (like SAFe) use the term lean/agile mindset and not just an agile mindset.

3.2 Other known definitions from the trenches

In addition to these aspects of agility, we have essentially heard four more reoccurring definitions of what agile is by practitioners. The first one is that agile is to have less handovers in the organization. Removing silos (if they exist, and they most often do) is of course a key part in being able to respond to change, but having less handovers and follow the value flow farther in the development of, e.g. a component, is only to be able to respond to change, it is not the end goal.

The second aspect is that small batch sizes is the definition of agile. Having small batch sizes is also, we argue, a means to enable responsiveness to change. With large deliveries, the world has changed too much between reality checks of what we think adds value to the customer.

The third definition is that agility is to time box deliveries. The time boxing is only to assure short feedback loops to assess customer value, which also a way to be responsive to change, i.e. the goal is not to time box deliveries per se. Progress is measured as implemented product features and not process progress because that is how we get feedback on customer value. We do not learn about changing customer needs through internal processes improvement since optimizing internal processes is lean and not agile, i.e. aspects of efficiency and not effectiveness.

We have always had a problem with using the catch phrase “fail fast fix fast,” which we have heard many times in companies we have worked with. There is no value in failing, and we could be accurate in our first try, but if that happens, agility is not an ingredient. However, we need a flexible (i.e. agile) development system to capture if our rapidly built initial solution was valuable or not and be able to respond to changes, and this is implemented by using feedback loops that are as short as possible.

The fourth definition is that agile is to deliver value, at least to a significant part. If we equal value delivery to being the definition of agile, we mix up agile and lean again, which, from what we have seen, is confusing to companies. Not changing a product on the way and only delivering value according to a business model implies that the requirements are stable over time and we can focus on removing waste in the process. We do not believe this has anything to do with agility, and practitioners are very much helped by making this distinction between the two.

By defining agility as responsiveness to change, we define the end goal and see all other aspects as means to that end. By doing this, it is also easy to understand that teams could be agile without anyone on the team having ever heard that term before. There are many ways of building a structure that can respond to change, but we must not confuse what we believe leads to agility and what it is. Structures and processes that lead to responsiveness to change have always existed and we must recognize those as in line with an agile approach and not something that needs to be replaced my other famous agile practices. The business of packaging agility and selling it to companies has, however, provided a case for why such approaches should be large-scale. Changing requirements has always been a challenge in software development (Buschmann, 2009). In agile approaches, by contrast, the mindset is to appreciate change. In other words, software engineers have come to accept the intrinsic flexibility of their own craft and finally adopted the working methods accordingly. What is interesting is that the increasingly fast changing needs in hardware development has also made hardware to behave more and more like software, but with sometimes longer feedback loops. What is very interesting is that software is now also a core part of much of the hardware development and, for example crash tests, can have very short feedback loops with the use of computer simulations. Making a waterproof distinction between hardware and software development makes less and less sense according to what we have seen lately in companies that do both.

4 An agile vision

4.1 The effects of blending other aspect into an agile vision

It is easy to make the mistake of picking from all the concepts bundled into “agility” when creating a vision for an agile transformation. Choosing whatever solutions to real problems that we hope that agile will solve is natural. We, however, are not doing ourselves any favors through this. Adding a range of concepts to the agile transformation will only confuse the employees. One example of this is to add both responsiveness and speed to define agility when initiating organizational agile transformations. The risk is then that some employees will think that agility is speed, which we have seen many times. Speed is necessary in the agile space, but only because we quickly want to update our assumption about the world and if it has changed recently. A company might want faster internal development and remove waste, but that should not be a part of the agile transformation then.

If we confuse agility for high internal production speed and start optimizing for that, we will soon make decisions based on that speed has value in itself (see Figure 1). We need to deploy “sub-optimal” solutions to collect feedback fast for us to be agile and learn what adds value, but value must have priority over speed, if we want to be agile. As we mentioned previously, agility is in relation to effectiveness (doing the right things) and lean is in relation to efficiency (doing things right). These core differences need to be made explicit when implementing the approaches. Just because lean and agile share practices, for example continuous improvement, does not imply that the core ideas behind this practice fully overlap. Instead, we view the shared practices as layers on top of, but separate from, the core ideas.

Figure 1: Optimizing speed instead of responsiveness to change.

4.2 Other confusions caused by the lack of an agile definition

It’s confusing that, for example, SAFe is build on four different paradigms, which includes Agile principles and methods, Lean and systems thinking, product development flow practices, and Lean processes (https://www.scaledagileframework.com/safe-lean-agile-principles/), but many practitioners think the SAFe framework is agility and react negatively when for example value stream mapping is introduced as an agile practice when they used it for many years in lean. We have also met companies that choose between agile and SAFe principles, which has devastating effects since it causes more confusion between agile and lean. It is also illogical since SAFe is partially built on the agile principles. One of the most severe negative effects of this choice is that the SAFe principles do not have any guidance for the mechanisms that need to be in place in the agile teams.

Another common resistance lately has been that agile is a software process and does not work for hardware development. If we define agility as responsiveness to change we can circumvent that resistance and also acknowledge what parts of existing hardware development processes that map very well into the agile idea. Many large companies have software and hardware developers even in the same team so the collaboration and the definition of a common improvement purpose is essential. If the opposite is done, i.e. to implement the SAFe framework top-down on hardware teams, this tends to trigger a lot of sound resistance. Again, if focus is on responsiveness to change, more people understand that focusing on structure as the end goal, that is, the ceremonies, is not what should be in focus in the agile transformation. We can then also avoid throwing the baby out with the bath water and lose context-specific knowledge essential to the companies’ survival and are in line with an ability to respond to change.

We have also met many teams and managers who do not understand how building teams maps into the agile idea of implementing features from a backlog. This is, of course, devastating since self-organizing teams with a mandate to make decisions within their own expertise is an absolute key to obtaining responsiveness to change (as also stated in the agile principle 11).

As mentioned, a common misconception of agility in practice is that it only means a team-based workplace. However, teams are a core part of implementing responsiveness to change, but it is not the definition of agility either. We have previously shown that the dynamics of an ideal agile team largely overlap with that of a mature team from a psychological perspective (Gren et al., 2017). In our experience, the structural part is the easy part for an agile transformation, but if a company focuses more on building mature teams from a psychological perspective, the teams’ responsiveness to change increases dramatically.

One of the most severe impediments we have seen in getting the teams to mature, is hierarchical “command and control” leadership. But the idea that an agile leader should only have a process facilitating leadership style does not make sense either. What makes sense is to train managers in that they are there for the teams and not the other way around, because leadership in old and large organization are often without such a mindset. The problem is that a more consultative leadership style is only appropriate for mature teams where they have integrated well over time, which means they must have navigated through forming, storming, and norming phases (Tuckman & Jensen, 1977). A completely newly formed team where no members have ever met before, a more laissez-faire leadership approach could be devastating if no natural leader steps in at an early point in time. Such teams are concerned with dependency and inclusion, and therefore need directive and clear leadership that provides the team with structure and direction to move forward (Wheelan et al., 2003). It’s critical, though, that the directive leadership stops when the team is mature enough to self-organize and share the leadership function (i.e. the initiative) among its members. The agile transformation, therefore, needs to focus on guiding teams towards high levels of team maturity so that they can act independently within the set frames, and leaders must serve the teams with what they need in order to become effective. Without helping teams to mature over time, they will have difficulty building an ability to respond to changes in their development ecosystem.

We have been involved in training more than 200 teams in team development using the model by Wheelan et al. (2003), and also trained more than 250 team managers in supporting teams differently depending on their maturity. Some of the effects we have seen are that teams reorganize themselves across departments because the realize that they can deliver more value in that way, managers back off and help each other to back off when teams are ready to lead themselves, the teams get aware of subgroups within the teams without a common goal, and they understand how new members need to be on-boarded into their teams from a collaborative perspective. One big issue that remains is that some managers fall back into old behavior when the pressure increases on the teams’ deliveries. The cultural change is, of course, what takes time and demands perseverance.

Teams must be given the opportunity to become mature before they can be expected to self-organize and efficiently respond to change, which is not done in a day. In connection to this, realistic expectations on teams’ performance must also be accounted for. According to Wheelan et al. (2003), reaching the most mature stage takes time. Their study on 114 teams showed that it takes, on average, 6.5 months to transition from the initial state (stage I) to the final stage (stage IV). This implies that organizations should not expect agile transformations to produce self-organized, empowered, agile, and high performing teams for several months, but it’s worth the wait and effort. Building trust among team members simply takes time. If we want highly intelligent agile teams that are responsive to change, we must also enable them to mature, which of course also implies that they need to be as stable as possible over time.

4.3 The effects of defining agility as responsiveness to change

By defining the construct as responsiveness to change, our experience is that the agile transformations get easier. It gives direction and clarity by reducing the confusion in the organization. Those organizations that make the concept concrete (i.e. by defining what they must focus on to obtain responsiveness to change) have a more natural way of aligning the company and give a clear direction to the transition. This is particularly important in companies that use a scaled agile approach in which multiple teams need to collaborate. If the interpretation of the agile transformation needed is left to each team, the risk is high that the teams’ interpretations differ, making it hard for them to collaborate efficiently.

By defining the core of the agile transformation, employees and managers alike can measure any changes against their potential increase in responsiveness, asking themselves “Does this increase our responsiveness to change?” If the answer is yes, they have a case and can defend that decision. We have met many teams that think the agile transformation only brought more hierarchies of decision-making and more waste to their process. By contextualizing the agile transformation as changes towards more responsiveness to change for the entire development system, it becomes much clearer why some teams might have to adjust their own process to obtain more overall agility even if it might hamper their own deliveries somewhat. Focusing on responsiveness to change also prevents them from thinking that agile is a set of concrete practices or simply to organize work in teams. There could be other reasons for using agile practices and arrange work in teams, such as obtaining transparency and increase job satisfaction. There could also be reason for other changes, like mapping value flows, however, the reasons behind organizational changes should be honest and explicit, otherwise people will notice and remain skeptical and confused.

We have recently tried deploying the agile definition as responsiveness to change to around 50 agile teams (including managers) in Sweden and 7 in China. Both team members and managers have found it very helpful to work with this definitions of agility since it makes the overall purpose of the agile transformation clear. Also, since the framework used in this particular case is built on both lean and agile ideas, they found it very useful to separate the two. Companies with a history of implementing lean, for e.g. manufacturing, need to know what is new in the agile idea. They have always focused on delivering value, high speed, self-organizing teams, etc. so it really helps in convincing them about the missing piece that agile adds.

5 Conclusion

This paper set out to define agility, giving clarity and direction to agile transformations. Through research and practical experience, we have found that agile transformations would be easier if agility was defined as responsiveness to change. This finding is an essential contribution to both researchers and practitioners in the agile space since they need to agree on the definition of the concept to efficiently study or apply it.

References

  • Beer et al. (1990) Beer, M., Eisenstat, R. A., & Spector, B. (1990). Why change programs don’t produce change. Harvard Business Review, 1.
  • Buschmann (2009) Buschmann, F. (2009). Learning from failure, part 1: Scoping and requirements woes. IEEE software, 26, 68–69.
  • Chow & Cao (2008) Chow, T., & Cao, D.-B. (2008). A survey study of critical success factors in agile software projects. Journal of systems and software, 81, 961–971.
  • Collier (2011) Collier, K. W. (2011). Agile analytics: A value-driven approach to business intelligence and data warehousing. Upper Saddle River, NJ: Addison-Wesley.
  • Gren et al. (2017) Gren, L., Torkar, R., & Feldt, R. (2017). Group development and group maturity when building agile teams: A qualitative and quantitative investigation at eight large companies. The Journal of Systems and Software, 124, 104—–119.
  • Laanti et al. (2013) Laanti, M., Similä, J., & Abrahamsson, P. (2013). Definitions of agile software development and agility. In Systems, Software and Services Process Improvement (pp. 247–258). Springer.
  • Petersen (2011) Petersen, K. (2011). Is lean agile and agile lean? A comparison between two software development paradigms. In A. H. Dogru, & V. Bicer (Eds.), Modern software engineering concepts and practices: Advanced approaches (pp. 19–46). Hershey, PA, US: IGI Global.
  • Tuckman & Jensen (1977) Tuckman, B., & Jensen, M. (1977). Stages of small-group development revisited. Group & Organization Management, 2, 419–427.
  • Wheelan et al. (2003) Wheelan, S., Davidson, B., & Tilin, F. (2003). Group development across time: Reality or illusion? Small group research, 34, 223–245.
  • Williams (2012) Williams, L. (2012). What agile teams think of agile principles. Communications of the ACM, 55, 71–76.
  • Womack & Jones (2003) Womack, J. P., & Jones, D. T. (2003). Lean thinking: Banish waste and create wealth in your corporation. London: Free Press Business.