Six Ways Agile Can Turn Static

Source – infoq.com

Digital transformation requires IT organizations to become business enablers that deliver differentiating customer experiences that provide a return on investment.  To keep pace with innovation, and deliver immediate results many IT teams adopt agile development methodologies where release cadences are measured in days, not quarters and quality is guaranteed.

This may be the Holy Grail, but this goal isn’t always possible. Idealistically speaking, agile development has all the right elements but it isn’t suitable for every project. Let’s consider how it works in the best case scenario.

Agile development accelerates the delivery of initial business value, and through a process of continuous planning and feedback.  As a result of this iterative planning and feedback loop, teams are able to continuously align the delivered software with desired business needs, easily adapting to changing requirements throughout the process. Measuring and evaluating status is based on accurate visibility into the actual progress of projects through all of its stages with all of the project stakeholders. As a result of following an agile process, at the conclusion of a project the software system addresses business and customer needs better.

Everything sounds logical, and reasonable, but often the reality of agile projects can be a bit different.  Agile methodologies don’t always deliver the expected results.

Here are six common ways agile projects can fall flat.

1. Unsustainable system architecture – The Agile Manifesto states that “good design and technical excellence enhance agility” and there is a strong push for “adaptive architectures”.  The trick to staying agile is having just enough design up front.  But the problem is knowing how much design is enough.   If the design process is short changed, without a formal design process, code can be layered on top of older code and until the end result, is a design that’s inflexible.  Unlike the traditional waterfall architecture where system architecture is well defined upfront, and development is a one-time activity with definite start and end dates. The agile software architecture is an ongoing process that enables developers to apply changes in the architecture, if necessary, on an ongoing basis. The design can be coherent during a sprint planning meeting, but it may lack a formal architectural model that results in long term sustainability.
2. Limitations of existing tools – Agile methodologies are often limited by existing tools.  Today’s Enterprise Application Lifecycle Management (“ALM”) market is dominated by monolithic, on premise solutions that were designed with a “waterfall” delivery approach in mind and lack key elements of agile delivery methodology. While the software development industry has moved on to a best of breed approach with agile tools such as Jenkins, VersionOne, and  RallyDev these tools weren’t designed for the needs of the Enterprise IT organization. They were designed to be used by professional software engineers and not business users who are unwilling, unable and shouldn’t be required to attend training classes to learn how to provide feedback from performing functional testing.  In addition, these systems can also be expensive to purchase and maintain because they include too many features that have excessive and unnecessary hardware and computing requirements.
3. Culture gap – Some developers who are accustomed to working autonomously may find that all the collaboration required with agile development slows them down. Many will resist changing their work style to adjust to a highly collaborative methodology if they are accustomed a great deal of freedom.  Highly innovative and bright individuals often just want you to give them the tools they need, then let them do what they do best.  Although eventually everyone on agile project teams will need to be more flexible. Gartner’s 2016 Magic Quadrant for Application Testing Services estimated that by 2020, 60% of testing resources will need to have a combination of testing skills, application development skills, and business process skills or industry skills. Given the level of change that happens around projects today the idea of the “lone wolf” developer sitting somewhere and not talking to others doesn’t make sense – collaboration is a key success factor.  There is lots of evidence that more diverse teams produce better results, so isn’t it incumbent on the individuals to “get with the program” and build some social skills?
4. Difficulty scaling up – Agile is a methodology that traditionally has been popular with small and medium size organizations, but recently has become more mainstream in large enterprises.  Compared to small projects, which are ideal for agile development, larger ones require additional coordination. A particular problem in applying agile to larger projects is how to handle inter-team coordination. Large-scale agile involves additional concerns in interfacing with other organizational units, such as human resources, marketing and sales, and product management. Because the methodology is largely dependent on close working relationships and lots of feedback from users, as the project grows the resulting communication flows multiply exponentially. In addition, as enterprises expand into new geographies, or merge with other organizations, projects are also dispersed throughout several teams at different locations.  Without a consistent approach across locations and collaboration tools with visibility into real time status of test cycles distributed over several development and testing teams, bottlenecks can occur which can result in project delays and cost overruns.
5. Not the right type of project – Traditional methods (like waterfall) work best on projects that are on both ends of the spectrum; clear and fixed requirements that are well established or a new technology or approach with no previous user base that can provide valid feedback.  Although truthfully, long projects are becoming less and less prevalent. The VUCA (volatility, uncertainty, complexity and ambiguity) nature of the business environment today means that 60+% of requirements will change in a 12 month project, so the “fixed” end of the spectrum is a myth.  There is no “clear and fixed requirements” in business systems today – technology changes rapidly, customer needs are emergent, legislation changes frequently and “I’ll know it when I see it” (IKIWISI – from Rapid Application Development, 1986) is the reality of user needs.  Project managers need to evaluate projects to analyze if a smaller number of developers working autonomously might be a more suitable and cost effective solution for projects that fall somewhere in the middle.   Projects that interface with several different user groups in a matrix management structure can also be unsuitable for agile methodologies.  For example with pyramid structures, it is often the case that the number of teams working together end up being larger than what is really needed for the project.  The problem gets even worse when those team members are partially assigned to projects as there is less commitment.
6. Bogged down by collaboration – To benefit most from agile methodologies software development team members need to see, feel, and hear what is occurring to users on a daily basis.  But the reality of most projects today is that the different team members are not in close proximity, or are not available for face to face communications. Relying on manual email communications can slow down communications and can be error prone.  Although my own experience with enterprise organizations shows that more than 50% of  organizations are using Microsoft office products like WORD or Excel or even pen and paper to manage aspects of application delivery like testing.  When teams from different physical locations, and often different geographies use the communication tools to capture all the different types of information and accelerate information flows, they often make improvements in both the process and the final outcome, making the teams stronger and more productive.  Tools prevent processes from being stuck because one test group is behind or a business user involved in user acceptance testing went on vacation without remembering to run procedures.  Since communication and coordination – both internally and with customers – is often difficult due to logistics, locations and timing, an end-to-end communication framework for agile team collaboration is essential to help people work together more effectively.

Agile development in the right circumstances enables organizations to release high quality software that changes rapidly to drive businesses forward.  It just doesn’t work all the time.  Success requires collaboration, transparency and real-time visibility into project risk and quality.

Consider user story mapping, mind mapping, and user story-splitting techniques to better break what look like large complex solutions into smaller, independent parts that can be quickly implemented to get quicker user feedback.  You want to look just far enough into the future to identify and do what you need to do to deliver value now and to enable the delivery of value in the future.

Keep the communication lines open and automate the communication of all feedback including test results, change orders, and retests to make things run smoothly.  At the same time admit the limitations of the methodology and if your experts insist on working independently, and bloated matrix management make all the feedback lines unmanageable, no shame and admitting Agile’s limitations and defaulting back to the old tried but true waterfall methodology.

In the end of the day, we are building products our customers can use to make them happy. Being able to frequently add new features based on their feedback makes them even happier. As a software customer, is there anything worse than investing in a product that doesn’t work, doesn’t do what I need it to do, and I don’t see a path forward for making it better. I’m willing to buy a first iteration product if I know it is going to get better over time. As a matter of fact, using agile methodologies, it can be fun seeing the product emerge as the development team continuously improves the product. Agile helps us build this kind of partnership with our customers, one where we are working together to get problems solved.

Despite the complexities of agile development methodologies, given the right conditions and tools, IT teams can implement agile development methodologies that move business forward and cement lasting loyal customer relationships.