Enterprise IT organizations need Lean to navigate the path to DevOps

Source – sdtimes.com

One can easily get lost in the great many resources on DevOps and digital transformations, yet still fail to come away with a concrete idea of what the journey actually looks like. We know there’s a better way to build and deliver software and what in theory that looks like, but in practice it’s not so simple. Change is difficult, even more so when the path ahead is unclear.

But time is of the essence – fear of evolution and innovation procrastination will kill your business. The ability to stay competitive in the market means using DevOps to create an efficient and effective software delivery capability. Fortunately, we can look to Lean to obtain the clarity and direction that is needed to understand what makes a successful transformation towards a more efficient model of product development.

If we want to understand Lean unambiguously, we can start with this: Lean prioritizes flow efficiency over resource efficiency. The discipline focuses on optimizing the connections between resources — how they integrate with each other — to achieve the smooth, end-to-end flow of work that makes a system efficient and effective at creating value.

IT has traditionally organized and managed itself explicitly in terms of resources, using frameworks like IT service management and project management to deliver value from resources that are functionally separated.

When IT customers are internal and the value being delivered is stability, resilience and consistency, both service management and project management are very effective.

Such frameworks are resource efficient, and they need to be because they manage shared services that are consumed by the rest of the value-creating organization. The functionally separated distribution of work is useful, as each function uses different methods and requirements to deliver on their commitments to the business.

On the other hand, the goal of software delivery is more singular: to delight stakeholders by delivering high-quality, full-featured software. In this case, the traditional IT functions need to be reorganized to build and deliver software end-to-end as a single, flow-efficient system. Among other things, this will reduce waste, eliminate bottlenecks, increase velocity and promote collaboration. This is DevOps.

In a software-defined world where the fundamental competitive advantage of a product or service is the quality of the code that powers it, we need to focus this: designing our organization around a flow-efficient software development lifecycle.

The goal of the DevOps transformation is therefore to transition enterprise IT off its myopic focus on resource-efficient functional silos and onto the flow-efficient value stream. This means breaking down the barriers created by islands within an organization to build a new culture around collaboration experimentation and feedback.

The Efficiency Matrix, a concept borrowed from This Is Lean (2013), is a useful abstraction for understanding this journey:

A: Perceived Starting Position

This is where enterprise IT organizations see themselves sitting at the start: high-resource efficiency, low-flow efficiency.

Traditional resources like teams, tools and processes are thought to be optimized locally, allowing them to maximize their ability to deliver on their own goals. Organizational leadership is aware that there are significant shortcomings in using these functions to deliver software, but there is a reluctance to embark on a transformation.

And let’s assume the status quo is working to some degree. Why take the risk of changing existing work structures when new ways of working might not even deliver on their promises? Maybe there is still some faith left in the existing structures. Why bet against the ability to just fix the resource-based approach so it works for software? In the worst case, maybe the organization has already tried to transform and failed due to lack of leadership or an inability to break down functional silos.

The truth is, though, that organizations using traditional IT functional silos to deliver software are already suffering from enormous waste, and there is no fix for this.

Actual time spent adding value to the product is extremely low due to multiple handoffs, broken workflows, unnecessary approval processes and constant rework due to poor communication and a lack of knowledge sharing. Even if a finished product can make it out of the door, it is likely loaded with defects, vulnerabilities and technical debt.

Here we have a key lesson from Lean — the poor quality of the connections between functional silos forms a natural impediment to any kind of product development.

The transformation process therefore starts with organizational leadership realizing and recognizing that they are not at point A on the Efficiency Matrix. Rather, they are already deep into the wasteland, and the only way out is through change.

B: Actual Starting Position

This is the reality of where most organizations are starting – their resources are unable to deliver software without extreme waste due to problems with, for example, handoffs and communication, and their goals are incompatible with the velocity and responsiveness needed to develop software.

The first step in moving forward with a transformation is therefore an awakening – the realization that the status quo is inefficient in all respects. Coming to terms with the fact that products cannot be built efficiently and effectively with resource-focused structures is fundamental Lean thinking.

It is perhaps the most difficult part of the whole process because it means changing the perception of leadership, convincing people that they are not in fact at Point A and that it is worth taking the risk to transform.

This realization is often obtained by validating the systemic benefits of DevOps with a single team or project to experiment and prove out the principles. Once you can show the techniques at work in a corner of your organization, you can then start talking about rolling out the approach on a large scale. For this, a transformational leader is needed to clearly demonstrate the benefits to key decision-makers, obtain the all-important executive buy-in and drive major change.

C: Increasing Flow Efficiency

Path C represents the crucial movement towards flow efficiency, and the goal here is clear – transform the software delivery capability into an integrated system that permits free flow of information.

There is no blueprint for transformation success in achieving this goal; however there are some significant signposts and key guidelines that create a foundation:

• Transformational leadership motivating change
• A culture of experimentation and collaboration
• Horizontally structured teams with shared responsibilities
• Extended reach of Agile methodologies
• Deep integration of the entire toolchain

The flow-efficient organization is designed to be extremely well integrated to allow resources to be used effectively as a system.

This is the fundamental idea and hallmark of Lean thinking that we have seen drive so many successful transformations in other sectors; once we are effective in obtaining flow efficiency to achieve system-wide integration, then we can start pulling levers and twisting knobs on the individual resources.

D: Increasing Resource Efficiency

In path D, the organization begins optimizing resources, but, importantly, as a function of systemic flow efficiency.

A key outcome of flow efficiency is the ability to intelligently develop cross-cutting data collection and reporting capabilities to provide fast feedback on system-wide effectiveness. This provides the traceability needed to identify bottlenecks and power continuous improvement over the long term.

Now the organization can begin implementing typical DevOps techniques such as continuous integration, continuous delivery, test automation, WIP limits and so on, with a clear view for how these changes affect the overall flow of the system.

Interestingly, if the integrations for flow efficiency are executed correctly, a “collect everything” approach is not necessary. Remember what W. Edwards Deming said, “Just because you can measure everything doesn’t mean that you should.” Using the visibility gained from your integrations, it is clear where to monitor and collect metrics that are highly targeted and verifiably useful for decision making.

E: Final State

The final state is not a resting state – the organization will continually come up against new limitations, customer requirements, organizational bottlenecks and market forces. Having flow efficiency allows the business to calibrate resources dynamically to respond quickly to emerging issues.

The interesting thing about the final state is that resource efficiency is not maximized – capacity is maintained to give the organization the responsiveness required to deal changes in customer requirements and market forces, defects and vulnerabilities, technical debt and other unexpected or unplanned work.

This highlights why an organization undertakes a DevOps transformation in the first place; the flow-efficient system is dynamic, responsive and able to adapt to change. This is where resources are able to deliver the most value to an organization.

Conclusion

At its core, a DevOps transformation means changing the way an organization delivers value – i.e. making the choice to design an organization around system-level flow efficiency. This means:

• Integrating the software development lifecycle to obtain flow efficiency
• Calibrating resources in respect of the emerging flow requirements
• Continuously monitor and rebalance as necessary

Flow efficiency is the ‘True North’ in the journey towards building and managing a lean organization as a single system, one that continuously delivers the right value to customers. With this direction organizations can forge ahead confidently with a transformation strategy for developing a first-class software delivery capability.