How does Agile project management relate to leadership?

How can leaders focus on agile principles that will add most value, especially in less agile environments?

As organisations tackle the challenges of a rapidly changing business landscape, the principles that underpin agile project management are increasingly being seen as a route map for leadership more generally. But is it really that simple?

Our online leadership and management courses present an array of different methodologies for leadership that can be used together to transform the way we manage each other and our businesses.

The Agile Manifesto, conceived as a roadmap for changing times

Back in 2001, 17 top software practitioners spent a long weekend at a ski resort in Utah's Wasatch Mountains to discuss what constituted best practice when it comes to software development. they concluded that the term 'agile' best described the most successful approaches, and encapsulated their thinking in The Agile Manifesto, backed up by 12 core principles.

The Agile Manifesto was a reflection and summary of project-management processes and techniques developed specifically to meet the fast-moving world of turn-of-the-century software development. In some ways, it can be seen as just another iteration of the famous Toyota Production System of the 1950s, the first project-management system based on lean, team- and workflow- based approaches to project management that have been adopted and adapted ever since.

Can Agile project management be used more widely than in just tech?

But the manifesto has become an influential blueprint and mantra for any number of tech companies and start-ups, spawning a community and fanbase — and a backlash that, today, asks if agile is really all it’s cracked up to be.

While the story of The Agile Manifesto has largely played out in the software industry, commentators and researchers have also asked whether this most contemporary vision of agile project management can be adapted and used in business more widely. Just as agile project management has pitched itself against more traditional ‘waterfall’ techniques, leadership based on agile principles and values has been seen as a similar foil for traditional top-down, command-and-control, business- knows-best approaches to running organisations.

How can methods of Agile project management be used in more traditional sectors?

As these debates continue, with almost cult-like status afforded to a variety of alternative approaches and systems, how can leaders navigate the hype and focus on the principles that will add most value, especially in less obviously agile environments, such as traditional manufacturing or the public sector?

Agile project management is most often compared and contrasted with more traditional waterfall techniques and systems, such as Critical Path Analysis, Prince2, Six Sigma or the use of the humble Gantt chart. Waterfall- based systems follow a sequential, linear process, which consists of several discrete phases or a critical path. No phase begins until the prior phase is complete, and phases do not overlap. Proper planning and documentation are a must; project requirements and roles must be clear upfront, and everyone involved in a project must be well aware of those requirements.

Waterfall is about effective planning negating the risk of failure

Product definition and processes are stable. It’s a methodology based on the assumption that time spent at the beginning of a project outlining the design and requirements will allow the actual project to flow fast and smooth, like a waterfall, when it comes to implementation. Think Abraham Lincoln’s famous advice that, if he were given six hours to cut down a tree, he would spend the first four sharpening the axe.

Agile is about development as an iterative process

Like waterfall, agile project management is an approach rather than a system or technique in its own right, although plenty of structured project-management tools and systems — such as Scrum Kanban or Design Thinking — have agile principles at their core.

At its essence, agile is about developing a project as an iterative process rather than a pre-set plan. Execution is integrated with planning, so that working prototypes become a means of marking project progress and of testing — and further iteration.

Embracing change as a methodology is a pragmatic approach to leadership

Flexibility, change and adaptation are embraced rather than avoided. It involves a relentless focus on customers and, crucially, requires high levels of commitment and collaboration from motivated, self-regulating and cross-functional teams. Agile practitioners generally see the commercial environment as being far more complex and unpredictable than planning to cut down a tree. They are more likely to take heed of Mike Tyson’s sage warning that “everyone has a plan until they get punched in the mouth”.

It’s easy to see why more traditional waterfall-style command-and-control management structures are under pressure. Bureaucracy and hierarchy seem increasingly at odds with the fleet-of-foot world of tech giants such as Amazon or Google. Examples of companies that have failed to adapt and change — from Kodak to the big hotel and car companies that just didn’t seem to see Airbnb and electric cars coming — are legion.

Holacracy as Agile run wild

In response, the battle lines have been drawn. Consider, for example, the rise of holacracy, a radical, ‘no bosses’ organisational structure which takes the agile principle of self-managing teams to the extreme. Developed by software engineer Brian Robertson, its proponents, led by Tony Hsieh, CEO of Zappos, an online shoe and clothing retailer, remain bullish.

But another early adopter, the online publishing platform, Medium, dropped it after a tricky three-year experiment. Others simply view holacracy as agile run mad, a recipe for chaos, characterised by the worst kind of survival-of-the-fittest office politics and dehumanising processes.

When founder Evan Williams announced that Medium was abandoning holacracy, he cited, as the major cause, an obsession with process that was getting in the way of doing the work. It’s a criticism that’s also been levelled at leading agile project- management methodology, Scrum; the sense that it’s become as inflexible and hierarchical as any traditional waterfall approach.

What does agility actually mean?

As far back as 2015, one of the authors of The Agile Manifesto, Andy Hunt, bemoaned the fact that “the word ‘agile’ has become sloganized” leading, at best, to meaningless and half-hearted attempts at agile; at worst, to agile zealots who continue to “redouble their effort after they’ve forgotten their aim”. Adopting abstract agile concepts can be difficult; in the race to make sense of them, “agile methods themselves have not been agile”. The irony was not lost on Hunt.

Work smart, not hard - collaborate across internal silos

Stephen Denning, author of The Age of Agile, firmly believes that agile concepts have become an essential route map for management more generally. But for Denning, agile management is about much more than harnessing data and technology or learning lessons from software development. It implies a different approach to leadership and management, focused on generating “more value from less work”. He argues that, on their own, traditional hierarchical bureaucracies cannot respond to the complex problems that organisations are increasingly facing, challenges that require collaboration across internal silos and routine interaction with customers.

Instead, he has identified three common characteristics closely associated with organisations that have successfully embraced agile management.

3 characteristics of successful agile management

1. The Law of the Small Team: small teams working on small tasks in short, iterative work cycles delivering value to customers.

2. The Law of the Customer: a continuous focus on adding value for customers.

3. The Law of the Network: co-ordinated work within an interactive network.

Hierarchy should be defined by competence, not by authority

Crucially, that doesn’t mean that agile organisations are completely flat or non-hierarchical; top management still sets direction. That management is based on entirely different assumptions, a different mindset — in Denning’s words: “a hierarchy of competence, not a hierarchy of authority” — but it still exists. His agile management paradigm is a journey, not a destination. Companies never really ‘become’ agile: rather, an agile mindset involves “never-ending innovation” that requires “continuous commitment and leadership from management”.

It’s a compelling thesis. Mindset change is one thing; wholesale abandonment of management approaches that have served organisations well for decades is quite another. Waterfall-style project management still has a place where iteration is simply not possible or desirable, regular customer engagement is not feasible or a critical path may still be critical. Similarly, we should be wary of embracing agile management to the exclusion of everything else. Context, of course, is everything.

An agile mindset involves never-ending innovation

Writing in Harvard Business Review, Darrell K Rigby, Jeff Sutherland and Hirotaka Takeuchi offer a rousing call-to-arms to reluctant or ill-informed executives to better understand the power of agile and how it can transform business beyond the world of project management. They cite useful case examples for how agile can be embraced to best effect.

But even these evangelists are clear that agile is not a panacea, and should only be deployed under the right conditions, when “the problem to be solved is complex; solutions are initially unknown, and product requirements will most likely change; the work can be modularized; close collaboration with end users (and rapid feedback from them) is feasible; and creative teams will typically outperform command-and-control groups.” They suggest that fertile ground for agile approaches include product development, marketing projects, strategic-planning activities, supply- chain challenges and resource- allocation decisions. Less productive might be routine operations such as maintenance, purchasing, sales calls and accounting.

Combining methodologies is also a shrewd way to lead with agility

Hybrid approaches to project management, based on waterfall and agile techniques working alongside one another, are another way to extend the reach of agile. Management consultants, McKinsey, believe that truly agile organisations need to reconcile the seeming paradox of both “stability and speed”.

Mastering this paradox means creating a stable core, while championing looser, more dynamic elements that can adapt more quickly to challenges and opportunities. A 2018 report from the McKinsey Agile Tribe charts the move away from what it calls organisations as machines (hierarchical and specialised) to organisations as organisms that combine “stable backbone elements that evolve slowly and support dynamic capabilities that can adapt quickly to new challenges and opportunities”.

What does all this mean for leaders?

In essence, few, if any, companies are purely agile or waterfall. These umbrella terms represent different mindsets that encompass a variety of practices and approaches to organisational development. While agile is often associated with large-scale software development, its roots lie in lean manufacturing and organisational learning. Agile techniques such as stand-up meetings, weekly iterations and visual management can apply to any industry. The trick is to balance these approaches, developing the judgement needed to know when to be more agile and when to focus on that more stable core.

Few, if any, companies are purely agile or waterfall

Simon Hayward’s work on agile leadership builds on his previous work on what he calls “connected leadership”; for Hayward, agile leaders are connected leaders. They need to balance the seemingly paradoxical roles of enablers and disruptors. And while leaders may be predisposed towards one or other behaviour, agile leaders need to be able to do both and, crucially, make the right decisions where and when to focus their efforts. The key skill is knowing how to read any specific situation and react accordingly.

In a 2004 Harvard Business Review article, Charles A. O’Reilly and Michael L Tushman introduced the concept of the ‘ambidextrous’ organisation, capable of “exploiting the present and exploring the future”. Their research showed that, with innovation, the key to success was ambidextrous senior teams and managers able to lead across organisations while “combining the attributes of rigorous cost cutters and free-thinking entrepreneurs” — a similar conclusion to McKinsey’s stability and speed thesis.

A leadership toolbox - select different methodology for different scenarios

Ambidexterity may also be a useful prism through which to view agile leadership. Few would doubt that the fourth industrial revolution has created an irrevocably changed business environment, nor that leaders must be able to deploy key agile techniques such as cross-functional teams, a focus on customers or greater flexibility in planning.

But even the Agile Alliance originators of The Agile Manifesto talked about balance. Perhaps the most important leadership mindset is one that views contrasting approaches to project management — and management and leadership more generally — as complementary, tools and techniques in a toolbox of options rather than silver bullets.

Our levy-funded online management courses use mentoring and technology to develop your managers in a way that will tangibly benefit your organisation from day one.