Agile Is Dead (Or so everyone tells us)

So, here’s the thing, how we manage our businesses has changed so fundamentally, that it’s often hard to understand and even if there are companies letting go of their agile support functions, they are more agile today than when they started and once an organization moves towards agile, even if they go back to older ways of working, it won’t be all the way back.

When I got out of college PCs had just made their way into organizations. I remember creating elaborate forecasting models in Lotus 123 that fundamentally changed how we managed our manufacturing inventory. To create these models, I had to manually add data from all sorts of input, from SMSA marketing reports put out by the government to green-lined production and sales data spit out by our mainframes. It took me months to assemble this information into the models we used for our annual planning. Can you imagine anyone doing this now?

Today we operate at warp speed with information at our fingertips captured in vast data lakes. One only needs to look at the massive investment in data management capabilities to understand that Data is King in the business management world. But here’s the thing, investing in data only gets you so far, you still need people to interpret the data and, in many ways, companies may have over-invested in their data capabilities.

These changes have occurred in my career, and I understood many years ago, either adapt to your surroundings or you will be left behind. Yes, there will be those that don’t, like the Regional Sales Manager who threw the PC I had just installed in the garbage or the other one who put it in a closet as soon as I left. Both people, regardless of their previous sales success, were let go from the company.

Business leaders today face ever-increasing challenges and frankly, the Agile industry has for the most part let them down. We blame them for our failures, but the reality is that we have been working with one hand tied behind our back, and we are the ones who did that. Leaders need to understand how their engagement in technical agility will change how they deliver value to the business and its customers. Let’s also be real in that most CEO’s did not take a technical career path, so they aren’t as connected to how technology is delivered. Their expectations are that projects are approved, work is completed to deliver whatever my leaders decided was important and I’m given updates on two things — Budget and progress, Agile doesn’t come into play in decision-making at my level, and Agile is a ‘technology’ thing.

Since they don’t understand and don’t engage, they keep us at a distance or worse ignore us.

So though there is a question about whether the investment in Agile has delivered sufficient value (it has but has more work to do) the reality is that Agile is a response to the way the world is changing. Let’s look back to the 1990s when Agile and Scrum had their foundational beginnings. It was during this time that we entered the dotcom era. There was a massive change in the technology landscape (the internet) and when a change like this happens money will follow. Can you imagine investors coming to a dotcom business only to be told, thanks for your investment we’ll let you know when we have a website in around 18 months…. That wasn’t going to happen. This is where Agile approaches provided the avenue for change.

Now that Agile is over 20 years old, it’s time to recognize what has worked (team productivity, and quality) and what hasn’t (traditional project management and funding still in place).

Things are continuing to change at a rapid pace, so much so that perhaps Agile in its current approach and mindset is not going to work. Agile needs to address what they have missed, which is leadership needs. Without providing a clear way for them to translate the way they want to manage into an Agile delivery model, Agile is yes going to fail to deliver on its value proposition and be excluded from the conversation at the highest levels of organizations.

Business Leaders Need to Understand Agile — It has NOTHING to do with Frameworks

I’m a businessperson first and foremost, having led several companies. As someone who has been involved with Agile for 20 years, I struggle with the myopic nature of what Agile has become, which is almost entirely focused on implementing and mastering an Agile framework.

Agile is now mostly about the operational optimization of a framework with value delivery an assumed attribute, but Agile hasn’t established an effective way to understand or identify value. Teams are expected to understand what it is and how to deliver it. The problem with this is that to deliver long-term value to my business I need to understand what I need to invest in first. Value delivery is entirely dependent on investing in the right things at the right time for the right reasons. Satisfying my customers is certainly important but there are so many peripheral aspects of my business that aren’t tied directly to customer satisfaction that also require that we make smart investments. And investment decisions should be coming from leadership with them directly tied to strategic outcomes.

Every part of the organization has strategic needs, but often what my operational leaders provide for funding are things they want or think will be valuable to them, and many times these requests end up creating organizational dissonance as the initiatives being approved aren’t functionally or strategically aligned. There isn’t an Agile framework out there that has provided a solution that helps organizations optimize their value stream investments so that they both understand the outcomes that are expected and how we will deliver on those outcomes. Value in Agile is not quantified, it’s either inferred or objectively assigned by people who are required to commit to the delivery of value, rather they are committed to delivering the project, which may or may not deliver value.

For Agile to succeed as a business framework we need to have a clear and unambiguous way to quantify value, prioritize it and confirm that the value we planned for is delivered. Failure to provide this is operational malpractice and yet we spend millions of dollars working to implement frameworks that have no hope of making your organization agile, at best you get some improved levels of transparency but without optimizing your delivery capabilities around value then Agile as it is configured today will only get you so far from an investment return perspective.

Predictability in Agile isn't Bad, Just Really Misunderstood

As a business leader I have a lot of pressure to deliver consistent results as there are people who rely on my business for their livelihood, customers want to have predictable quality in the products they purchase and investors who make decisions to invest based upon the expectation of predictable financial results, the key work here is predictable.

In Agile if I speak about needing predictability from teams, I get strong pushback. There are of course reasons for that as most technology projects start with fixed date and scope and then are expected to deliver, because as a Business Leader I need predictability.

Leadership who is typically removed from the day-to-day work of technology teams expects the same commitment to predictable results that they are held to. Leaders who don’t deliver don’t stay Leaders for very long.

For technology teams to not be expected to have some level of predictability to their work is something that doesn’t resonate with leadership. In many ways they see Agile as a way to not make commitments or be held accountable in the same way they are.

The problem comes from where each group is coming from.

The leader seeks to tell their customers, investors, etc…concretely something that will occur in the future. Future sales, new product features, improved cost savings, etc.. are what these personas seek. The stock price for companies is based upon the future looking as predictable as it does today, investors HATE surprises.

To solve this problem, traditional project management was brought into the realm of software development and applied with the expectation that software development was as predictable as the construction industry, it's not.

Agile sought to address this misalignment by focusing on technical excellence (XP) and improving the way business and technology work together (Agile Manifesto).

Unfortunately, the key changes that must occur in an organization, which is the tight alignment of customer, business and technology have largely remained the unchanged.

What needs to change is that how work is defined and delivered which is aligned to value and broken down in small valuable increments that can be delivered. Leadership cannot expect large transformative projects that take months and months to complete to be managed without any risk or changes in direction, it’s just not realistic.

Where we need to focus then is building strong and stable teams who can deliver work predictably whether they are working on a POC to review with leadership, new features for our customers or implementing new systems that manage the business more effectively, everyone in the organization requires both predictability and accountability, two things that create a strong business and trusted relationships across the organization.

For Leaders put it in this perspective — Is it more valuable to deliver a $3 million dollar 18 month and not know if the value you seek will be delivered? Or is it more valuable to deliver a sub-set of capability for $300k in 3 months that delivers 70% of the entire value of the $3 million dollar project. Do you really want to spend another $2.7 million to deliver the rest of the 30% in value?

This is the value proposition that is essentially missed in Agile at the leadership level.

Why you need to have an Investment Mindset to Manage Your Product Development Intake

If your organization is like most, you have an annual planning process where your functional leaders come up with things they think are important to do and then provide a cost estimate to your PMO, ­then you are not having the right conversation in your planning process, nor are you making an informed decision as to whether these disparate projects have strategic value.

In the worst-case scenario, you have two different groups working on opposing efforts or they may both be working on the same capability, meaning you have doubled your cost for the same capability (at a minimum).

By focusing only on cost, you are missing the key aspect of your investment in your software and product capabilities that support and drive your organization. 

To ensure that you are building real long-term value you need to develop a value-based investment mindset that incorporates expected value (outcomes), cost, and risk associated with an anticipated investment.

My valuation model translates your organizational strategies into value scores that are associated with quantifiable outcomes.

When an idea is submitted, it is accompanied by a lean business case and a value score.  The value score defines the outcomes at the outset of any planning.

It allows for healthy conversation in the Intake stage as to whether or not the idea should even be considered, value score aside.

You have limited investment dollars for your software/product development and you cannot waste them on work that is not valuable or aligned to strategic outcomes. 

In truth, your software is filled with features and capabilities that are rarely or ever used.  Keeping people busy working on things that have no value is not any fun from a technology perspective and it fills your code base with significant tech debt since many of these unneeded features were part of a project plan associated with unrealistic dates. 

Moving to an investment mindset also gets you thinking about the flow of work to teams over the project and date-driven work that is already decided upfront.  Developing a consistent flow of work for your teams is one of the single most important steps you take to develop mature agility.

Additionally, by taking an investment approach you allow for investments to be stopped, just as you might drop a stock from your portfolio if it isn’t performing to your risk/return profile. 

You would do well to adopt the investment maxim that I was taught, have a sell trigger as soon as a stock price drops below a certain threshold, meaning don’t hold onto bad investments to the point they have no value.

What this means from an agility perspective, is that you have the power to stop investing in an idea if you discover that the cost or the efficacy of the idea won’t deliver the value you had expected.  In your waterfall project approach, you would be forced to continue the project even if you had realized it wasn’t going to deliver what was expected.

If you would like to learn more about my Portfolio Valuation Model connect with me at michael@soundagle.

Software Engineering is Not Engineering (More Like Art)

Recently I had an interesting conversation regarding whether or not software developers were engineers or not. The most compelling argument that they could provide for why Software Engineering was truly an Engineering discipline centered around the fact that Universities and Colleges have their Computer Science group in the Engineering department.  That is not sufficient in my mind to by default call Software Engineers well, Engineers. Why? Consider the following

Engineers who build skyscrapers, airplanes, bridges and products that require low tolerance for defects are often operating under the laws of science and as such require that they account for many scientific elements when building their products.

They perform a great amount of 'testing' both in theory and with product before something is released for use.  Their engineering practices are based upon scientific principles that go through extensive peer review and have been building in knowledge for hundreds if not thousands of years.

Now lets look at the definition of Engineering:

1. The branch of science and technology concerned with the design, building, and use of engines, machines, and structures.
2. The work done by, or the occupation of, an engineer.
3. (Business / Professions) the profession of applying scientific principles to the design, construction, and maintenance of engines, cars, machines, etc. (mechanical engineering), buildings, bridges, roads, etc. (civil engineering), electrical machines and communication systems (electrical engineering), chemical plant and machinery (chemical engineering), or aircraft (aeronautical engineering)

Though of this I think can be applied to the science of computers, ie the hardware = Machines I don't however see much if any of this being applied to Software Development.

A paper in 1996 (William Aspray, Reinhard Keil-Slawik, David L. Parnas) talks about some of the reasons that writing code isn't associated with Engineering:

"Computer science is often characterized as an engineering discipline with the systematic study and development of software as its principal subject matter. Software Engineering, however,although combining both key words, has not become a central discipline in most computer science departments.  In many respects, this discipline embodies the same ideosyncracies that can be observed within computer science as a whole such as:

• Highly innovative and rapidly changing field with no broadly recognised core of material that every practitioner must know; • Few results are supported by empirical or comparative studies; • Work within the field older than 3–4 years is rarely acknowledged or referenced; • Old problems are given new names and old solutions overlooked; • Evolution of the discipline is tightly coupled to economic and societal demands; • There is a need for interdisciplinary work comprising e.g. mathematics, psychology, business or management science, … ; • Continuing debate about whether there should be a discipline called software engineering, and if so, whether this should be treated as another discipline among the set of traditional engineering disciplines."

As you read through this I think we can start to see a clear delineation between the disciplines required of 'Engineering' and the disciplines that Software Development requires (Note - This is not an article to criticize people who write software, but rather a way for us to better understand how to manage them and how to help them deliver even better software)

Go to almost any software development team and you will find a mix of people who have ended up going down the path of writing software, they can be people who were Computer Science majors all the way over to people with a Liberal Arts degrees.

The languages that allow us to write software today have matured over the years and as such have become easier to learn and work with.   This mix of skills and backgrounds is not bad, but it deviates from what an Engineering team building an airplane might look like.  Full transparency here, I have never worked in an organization that does true Engineering, but I know many people who do, and I suspect that you would not find people who are designing/engineering bridges or airplanes who don't have a strong educational background in Engineering, but you will find these people in your software development teams.

So why is 'Software Engineering' more art than science?

I think that Software Development is more art than science because we must create software that we interact with on a more human basis.  Creativity abounds in web development as we continue to strive to bring intuitive and easy to use interfaces to the people who are our 'users'.  The people who write software must be able to synthesize multiple layers of code, from Front End to Back end and everything in between. This talent is probably more aligned to writing a symphony in that you have to deal with an entire array of players, each with different instruments (API, Legacy, etc..) and when you get it right you get beautiful music, when you get it wrong you get noise.

When writing software we have the option to try multiple paths to an end, whereas when building a bridge our ability to deviate from proven engineering practices success is much more limiting and catastrophic when we do when we don't.

One can also argue that the newer languages being created are designed to make writing software more accessible to more and more people who have not be through a computer science programs, we are striving to get our languages to ever more English like syntax.

Consider also that people who write software are not held to specific engineering practices.  Software, as it has evolved, has provided people who 'develop' software product with endless ways to implement it the underlying code.  The fact that there are not any set standards that every Software Engineer must use I think clearly shows that software is not a true engineering discipline, but rather a creative one.  Artists embellish, they innovate, the push boundaries into new ways of performing....I think this better describes many of our software developers more than engineering.

The beauty of current software development is that you have the ability to envision new ways of implementing code that delivers functionality and in doing so create new coding processes that others can use.  Software languages provide many different ways to write code to deliver a product functionality, that might be 3 lines of code or it could be 100, it depends on the background and experience of the individual writing the code.

I think when we start thinking about how to manage and support our Product Development teams in our organizations we would benefit from providing them the framework that encourages high quality through innovation, failing fast, and supports both the individual and groups ability to determine their success.  That is not to say that Sr. Leadership doesn't play a part here, it certainly does, by providing clear vision of what is wanted (think of a symphony conductor, who must bring out greatness while providing their interpretation of a particular piece), an ability to accept failure as part of success and a fearless ability to have confidence that when you provide the other two components people will bring greatness as a by-product.

Artists operate in a very Agile manner, we listen and feed off of each other and in the process discover new ways to approach our art. Listen to jazz music and you see the elements of Agile and creativity evident as the small group (Scrum team) takes a song and through listening to each other can create new music, sounds or approaches to an old song.

As with Artists, Engineers don't like to be told something can't be done, that quest to make something happen that has never existed before is what an artist strives for, there is an emotional rush that comes with artistic creation and I think the same is true of software product creation.

Engineers who build skyscrapers, bridges and planes are no less artistic or innovative in their designs however they must take a much more rigorous approach to what they deliver, they simply don't have the ability to go back and redo it.  If they get it wrong people die, though there is of course software that is also mission critical, but for the vast majority of us who develop software for a living, we are able to take a less rigorous approach when we create our products and we expect in Agile to move to a continuous delivery model so our work is never 'done'.

Agile and the Management Impact

You're a technology manager in an organization that has decided that they are going to adopt Agile.  Thunk......now what? You've spent years becoming the high performing manager you are today, managing the day-to-day details of your team, holding things together, making decisions big and small....You are the leader, success of your team hinges on your ability to make decisions that impact the way the team works. Your team is successful because of your management efforts....Sound like you?

This Agile thing is telling me that my teams are now going to be 'self-organizing' and be able to make decisions on their own...what the heck am I going to do?

Managers who are asked to move their organization to Agile may often believe like they are providing the path to the doorway out of the company.  This resistance can be one of the primary impediments to successful Agile adoption and as an organization Sr. Management needs to be aware of this paradigm and provide support to middle managers that Agile is not about replacing them but truly about getting them into a position where they are focused in the truly valuable parts of their job - Strategic Planning, Employee Development..the bigger picture.

Stephen Covey teaches that successful managers are the ones who are able to remove themselves from the quadrant of activities that are Urgent and Important and focus on the activities that were Important but Not Urgent, meaning remove yourself from day-to-day decision-making over planning and strategy development.

So many of us can get sucked into the day-to-day minutiae of our teams that we end up ignoring the bigger picture areas which should be our focus.  Why?  Because dealing with Important and Urgent activities is an addictive feeling, you feel like you are making important decisions when in fact your team is probably more than capable of making them without you.

Agile provides your team with leadership opportunities that aren't found in more traditional process control organizations.  Step back and let your team grow.  I've been amazed how teams, when given the opportunity, can solve process issues and impediments to productivity that I would have never thought of.  The collective minds of your organization are capable of great things.

As a manager you need and want to embrace this power and provide support for you team.  Let them help you and help the organization deliver on the promise that Agile provides.

How can you help?

• Understand that for your teams to succeed they need to fail, yes I said that.
• Don't attend their ceremonies (standups, retrospectives)  Why?  The team needs to be able to speak openly and honestly about what is working and not working without you being present.
• Use the working metrics available to you, velocity, burndown.  If your team says they are going to do 'x' story points and they do that consistently, what else do you need to know?
• Don't force your opinions on what they 'SHOULD' be doing.  Have confidence that you have hired great people and they like you want to succeed.
• Work closely with the Product Owner and Scrum Master to discuss any concerns you have.  Understand why things are working a certain way before leveling your judgement.
• Don't act as a Scrum Master or Product Owner of the team. Why?  You manage their careers and it's highly unlikely that the team will receive the benefits of iterative continuous improvement if they don't feel confident that they can say when things aren't working well.
• Don't believe that everything is great the way it is, anything can be improved.

Agile will provide transparency with respect to the areas of your organization that isn't working efficiently.  As a management team you will need to focus your attention on addressing these impediments as your Agile teams mature.  If these hurdles and impediments aren't dealt with you will reach a ceiling on the ROI that Agile can provide.  As with your teams, management needs to be honest with themselves about broader organizational issues so that a stronger and more effective organization emerges.

Agile isn't just about delivering software faster, it's much more than that.  Understanding that will make the transition your Agile smoother.