product development

Lean/Agile - The Art of Doing Less

Our traditional ways of managing software development projects center around generating ideas that are deemed valuable and then attempting to estimate at a detailed level the effort involved by having a team of people work to document ‘all’ of the requirements up front before the project even begins. These projects often will take many months to complete before anything of value can be delivered to the business.

Agile focuses on delivering value as quickly as possible, often within 2 weeks to 2 months. Lean focuses on maximizing work not done, which combined with Agile speaks to doing less as a part of our delivery process.

But what do we mean by less when looking at taking on a large feature/product development effort? 

There are couple of ways that this comes into play:

1.   Software is often littered with features that someone thought would be important to the customer, yet fails to deliver the value (need) to the customer. Worse still this code becomes part of our legacy application, stuff we have to code around and test against regardless if the feature is being used or not.

2.     Often the perceived value of something changes once the project begins, causing the business to pull the plug on work before it is complete and anything of value can be derived by your software development team. This is costly both in terms if financing and employee happiness.

In both cases the business has expended money to have software developed with the end result being the business received reduced or no value.

When looking at Agile Product/Project Management we are asking the business to act not just as the agent of funding but as the consumer of the work as well. Too often the business is a distant or non-existent participant with respect to how their product is unfolding. Instead they rely on project plan updates that speak solely to progress against the agreed upon scope of the project at its inception over viewing what has actually been developed and delivered to date.

In an Agile setting business needs to review the work at regular touch points, often the Sprint review and actually see what is being developed. There is often a great difference between what the business ‘thinks’ they want and what they actually need. Seeing working software lets them see their idea in action, often with enlightening results. What they envisioned may not in fact be what they are seeing and more importantly not what they need (the value). They funded a Ferrari but maybe only needed a good old family sedan.

Business is typically missing in this very powerful process in Agile, a process which allows them to make course corrections and even stop working on features as soon as they reach their maximum value. 

At one large entertainment organization I worked for several years ago, the business had us working on a large rewrite of a heavily used widget that resided on hundreds of thousands of pages. After performing a Discovery and Story mapping session we determined that the work should be done in three releases. At the end of the first release the business reviewed what we had completed and decided that this was actually good enough and then pivoted us to working on an even more valuable redesign of a key customer facing web app.

In the waterfall days the work that we did for that first release would have been put on the shelf, the money invested would have delivered no value to the business. With Agile we were able to complete the work and deploy to production, providing the business real value for their investment. They made the decision to stop this work and move on to more valuable work.

The Art of Doing less is about developing more mature approaches to how your organization funds and manages work. Instead of funding projects fund your teams, allowing for work to flow to them over having them focus on managing project scope to the bitter end of the project.

Move away from believing you have to have fixed scope to one that understands that Agile is about powerful flexibility and maximizing the value of the work that we do, while minimizing the amount of legacy code we leave behind. 

A legacy system that is littered with large amounts of un-needed or under-utilized capabilities is code that is harder to scale, develop in and test against. When your teams talk to you about Tech Debt, this is what they are referring to and as a business you are both a direct owner and contributor to this reality.

The Art of Doing less brings with it a greater ability to maximize an organizations investment in its software and product development.

The focus for business should be on what is needed not what is wanted. Project funding models encourage asking for more than is needed so one of the first things you need to do is to move to a team based funding model. Develop a framework to assess value and risk so you can appropriately prioritize work as it flows through your teams.

Understand that each of your Scrum teams has a fixed cost (~40k per sprint), this should guide you further in questioning what you want your teams to work on.  At some point in time in most projects there comes a point where there is a diminishing value relative to the investment.  

Lean and the Art of Doing less requires changing your organizational mindset to understand there is also value in work NOT done.

 

 

Software Engineering is Not Engineering (More Like Art)

Recently I had an interesting conversation regarding whether or not software developers were engineers or not. The most compelling argument that they could provide for why Software Engineering was truly an Engineering discipline centered around the fact that Universities and Colleges have their Computer Science group in the Engineering department.  That is not sufficient in my mind to by default call Software Engineers well, Engineers. Why? Consider the following

Engineers who build skyscrapers, airplanes, bridges and products that require low tolerance for defects are often operating under the laws of science and as such require that they account for many scientific elements when building their products.

They perform a great amount of 'testing' both in theory and with product before something is released for use.  Their engineering practices are based upon scientific principles that go through extensive peer review and have been building in knowledge for hundreds if not thousands of years.

Now lets look at the definition of Engineering:

  1. The branch of science and technology concerned with the design, building, and use of engines, machines, and structures.
  2. The work done by, or the occupation of, an engineer.
  3. (Business / Professions) the profession of applying scientific principles to the design, construction, and maintenance of engines, cars, machines, etc. (mechanical engineering), buildings, bridges, roads, etc. (civil engineering), electrical machines and communication systems (electrical engineering), chemical plant and machinery (chemical engineering), or aircraft (aeronautical engineering)

Though of this I think can be applied to the science of computers, ie the hardware = Machines I don't however see much if any of this being applied to Software Development.

A paper in 1996 (William Aspray, Reinhard Keil-Slawik, David L. Parnas) talks about some of the reasons that writing code isn't associated with Engineering:

"Computer science is often characterized as an engineering discipline with the systematic study and development of software as its principal subject matter. Software Engineering, however,although combining both key words, has not become a central discipline in most computer science departments.  In many respects, this discipline embodies the same ideosyncracies that can be observed within computer science as a whole such as:

• Highly innovative and rapidly changing field with no broadly recognised core of material that every practitioner must know; • Few results are supported by empirical or comparative studies; • Work within the field older than 3–4 years is rarely acknowledged or referenced; • Old problems are given new names and old solutions overlooked; • Evolution of the discipline is tightly coupled to economic and societal demands; • There is a need for interdisciplinary work comprising e.g. mathematics, psychology, business or management science, … ; • Continuing debate about whether there should be a discipline called software engineering, and if so, whether this should be treated as another discipline among the set of traditional engineering disciplines."

As you read through this I think we can start to see a clear delineation between the disciplines required of 'Engineering' and the disciplines that Software Development requires (Note - This is not an article to criticize people who write software, but rather a way for us to better understand how to manage them and how to help them deliver even better software)

Go to almost any software development team and you will find a mix of people who have ended up going down the path of writing software, they can be people who were Computer Science majors all the way over to people with a Liberal Arts degrees.

The languages that allow us to write software today have matured over the years and as such have become easier to learn and work with.   This mix of skills and backgrounds is not bad, but it deviates from what an Engineering team building an airplane might look like.  Full transparency here, I have never worked in an organization that does true Engineering, but I know many people who do, and I suspect that you would not find people who are designing/engineering bridges or airplanes who don't have a strong educational background in Engineering, but you will find these people in your software development teams.

So why is 'Software Engineering' more art than science?

I think that Software Development is more art than science because we must create software that we interact with on a more human basis.  Creativity abounds in web development as we continue to strive to bring intuitive and easy to use interfaces to the people who are our 'users'.  The people who write software must be able to synthesize multiple layers of code, from Front End to Back end and everything in between. This talent is probably more aligned to writing a symphony in that you have to deal with an entire array of players, each with different instruments (API, Legacy, etc..) and when you get it right you get beautiful music, when you get it wrong you get noise.

When writing software we have the option to try multiple paths to an end, whereas when building a bridge our ability to deviate from proven engineering practices success is much more limiting and catastrophic when we do when we don't.

One can also argue that the newer languages being created are designed to make writing software more accessible to more and more people who have not be through a computer science programs, we are striving to get our languages to ever more English like syntax.

Consider also that people who write software are not held to specific engineering practices.  Software, as it has evolved, has provided people who 'develop' software product with endless ways to implement it the underlying code.  The fact that there are not any set standards that every Software Engineer must use I think clearly shows that software is not a true engineering discipline, but rather a creative one.  Artists embellish, they innovate, the push boundaries into new ways of performing....I think this better describes many of our software developers more than engineering.

The beauty of current software development is that you have the ability to envision new ways of implementing code that delivers functionality and in doing so create new coding processes that others can use.  Software languages provide many different ways to write code to deliver a product functionality, that might be 3 lines of code or it could be 100, it depends on the background and experience of the individual writing the code.

I think when we start thinking about how to manage and support our Product Development teams in our organizations we would benefit from providing them the framework that encourages high quality through innovation, failing fast, and supports both the individual and groups ability to determine their success.  That is not to say that Sr. Leadership doesn't play a part here, it certainly does, by providing clear vision of what is wanted (think of a symphony conductor, who must bring out greatness while providing their interpretation of a particular piece), an ability to accept failure as part of success and a fearless ability to have confidence that when you provide the other two components people will bring greatness as a by-product.

Artists operate in a very Agile manner, we listen and feed off of each other and in the process discover new ways to approach our art. Listen to jazz music and you see the elements of Agile and creativity evident as the small group (Scrum team) takes a song and through listening to each other can create new music, sounds or approaches to an old song.

As with Artists, Engineers don't like to be told something can't be done, that quest to make something happen that has never existed before is what an artist strives for, there is an emotional rush that comes with artistic creation and I think the same is true of software product creation.

Engineers who build skyscrapers, bridges and planes are no less artistic or innovative in their designs however they must take a much more rigorous approach to what they deliver, they simply don't have the ability to go back and redo it.  If they get it wrong people die, though there is of course software that is also mission critical, but for the vast majority of us who develop software for a living, we are able to take a less rigorous approach when we create our products and we expect in Agile to move to a continuous delivery model so our work is never 'done'.

Agile Planning - I have a need a need for speed

Working at Disney a number of years ago was in so many ways transformative for me (not sure why I left) because it provided me with an opportunity to work with an organization that needed to get better at delivering software for our partners and we ended up choosing Agile as our path. Disney was the place where I had the opportunity to help build an Agile process from Requirements to Delivery and what we discovered was that we needed to develop an effective planning process that allowed us to build a solid backlog of work before we just started coding.  I here that so often in organizations that are just starting to adopt Agile.  I think a statement I heard recently is descriptive of organizations that just start coding - Shoot and Point.

Disney is a largely creative driven organization (Not surprising) and because of this we typically had a disconnect between our creative (UX) groups and the Product Delivery teams.  The UX team primarily worked independently of the PD teams and as was the case when I arrived, UX would deliver a creative design that didn't align to our technical capabilities.  This is a common issue in today's web development environment.

Our first 'Release' Planing went very poorly and after a round of retrospectives we came up with a format that at first pass you would say wasn't Agile (trust me we used that phrase a lot in the early days of our becoming Agile).  But in the end this first step of Discovery ended up being what I believe is the most critical element of being able to go fast in Agile.

The basic process that we ended up with was as follows:

  • Pre-Discovery - Sr level PD, PO, UX, Marketing and other Stakeholders would review a specific new feature that was being considered. The group would utilize several tools such as Mind Mapping to understand the scope, parameters and potential dependencies at a high level.  If the feature work was approved then we went to the next phase.
  • Discovery - Depending uponthe the size of the potential project Scrum teams and extended stakeholders would meet to go through a low-level review for that feature development.  For many of our larger efforts it was not uncommon for us to sequester the teams into a room to work through the entire effort, UX to QA to Delivery.
    • Process
      • Kick off - Have your PM or PO along with the key stakeholder(s) of the effort describe WHY this feature is so important.  We learned in our process development that it helped our PD teams to have an understanding as to why this feature was important to the organization.  It helped them feel connected to the value that was being delivered and not simply code jockeys running a race.
      • Competitive Review - Another great exercise was to have the entire team go out and find competitive features that we either did or didn't like and describe why.  This helped the next phase of our Discovery process as we worked to define what our feature sets would ultimately look like
      • UX and Story Development - This was the primary scope of our Agile workshops.  Typically led by the UX lead for the project we would begin developing low-fi wireframes and discuss the issues, constraints and code complexity that the low-fi would entail.  We discovered in this process that we could work through the types of issues that come much later in a typical product delivery effort.
        • Outcomes -
          • UX ended up with designs that they could utilize to develop prototypes that would be used in User testing prior to any significant development work being completed.  This allowed changes to UX to be found at the very beginning of the PD process rather than at the end when refactoring consumes a much larger amount of time and leads to lower quality of code.
          • PD ended up with a solid design at the beginning of the PD process which led to high quality code and higher levels test automation.
          • QA ended upon with an ability to write higher quality acceptance criteria which lead to high quality in the delivery and higher levels of test automation (sensing a theme here?)
          • User Story development was done during the Discovery phase and with it I was able to have a fairly accurate model to predict the number of stories at the beginning of the Discovery phase (typically between 100-120 higher level Epics, we strove for stories to be between 21-34 points in this phase as PD would start fairly quickly after the discovery phase 2-3 weeks) and how many that would translate into for a full project (typically 350 - 400).  This provided me with input as to how many BDD acceptance criteria would come out of this as we used a marker to determine when a story should be broken down - More than 7 variables in the BDD would be an indication that it's time to think about breaking down the story and more than 14 tests in a single test scenario would also trigger the conversation of whether to add a scenario to a story or create a new story.
            • Benefit - Keeping your BDD test automation in small increments makes it much easier to understand what broke, who probably broke it and what is needed to fix it.

I know this doesn't 'Sound' Agile (like the name of my company), but in my experience doing this small amount of work up front does provide teams the base to go really fast once the PD process begins.

I have used this process now for many years and when we do it right it's like writing a symphony, all of the moving pieces make beautiful music.  When it isn't done right, then all you get is noise.

This process probably does work for larger and more complex organizations over small organizations, but really would you start building a house with no blueprints and no idea of what you wanted?  If you had builders just show up and you told them I need a house to live in and I need it fast you will get that, but I doubt it will be anything that you want. And in reality it wouldn't be done fast as they probably wouldn't have the right materials scheduled to arrive at the right time.  I have my roofing supplies but the foundation company can't some for a month, see what I am getting at?

Slow down a bit, understand what you want, how to get it and then go fast to get it.

Behavior Driven Development is more than Automation

Recently while working with an organization going through a large Agile adoption I had the opportunity to work with a team that was open to adopting BDD acceptance criteria story development.  One of the key differences for this team was that this was the first time that they had experienced a collaborative story development effort.  Prior to this most teams in the organization looked to the Product Owner to write all of the stories, which led to stories that were not contextually rich, had little to no acceptance criteria and were difficult to demo at the end of the sprint.
As I discussed the success of the teams use of BDD I was surprised to hear feedback that indicated the organization wasn't ready for the heavy lift of using BDD and that I hadn't moved the team into BDD because their final work wasn't automated.
If that is your position then I think you are missing the point of BDD.  Yes BDD is great in that when you right acceptance criteria in the Given/When/Then format and build your example tables correctly you can obtain automation fairly easily with tools such as Cucumber, Fitnesse and a whole host of tools that have been developed to support this very successful way to build out what I call 'contextually rich' user stories.
However even if you don't automate your BDD acceptance criteria, the value you get from understanding the behavior of the features you are going to build is invaluable.
BDD came about because Dan North realized that TDD and really great code meant nothing if it didn't deliver the value or functionality that the stakeholder needed.  This concept was clearly illustrated for me at one organization I worked for when the Development Manager was lamenting about a Product his team was enhancing was pulled from the market because they didn't deliver on time in order for the organization to stay competitive in the market.  He said 'If only they could see all of the cool code that we've written'....To which I said 'Business doesn't care about code they care about delivery" no matter how great your product is from a code perspective it matters not at all if you don't deliver it when your customer needs it.
For those of you who are exploring the use of BDD with your Scrum teams understand that there are essentially two separate efforts that effective teams are working on:
1. Team collaboration - Successful teams using BDD understand that they all need to be involved when building out user story context with Given/When/Then.  In fact I would argue that this is the most important element that drives a clear understanding of what we are building and guidance to when a user story is done.
2. Automation - One of key components of successful Agile teams is the development and maintenance of an effective automation suite.  Utilizing BDD provides an effective way for teams to obtain high levels of automation and do so within the Sprint that they are working in.  A common mistake of newer Agile teams is to develop in one sprint and test in the next and sometimes automate in the third.  Using BDD automation tools such as Cucumber with well written acceptance criteria (I'll provide examples of well written BDD in my next blog post)
A third benefit of BDD is that a Scrum Team builds a clear understanding of the scope of the story.  Since everyone participates in the development of the acceptance criteria, if a scenario is missed, it's a shared miss - No finger pointing, just manage the new scope with a new story and prioritize it in the backlog.
I've helped a number of organizations implement BDD effectively,  one of which did not have any automation at all.  The use of BDD acceptance criteria writing almost immediately improved the understanding of the features being developed and helped my QA team write significantly more test cases than they had been doing.  This effort led to a measurable improvement in Quality even before we added the automation suite later.
Having come from a waterfall world many years ago I can tell you that when you 'get' BDD,  it changes the way that you look at requirements and provides you with a mental framework that you can quickly use to model what is going to be built.
After a few months of using this format at Disney the Product Owners were uniform in their agreement that there simply was no other way that they would want to work with their teams.
Automation IS our goal, but it is not what defines BDD.

Hurdling through Product Delivery part 2

article-1255763-08965ee6000005dc-57_468x286.jpg

article-1255763-08965EE6000005DC-57_468x286 Product delivery is a discipline that requires the involvement of all segments of an organization:

  1. Customers and Stakeholders
  2. Product Owner
  3. Scrum Master
  4. Delivery Team - UX/Dev/QA
  5. Operations
  6. Services and other specialized teams that touch the product in anyway as it moves from concept to delivery to production.

As I mentioned in my previous entry, maximizing the smoothness of product delivery is like running the hurdles.  How so?

What happens when you first start running the hurdles with no experience?  You stand at the starting line and then you start to run.  As you approach the first hurdle you try to determine how you are going to make that first jump.

Is that first approach smooth?  Probably not.  You probably came up to that first hurdle and maybe stutter-stepped before trying to jump or even had to stop before you actually tried to get over the hurdle.   Why?  Because you didn't know what to expect, you didn't know what you didn't know as you ran toward that first hurdle.  Maybe you made it over all of the hurdles but it probably wasn't pretty, perhaps you fell a few times in the process.

So with skinned knees and wounded pride you make your way back to the starting line and you try again and again and again.

In this process you are setting your baseline performance.  Are you getting better with each attempt?  Hopefully....because we really want to be a successful sprinter.

As you continue training, perhaps even getting a coach, you start to break down your individual steps as as you go from hurdle to hurdle.  Over time and through practice and experimentation you make small changes in order to maximize your speed and efficiency in getting to, over and past each hurdle.  Not paying atttention to how you land after a hurdle means you may not approch thet next hurdle most efficiently.  As you continue this practice and feedback loop (Retrospective) you can begin to reach ever higher levels of speed and success.

Eventually you don't see the hurdles, they are just part of the process of completing the race.  They aren't obstacles anymore.

This approach is how athletes get to world class levels.

How is Product Delivery like running the hurdles?  With each step from ideation to delivery there are hurdles that we face that keep the entire delivery process from being efficienct and effective.

And more importantly, whereas a sprinter has the luxury of knowing that their hurdles are 30ft apart and either 39"-42" tall, Agile Product Delivery teams don't always know where their hurdles are at, but through effective use of Scrum, teams can begin to identify where the hurdles are and start the process of learning how to approach and exit each of those hurdles, using Retrospectives.

We too can get to a point where our hurdles are just part of the process and if you aren't running the hurdles anymore what are you running?  A sprint.....

What are some of the common hurdles that we face in Agile Product Delivery?

  • UX Design not clear
  • Lack of Product Vision
  • Poorly formed product backlog/user stories
  • Lack of technical excellence - Continuous Integration, Automation, code reviews.

Successful Agile delivery is all about identifying your hurdles, getting them lined up appropriately and working to maximize the efficiency of the steps we need to take to get to the end.

As with world class athletes we need to continue to evaluate our approach and drive world class product delivery for our customers.

Delivering Software Fast is Like Running the Hurdles

I had the opportunity to attend a couple of Dan North's Deliberate Discovery sessions at the Better Software East conference this month and as he talked about how we want to deliver software faster I started to envision that delivery of software is much more about running the 100 meter hurdles than it is about running the 100 meter sprint. With the 100 meter sprint the runner knows that from start to finish there are no obstacles and that they can maximize their speed through focus on the finish line.

Organizations aren't like that, we pose so many different hurdles that even if you believe that you are running a sprint, in reality you are probably running the hurdles.

With hurdles a sprinter must continue to work on the way that they will approach the first hurdle and then the next and so on and so forth.  Along the way anything might 'trip' them up.  When they clip the top of the hurdle they have to make adjustments almost in mid air to try to recover.  The ability for great hurdlers to continue to work on their approach to each hurdle ensure that they can run fast competitive races.

Software development is very much like the hurdles.  Even if your Engineering team is able to deliver features quickly, if your Product Development or QA groups aren't aligned with that speed then no matter how fast you run between one hurdle you will be struggling to make the next one because you haven't optimized the steps in front and behind the hurdle that you do well.

In order to deliver software fast organizations need to look at each of their Product and Software Development cycles so that each one maximizes the delivery of the previous steps. More to come on this......

Member Login
Welcome, (First Name)!

Forgot? Show
Log In
Enter Member Area
My Profile Not a member? Sign up. Log Out