There have been 3 distinct phases in this debate:

1. CMM Phase: Manufacturing has transitioned from craftsmanship to mass production – productivity and quality has improved many-fold. Software development can also benefit from such transition. CMM movement was born from this thought.
2. Agile Phase: Manufacturing deals with machine, software development deals with people. Processes involving machines can be controlled precisely. People are inherently different and are not interchangeable. People communicate better face to face rather than through written documentation. From this realization agile movement was born.
3. Lean Phase: Toyota revolutionized manufacturing through lean manufacturing system and dramatically improved quality and optimized cost. The core of lean manufacturing is empowered teams. Since agile movement also is based on self-organizing teams it must be possible to transplant the learning from lean manufacturing to software development. This lead to lean software development.

There is an apparent logic in all three reasoning. So, which advice should you follow? Are they compatible with each other? Before answering these questions you should look at the differences between manufacturing and software development.

Will this gap ever be bridged? Will software development move closer to manufacturing?

I doubt it – here is why.

Repeatability vs. Uniqueness:

“…Every advance for the future state of the world requires the presence of software yet to be written…” – Grady Booch (here)

You are never going to write software which already exists!

Well-defined vs. Incomplete & Evolving:

The authors of the Agile Manifesto understood it when they wrote:

“…Responding to change over following a plan…”

“…Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage…”

Software is about advancement … software is about new idea … not all new idea succeed. Trial and error cannot be avoided.

Known vs. Unknown:

Todd Anglin in his very interesting post The Future of Software Development states:

“…There is no end in sight to the deluge of new technologies being delivered to market…”

“…If there is any one concept or idea that software developers must absolutely comprehend, it is this: it really is okay not to know it all…”

The rate of technology change is accelerating. Technology options are increasing. Current software, for all practical purpose, has infinite features! The world is becoming more and more interconnected. Hence, software developers of the future will need to handle more unknown, not less.

Machine & Tool vs. People:

Daiv Russell, in the article Hold On to Your Top Performers, states:

“…Most CTO’s realize that The Pareto Principle applied to IT staff means that 20% of your people are delivering 80% of your entire team’s bottom-line value…”

“…The difference between the extremes has been reported as high as 100:1…”

Unless we can find a method of completely automating the process of software writing, the dependency on people will remain. The chances of automating the process of software development in foreseeable future look remote.

What can we conclude from this?

Reject all idea which is derived or borrowed for the purpose of managing predictable processes.

Accept all idea which helps you to deal with unknown and uncertainty.

Now, I am also a manager in a business. That business has stockholders and the stockholders expect a return on their investment.  Moreover, they (and my managers) expect not just a return, but the maximum return on their investment.

Here is my dilemma and here is my epiphany:

As a problem solver, the best solution is what I strive for, but I am often frustrated when I am told to stop short of the best solution or to solve a much smaller problem instead.  I am not talking safety issues here, those are always handled as a top priority (really they are), but I am speaking about real functional defects that users hate and that I (and many of the people I work with) cannot stand the existence of.  The stupid fact, that actually only dawned on me recently, is that a business is not interested in solving every problem there is with their products.  The goal of a business is not some ultimate quality product that will clobber the competition, but instead a business is interested in finding (what I call) their customer’s “toleration point”.  The “toleration point” is the point at which the market/customer will continue to pay for and be satisfied with something that is incomplete, defective, or marginally effective. If you have ever used software, then you know what I mean.  You probably have lost documents in your favorite word processing program – but it works most of the time, everyone else uses it, and you know how to do stuff with it.  Corporate tools must determine what you can stand and then walk right up to that line.  This will never be the best solution from the stand point of a craftsman, but it will be the best solution from the standpoint of a stockholder.

Why do I care?

Well, engineering is about doing the right thing the right way (to the best of your ability).  Even when you make compromises in your designs, it is for a larger purpose that makes sense in the physical world.  When you put on the business hat, you have a total shift in reality and it is very jarring.  Getting something out that will build a habit in your users becomes a top priority — regardless of the quality.  The business will look at revenue factors and consider that they can always find someone to cheaply fix stuff later (as needed) as they make money now.  It does not always end with the best product or solution winning, but with the most popular (and often the cheapest) product winning.  That really bugs me, and yet that is how economies have run since there were economies.  So as you read this on your computer that downloads patches, updates, and security fixes almost weekly, you need to think about how the world really runs and whether we are all corporate tools or the ultimate pragmatists.

 Agile is more about the journey than a specific destination.  You should maintain awareness of how well the journey is going instead of how long you’ve been traveling.  How can we explain this, in a simple manner … Read More

Empty Production Line

You would think after 18 months of forming and stabilising my development team that we’d have a pretty strong and settled process in place.  But the development team is only one player in a larger ecosystem and how that team is fed depends heavily on the requirements that they are given.

When you’re in the software building business, the pressure to deliver something – anything – by a certain date can be quite intense.  Sometimes its because a marketing campaign is scheduled, sometimes because a product is being released that the software needs to tie into or it may be that a service has been promised to customers or end-users by that date.  Whatever the reason, developers work best when given a fixed set of requirements to deliver by a committed, set target date.

Contrast this with a situation where one or more of the following situations occurs regularly:

• The client provides requirements with an assurance that they are “final and signed-off”, but continues to insist that small additions or amendments be made right up until – or even after – the final testing process has begun.
• The client – in agreement with you – settles on a deliverable date by which all scoped work should be provided.  This is then casually pushed out to accommodate scope creep or alter the requirements.
• The functional requirements are running late and the client asks that you just get started with what they’ve documented so far and the rest will be added in as you go.
• The final requirements provided to the development team include numerous assumptions, vague functional specifications and “documented omissions” that will – assuredly – be provided in due course.

People generally don’t enjoy working like this.  There is motivation, job satisfaction and a personal sense of pride (among other benefits) to be found in the challenge of achieving a goal.  People – and developers in particular – work well under conditions of managed stress and challenges where there is a tangible reward to be found at the end.  That reward is self-respect, the pride in achievement, appreciation, recognition or respect of the client and stakeholders and satisfaction in the delivery of a quality product.  Take away the measures for those motivators and you take away the motivation as well.

The Problem

Consider these scenarios:

1. Your development team is working to a tight deadline, busting themselves to achieve it.  At the last minute it gets moved out. Your team was expecting victory, some congratulations and appreciation (maybe a cake), and instead all they get is more work.
2. The team is working to a difficult deadline, and must cut corners. They build things quick-and-dirty because time is short and it’s better to have something ugly that works than something beautiful half-built. The deadline is pushed back and now the team (or, more likely, the Team Lead) have to explain why they did such a rough job. The time to clean it up is longer than the extension.
3. The deadline gets moved out, so the team realigns priorities, working on the new goals for the altered requirements.  The estimate for work is recalculated and work carries on towards the new deadline.  This happens once – maybe twice – more before the final deliverable is released.  The line manager asks the Team Lead how the original estimate compared to the actual development cost and the Team Lead must explain why the scope and cost started at Point A for $10 000, but finished at Point C for a total cost of $25 000.  (Even worse is if you have no adequate measurements of effort spend per developer and cannot even provide figures for how the estimate compares with the final product).

Vague deadlines open the door for:

• Delivery dates that are hard to commit to
• Meaningless estimates
• Impact on scope for other deliverables

Conversely, vague requirements lend themselves to

• Scope creep
• Technical deficiencies
• Redundant or inaccurate estimates

As a product owner, line manager or client, you should be concerned about this because

1. Your development costs begin to run away on you
2. Your product quality slips
3. Your development team becomes disillusioned, disaffected or cynical
4. Future projects are put at risk as your development team becomes accustomed to deadline extensions and scope changes

Any one of these issues should be enough to cause concern – if not from a purely cost point of view then from a risk and succession planning one.  So what steps can be done to ensure that the psychology of your software shop team is healthy and robust?

The Solution

When it comes down to it, there are two broad ways in which you can remedy the state of delivered requirements and conditions under which you are expected to deliver the final product: influence the client (i.e. require them to adapt their processes to accommodate your needs better) or influence your environment (i.e. adapt your own internal processes to compensate for those factors outside your control).  The first is harder to achieve in large amounts although depending on what is asked of the client and how the request is packaged up, some results may be better than others.  The second allows more immediate and practical scope for change, but may not provide the breadth of control needed to effect meaningful results.

To this end, the following strategies can help you add value to your own service delivery:

• Ensure you have tools for defining when a deliverable has been satisfied.
• Ensure estimates are based on tiny tasks, each no longer than a day or two.
• Ensure requirements are measurable (e.g. “Function X should work in exactly this way…”) and work with the client until they are.
• Build to your own shorter fixed time frames (e.g. monthly) that can be dove-tailed into the client’s delivery deadlines.
• Deliver only to your agreed time frames and initial scope, with any deadline or scope changes being pooled into a backlog to be commenced on completion of the original project requirements.
• Ensure you leverage the tools you have to give you meaningful information about how your production process is performing and identify ways in which it could be streamlined.

The concepts around Lean Software Development provide excellent, non-prescriptive strategies for approaching the development process and for looking at key areas in which to improve: in particular, minimising waste.

Well-Tuned Production

Ultimately, the best approach is to engage with the client and agree on a requirements / product delivery process that affords each party the flexibility they need coupled with the policies against which they can most effectively operate.  Setting the expectation with the client that late or incomplete requirements will result in those affected features being delayed for a future iteration (or until the requirements are satisfactory) and then committing to that will create a culture where there are consequences for a relaxed approach and reduces the waiting around or re-work for the development team.

No project – or any other relationship for that matter – can operate in strong and healthy way unless both parties are communicating effectively and often.  The development team can feel empowered and resolute with fixed, predictable deadlines and a set of confirmed, explicitly-defined requirements.  The client can feel more comfortable about their requirements delivery knowing that they can provide only those instructions that have been properly thought-out and need not operate under pressure to some intangible time frame.  Both parties know that if something can’t be provided in this iteration, it will not be forgotten and gets scheduled for completion on the next pass.

Protecting the head space of the development team is often stated to be the sole responsibility of the TTL, however the reality is that it often takes the contributions of a number of stakeholders to really create the sort of environment where the developer can fully thrive.

Traditional testing is focused too much on bug hunting, mostly after a development cycle and including a devious rewarding or penalty policy. There is too little attention for verification of quality while building.

The development process should be oriented towards creating quality upfront (i.e. before UAT, release or whatever post-process control), and verification during the development process. If verification reveals a defect, the ‘line’ is stopped, the cause identified and resolved.

My My.Fragility framework holds Quality Loops. These are performed continuously and at least daily. It is based upon eXtreme Programming practices. They serve as engineering standards within the Scrum process that is at the heart of my framework. It shows our focus on quality and integrated testing:

I’ve been fan of the concept of kaizen or continuous, incremental improvement, for a couple of years now. Now I can add to that muda, the Japanese term for an activity that is wasteful and doesn’t add value or is unproductive. I wish I’d been exposed to lean principles last spring when I was talking to my manager about how I wanted to handle documentation bugs.

When I was hired last October there was a pretty big backlog of documentation bugs because 1) the company had been without a technical writer for six months before I was hired and 2) the Support manager had recently had the members of the Support organization read through the entire doc set and enter bugs and enhancement requests.

Since I’m a lone writer, I would prefer that when someone finds a simple typo (misspelled word, missing punctuation) that they simply e-mail me so that I can correct it. This would lead to a very simple workflow:

1. Employee sends an e-mail to the technical writer.
2. Technical writer reads e-mail.
3. Technical writer opens document files and fixes typo.

This workflow can lead to a simple typo being fixed within a few minutes of it being found. As a fan of efficiency, this is my preferred method of dealing with typos. If someone e-mails me a problem that takes more than five minutes to fix, it makes sense to create a documentation issue to track the work on that issue (and to make sure that it doesn’t fall between the cracks and get lost in my inbox).

However, my manager wanted to track ALL documentation issues, no matter how small, in our issue tracking system. This leads to a workflow like the following:

1. Employee enters an issue. Note that not all of our employees have permissions to create issues in our system, so the employee may need to find someone else to enter the issue for them.
2. The issue has to be triaged by one or more managers, assigned a priority and a target release, and assigned to the technical writer.
3. When the writer receives the automated e-mail that the issue has been assigned to them, they may have to go to the issue tracking system to view the issue.
4. Writer opens the document file and fixes the typo.
5. Writer has to go back into the issue tracking system and change the issue status to “fixed” and then assign it to the QA manager. Because I don’t generate PDFs on a daily basis, this may not happen until days or weeks after I’ve actually fixed the typo.
6. QA manager assigns the issue to a QA tester to verify that the issue has been fixed. Sometimes this doesn’t happen until we are nearing a patch release, as the testers are testing code, not checking that typos have been corrected.
7. QA tester checks the documentation and closes the issue.

This was the workflow that we used at one of my previous jobs, where I was a member of a large doc group and the writer responsible for any particular document might change at the end of a release. But since I’m the only writer at my current job, and typos aren’t something that I need other members of the team to assist me in correcting, it doesn’t make sense to me to have so many people involved in verifying whether or not a word was spelled correctly or a period was added at the end of a sentence. Instead of a process that takes two people and five minutes, my manager wanted a process that involved at least seven people and could possibly take days or weeks to complete.

This just seems wasteful to me and not very Agile.

How do other lone writers, who don’t have peer reviewers or editors, handle editorial tasks? Especially in an Agile environment?

Since I’d never heard the term Kanban before, I hit Wikipedia before the meeting to look it up.  But only ended up confused about how a manufacturing process applied to software development? 

After the meeting, it seems that the answer is not so much kanban, but “lean software development.”

I only took a few notes, mostly listening and trying to understand what was, for me, a new concept.  Damon started out giving a brief overview of Agile (which was useful for the majority of the audience, who were not working in an Agile environment).  Then he talked a little bit about Lean.  My notes say “providing value” and “continuous improvement”.  Aha,  I know that one, kaizen!   He mentioned 14 critical mass Agile practices, but I was only able to scribble down five before he flipped to the next slide.  I do hope that the slide deck is posted online at some point.

The next part of his talk focused on process.  He had some color coded slides that showed several iterations, with Development doing their work during one iteration, and QA doing the testing during the next iteration.  My notes say “This is not Agile.”

He talked a bit about how to break down large stories into smaller ones by listing the tasks, and then determining the critical tasks and the dependencies between them.  He had some great slides for this, which I appreciated, because I’m a visual learner.  And I was grateful that when he talked about workflow that he included Documentation (Specify > Design > Code > Unit Test > Integration/Doc > Testing) because so many people seem to forget or ignore Doc when talking about Agile.

Then there were color coded slides, showing several small stories being worked on during several different sprints and showing how there were bits of downtime for the testers while they waited for the code.  Then he had a rather spiffy animation that removed the lines indicating the iterations, and sliding the testing tasks right up to match the coding tasks.  “Getting rid of the iteration fills the gaps between coding and testing.”

He talked about decoupling the tasks of Agile from the iteration/Sprint.  Backlog grooming can happen anytime.  Story point estimating can happen anytime.  These tasks can be decoupled from the Retrospective or the Planning phase of a Sprint.  “Done” is decoupled.  Done is no longer the end of the iteration, but the end of the story.  And when you’re finished with a story, you pull the next story off the top of the backlog and start working on that.  He also talked about a Work In Progress (WIP) limit, that no more than X story points could be in progress at any one time.  During the Q & A someone asked when you demo?  His answer was not every time you finish a story (which might be every day or two) but when it makes sense to the team.

Damon finished his talk with a list of Lean and Kanban concepts that he thinks can be applied to Agile:

• Decoupling
• Lean thinking
• One piece flow
• Work in Progress limits
• Eliminating waste

*****

Related Blog posts that I found while researching Kanban:

Defining Kanban

Between kanban and pair programming lies the feature brigade

Related Books:

Do It Yourself Agile by Damon Poole – Free download!

Lean Software Development by Mary and Tom Poppendieck

Agile Minds, Lean Thoughts

(This entry was originally published in the Fast Frontier blog (Part 1 | Part 2) by Drew Jemilo)

You’ve likely read the Agile Manifesto, the values which were developed by representatives from Extreme Programming, Scrum, DSDM, Feature Driven Development, Crystal, and other alternatives to the heavy, document-driven Waterfall methodology.

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The Manifesto statements offer traceability to many of the Agile management practices. If we use Scrum management practices for our examples, we can make the following connections:

• The daily 15-minute Scrum meeting with the team standing around the low tech task board supports the value of individuals and interactions over processes and tools
• The Sprint Planning meeting, where the light-weight User Stories are transformed into a shared understanding of the features to be implemented, supports the value of working software over comprehensive documentation
• The application of “good” versus “good enough” in order to deliver on time supports the value of customer collaboration over contract negotiation
• The re-prioritization of the Product Backlog before each Sprint Planning Meeting supports the value of responding to change over following a plan

Agile Values vs. Business Value

Though the Manifesto articulates the values that provide the foundation for the “what” and “how” of Agile, it doesn’t answer the question “why.”

Why are individuals and interactions more valuable than processes and tools? Many people could argue that individuals are inconsistent, their memories are fallible, and their interactions are unpredictable. Shouldn’t defined processes bring consistency and tools record decisions which may otherwise be lost or reinterpreted?

How do we move from Agile values to business value?

Finding the “why” in Lean

Birthplace of the Agile Manifesto

When the Agile Manifesto was developed in 2001 at the Snowbird Ski Resort in the Wasatch Mountains of Utah, Mary and Tom Poppendieck had not yet released their book on Lean Software Development.

Lean Software Development draws heavily from the rules of the Toyota Product System, later called Lean Manufacturing. Simply put, it is a time-tested method which focuses on providing more value with less work.

More value with less work. Now that’s something which any CEO could understand and support.

Mapping Lean Manufacturing to Agile

There are ten rules which can sum up the fundamental practices of Lean Manufacturing in the 1980′s. By looking at both Lean Manufacturing and Lean Software Development, the “why” emerges for the Agile management practices. These management practices can then be traced to the fundamental goal of producing more value with less work.

Why Agile Management Practices Produce More Value with Less Work

Birthplace of Lean Manufacturing

Rule 1: Eliminate Waste

In Lean Manufacturing…

• Value Stream Analysis is used to identify all activities in the manufacturing process and the value they add to the final product, not the process. The process is then re-evaluated to find the most efficient way to add the intended product value.

In Agile…

• The biggest waste in software development comes from building features which will be outdated or have little value when finally released. A high priority feature today may be irrelevant in a year. In Agile, the business re-evaluates requirements against changing customer needs before each Sprint so that only the highest priority ones are built.
• In Agile, the User Stories are only frozen for the Sprint (though how they are implemented is negotiated throughout). The business is less hesitant to commit to requirements for the Sprint because they have the opportunity to re-prioritize before each Sprint. They waste less time on forecasting needs and the sign-off process.
• The production of documentation which has no value after the completion of an iteration translates to the scrutiny of document production. Which documents add value to the final product, and which simply support the process? Agile replaces verbose requirements documentation with light-weight User Stories and Validation Conditions. The design is then elaborated through conversation in the Sprint Planning Meeting and the daily Scrums.
• In the Sprint Planning meeting, only the highest priority requirements are broken down into tasks so that time is not wasted on features which may never be implemented. This process is often referred to as “just-in-time elaboration.” Time is not wasted on details which may never be needed.

Rule 2: Minimize Inventory

In Lean Manufacturing…

• Inventory is minimized because it costs money, consumes resources to manage, and becomes obsolete.

In Agile…

• The inventory of requirements is kept at a high level and broken down “just in time” so that unnecessary details don’t need to be managed. During the Release Planning process, high level Epics are created, but not broken down into stories until they are targeted for a release.
• Epics and stories are managed by the business and regularly pruned to eliminate the constant review and re-shuffling of the low or no-value items.

Rule 3: Maximize Flow / Reduce Work-In-Process

In Lean Manufacturing…

• Smaller batches and faster cycle times increase the flow of products from the manufacturer to the customer.

In Agile…

• Short Sprint cycles produce completed units of functionality which can be released to the customer. This allows a smoother flow of features, more frequent customer feedback, and faster responsiveness to change.

Rule 4: Pull From Demand / Decide as Late as Possible

In Lean Manufacturing…

• Pulling from actual demand reduces reliance on and investment in forecasting models. Driving from actual demand is more reliable in volatile environments.

In Agile…

• Freezing requirements at the beginning of a long development process relies heavily on forecasting. By re-prioritizing the Product Backlog before each short Sprint begins, features are driven by actual demand rather than speculation.
• Rather than trying to forecast architecture needs, Agile software engineering practices keep the initial architecture design light-weight and allows for refactoring (reworking of code). This keeps designs simpler, but allows additional architecture elements to be added if needed.

Rule 5: Empower Workers

In Lean Manufacturing…

• Teams are trained in work measurement and improvement techniques. Changes which are driven from the workers “on the floor” improve productivity and morale over changes mandated by management.

In Agile…

• Teams are self organizing, and re-evaluate their processes after each Sprint in the Retrospective Meeting.
• The team is empowered to decide how the stories are implemented. Through User Stories, the Product Owner defines the “who,” the “what,” and the “why.” The “how” is collaboratively developed by the team.

Rule 6: Meet Customer Requirements

In Lean Manufacturing…

• This rule is self-evident, and follows Rule 1 (Eliminate Waste). Manufacturing products which no one buys results in inventory which will never be sold.

In Agile…

• In traditional Waterfall, success is measured by delivering the required features on time and on budget. The delivery of the right requirements is often overlooked. As in Rule 1, the creation of low or no-value features is the most wasteful aspect of software development. Success is measured by delivering only the highest priority features each release in a “good” or “good enough” manner.
• Before the Sprint begins, the business creates validation conditions for each User Story. These validation conditions become the business’ acceptance test criteria. This means coding is driven by tests written by the business, not designs and tests created by the developers.
• During the Sprint Review meeting at the end of each Sprint, the functionality is reviewed with the business to confirm that the requirements have been met.

Rule 7: Do it Right the First Time

In Lean Manufacturing…

• Lean integrates quality at the source, putting controls in place at each step in the process. Before Lean, manufacturing was optimized for product throughput. At the end of the process, tests were conducted and defects were fixed at rework stations.

In Agile…

• Tests are written before code, and defects are fixed at each step in the development process. Barry Boehm observed that it costs 100 times more to fix a problem after release than early in the process.
• In an Agile organizational structure, QA is not a silo. Instead, QA personnel are integrated into each team.
• The change control process becomes simpler and less costly because fewer defects are passed on.

Rule 8: Abolish Local Optimization

In Lean Manufacturing…

• For rapidly changing markets requiring responsiveness over mass production, faster machine change-over is more important than the higher throughput of individually optimized stations.

In Agile…

• Cross training and team collaboration are more important than siloed roles. This allows team members to jump in where needed rather than passively waiting for someone else to complete a task or correct an issue blocking their progress.
• Teams are responsible for delivery rather than individuals. Rewarding individual performance (local optimization) rather than team performance detracts from each persons focus on the overall objective.

Rule 9: Partner With Suppliers

In Lean Manufacturing…

• Companies are able achieved the the greatest efficiencies in their supply chain by reducing the number of suppliers and working with them as partners. The efficiencies gained by trust, collaboration, improved product flow, just-in-time movement of goods, and less supplier turnover outweigh the savings which come from competitive bidding.

In Agile…

• Just as moving parts between suppliers is expensive, so is moving tasks between siloed departments. Agile distributes siloed departments into the teams.
• Integrating the business into each team with daily contact creates a partnership which provides just-in-time elaboration of business requirements and more cooperative relationships. Requirements are not “thrown over the wall” with visibility only at the end of a long development cycle.

Rule 10: Create a Culture of Continuous Improvement

In Lean Manufacturing…

• As Rule 5 (Empower Workers) states, everyone involved in the process has the power to improve it. In contrast, a command and control environments rewards adherence to a strict process. Assuming that processes can continuously be optimized, especially in changing markets, a culture is needed to support continuous improvement.

In Agile…

• At the end of each Sprint, the team conducts a Sprint Retrospective, discussing two simple questions: what went well and what can be improved. Software maturity models such as ISO9000 and the Software Engineering Institute’s (SEI) Capability Maturity Model (CMM) reward adherence to strict processes. It assumes that their specific process is right for every situation. Agile assumes that as markets change, requirements change, and the processes to meet those requirements change as well.

Agile: More Value with Less Work

Tracing Agile back to the time-tested rules of Lean Manufacturing, we begin to move from philosophical values to business value.

• Rather than telling a CEO that Agile values individuals and interactions over processes and tools, you can tell her that Agile eliminates waste and creates productive partnerships between all parts of the business.
• Rather than telling a CEO that Agile values working software over comprehensive documentation, you can tell her that Agile speeds up the delivery of valuable functionality to the customer.
• Rather than telling a CEO that Agile values responding to change over following a plan, you can tell her that Agile delivers functionality based on current demand rather than forecasted needs or outdated requirements.

And the list goes on: higher quality, continuous process improvement, and stronger collaboration. Your Agile value proposition is suddenly defined.

Keeping things neat and simple and fulfilling the acceptance criteria of implemented features with the least amount of code helps the project move forward quickly and achieve its goals.

How to implement the same amount of functionality with less code?

• Practice test-driven development (TDD) – TDD is in part a method for testing code, but mostly a method for design. Adding new code just to make a failing test pass keeps things simple design-wise. Once all the tests pass, you’re done, no bells and whistles necessary.
• Use third party libraries – Using proven third party libraries saves you code. And most of the time they’ll work better than homegrown ones. Example for this is the Apache Commons project, which provides various useful Java libraries. Delegate as much work as possible to the underlying platform – use EJBs and JPA for persistence instead of JDBC, rely on container managed transactions and not on application managed transactions, use JSF instead of pure Servlet/JSP.
• Refactor continuously and mercilessly – Whenever you spot an area in the code that just looks too complex and bloated, try to simplify it. Practicing TDD, you already have the tests which will verify that the functionality is not broken.

The picture and more information around the topic was found at: http://www.codinghorror.com/blog/archives/000691.html

1. Time-box everything and stick to it!
2. Anticipate change in everything you do.
3. Don’t build monuments, keep it light and malleable.

And the market compulsions have been explained succinctly too : Change happens fast, our competitors will not wait for us while we get ready! and Learning faster than our competitor is our only sustainable competitive advantage

Lean Thinking Introduction by Vikrama Dhiman

From Google Tech Talks. It is a long video but well worthwhile to watch it.

Kan

In an Agile context, a Story Card is a Kanban. Out of eXtreme Programming it grew into an established Agile practice in describing a feature. A Kanban Board holds the active Story Cards per ‘status’, thus being an Information Radiator (term by Alistair Cockburn, also my inspiration for the Games metaphor).

My Scrum Product ànd Sprint Backlog items are User Stories. (‘Minimal Marketable Feature’ is more generic, but I stick to ‘User Story’) The Sprint Backlog is made visible by sticking a printout from my (Excel) tracking model on a wall. Around the Daily Scrum (we do it standing up) each Story Lead writes the estimated time to finish a Story on the printout and I create the Sprint Burndown. That works fine. We use a Kanban Board only for feedback from functional testing. This is generally too small to create a Story (would be ‘waste’).

Velocity is a way to optimize the inventory and the continuity of work. Traditionally ‘velocity’ equals the #Story Points (gummy bears) that can be finished in one iteration (a Sprint). However, I apply an overall Velocity as a multiplicator on Story Points (ideal time) to result in #planning days. The size of the inventory (in #Story Points) for a Sprint is determined by dividing the available #planning days of a team (minus 2 slack days) by expected Velocity (Yesterday’s weather). Note: continuity is also assured by the presence of the Product Owner. He/she makes sure that no functional issue remains unsolved, on the spot!

We started with a cup of steaming coffee on a beautiful rainy day at Hotel Claridges. When the clock struck 10 AM, I heard Mary telling me “Lets get started” – I was impressed by her punctuality which she displayed at many more ocassions during the sessions – and proceeded to quickly introduce them and thank Xebia (the sponsors) and ASCI (they are behind gettin all these great guys to India to share their learnings with us). Mary took over from there on.

Mary narrated how the company she was working with faced stiff competetion during mid 80s. Innovating and “cutting the waste” seemed to be the only way to survival – as they say: Necessity is the mother of all inventions

During 1998-99, Mary happened to work on a government project and was introduced to the “Waterfall Model” – she was surprised and could hardly believe people were developing software that way! On completing the project, she decided to formalize her thinking about software development and her learnings from other industries, specially 3M- her former employer.

Principles of Lean Software Development

• Eliminate Waste
• Embed Quality
• Learn First
• Deliver Fast
• Improve Forever
• Respect People
• Think System

Having defined the above principles, she then moved on to giving a few crisp and concise definitions:

Waste is…
Anything that depletes resources of time, effort,
space, or money without adding customer value.

Value is…
Seen through the eyes of those who pay for, use,
and derive value from the systems we create.

A System is…
A complete and useful product, along with whatever
is needed to create and distribute the product.

This was followed by drawing up an intersting parallel between “The Seven Wastes of Manufacturing Industry” as defined by Taiichi Ohno and the Seven Wastes of Software Development:

1. Extra Features
2. Partially Done Work
3. Relearning
4. Handoffs
5. Task Switching
6. Delays
7. Defects

The baseline being: – “Put on the customer’s glasses and then decide – if its not value, its waste”
This naturally led us to the mention of the famous 80-20 principle that states that “80% value is derived from 20% features“

We then pondered over the thought that this should mean fewer lines of code are better, fewer function points (only as much as required by the customer) as better than lots of FPs – but traditionally the companies measure productivity and performance in number of function points – thus it needs a change of mindset rather than just the processes to really adapt Lean Way successfully.

We all nodded that there’s a need to think out-of-the-box.
Mary illustrated with a beautiful example from manufacturing industry:
One of the machines took 3 days to setup whenever there was a change of die required. So the “common knowledge” said

• Die changed have a huge overhead
• Don’t change dies very often

While Taiichi Ohno’s out-of-the-box thinking believed that:

• Economics requires frequent die change
• One Digit Exchange of Die

As it turned out, it really was possile to do a “One Digit Exchange of Die” – The setup time came down from 3 weeks to a whooping 9 minutes!
If we apply this theory to software dvelopment, it turns out that while “common knowledge” states:

• Releases have a huge overhead
• Don’t release very often

The Lean would suggest:

• Economics requires many frequent releases
• One Digit Releases – welcome to the concept of iterative and incremental – did we hear “sprints”

Indeed, if we come to think of it, “One digit releases” do make a lot of sense – Learning from the “real user” of the system would be the best learning.
Which in turn means frequent customer feedback that comes from frequent “one digit” releases.

An interesting kind of waste was discussed – “Wishful thinking”. Its when we “hope” or “assume” things like availability, roadblocks, capabilities et al rather than “knowing” these parameters based on facts.

“Hand offs”, another kind of waste occurs when we try and seperate:

• Responsibility – What to do
• Knowledge – How to do it
• Action – Actually doing it
• Feedback – Learning from results

This is when we discussed the importance of “Cross Functional Teams”.

Mary quickly discussed “Task Switching” and “Delays” as other forms of wastes, illustrating with the example of The Empire State Building – how it was completed in 1.5 years defing the common beliefs and all odds.

“Pulling” the work rather than the work getting “Pushed” seemed to be the common sense way of working.

“Defects”, arguably the most important waste, can be avoided by “Inspection”.
Again, inspection itself might be a waste!

Inspection to FIND defects is a waste, pro-active inspection to PREVENT defects is essential.

The importance of making the system error-proof at every stage came up – taking an example from hardware industry, its impossible to plug-in the monitor cable into your laptop the “wrong” way – if it fits in, it got to be right!

Testing at each step was emphasized and four categories of testing were introduced.

At this point Tom pitched in with an enticing statement – “If you don’t engage in getting what the customer really wants, in getting the big picture, in interacting with the client, then you are not a developer, but a mere coder!”

Mary then introduced “Value Stream Map” as a diagnostic tool to find the biggest waste (opportunity for improvement) in a development process.
We quickly went on a short break and reassembled in 10 mins to do a hands-on excercise Mapping the Value Stream of one of the real life processes from our companies. We realised how small wastages at each step accumulated to give a very low efficiency and why it is so important to eliminate waste and staying lean.

As Claridges people gently suggested that lunch was ready and its aroma made us hungry, Mary and Tom Poppendieck decided to call it a day. We had some sumptuous food and proceeded to Xebia office for the second part of “Open Discussions” – this one being exclusively for Xebians.

I’m applying Scrum (over 4 years already), in my projects and in my management. It continuously helps me to cross fences; between suppliers and customers, business and IT, analysts and developers, x-layer developers and y-layer developers, etc.

In my Scrum-based development process I use Pre-Game Staging to name the barely enough preparative phase for 1 stage of software development. The “Games” metaphor was described by Alistair Cockburn in his biblical work Agile Software Development. I strongly agree with him that software development should be a Cooperative Game.

• Cooperative: a team of people works together towards a common goal, not to fight each other.
• Non-zero sum: the players are not opposed and do not try to win by getting the other at zero. There are multiple winners.
• Finite and goal-seeking: the game ends when the goal is achieved. The game is not meant for just staying in existence.

You will find this as well in the integrated prerequisite for supplier-customer relationships (possibly multi-tier), from Toyota’s Lean Production to Poppendieck’s Lean Software Development. It offers far more certainties than our commonly applied bidding processes. Think (read) about it!

As developers we naturally tend to make assumptions as to what the user, or system, will need. In truth, it is very difficult to accurately predict the future requirements of a system. Coding to meet our assumptions will inevitably lead to over engineering.

We should not let our intuition guide us into false requirements. As strange as it may sound, it takes a concious effort to stop ourselves from doing extra work. We need to adopt a ‘lean coding culture’. We need to constantly remind our selves to stick to lean software development. In agile teams it is common to hear developers reminding each other with certain acronyms and expressions e.g. ‘do the simplest thing that works’.

‘Do the simplest thing that works’ is in-line with lean software development, although it can be misleading. Some people interpret this as ‘do the first thing that comes to mind’ e.g. slap on another if statement – that is a simplistic solution and not a simple solution. If you take a quick and dirty simplistic approach you will end up with a big ball of mud. You need to do the simplest thing that works. The thing that works is something that can be tested, it is something that does not add unnecessary complexity. Therefore the simplest thing that works is not necessarily the quickest and easiest thing to do.

When talking about lean software development we often refer to KISS (Keep It Simple Stupid). I agree with keeping things simple but I don’t like to be called stupid, the truth hurts. I prefer to use: Keep It Simple and Sensible. This is more positive and the sensible part indicates that we should give some thought to our code. For example, instead of littering your code with conditional statements you could refactor the code to use an established design pattern. Having said that, design patterns are not always a good idea.

You should not implement a design pattern unless there are real signs indicating that you need a smarter solution. It can be detrimental to implement a design pattern for the sake of having one. You are likely to implement the wrong pattern and that code will be a hindrance. Only do what you really need to do. This is also referred to as YAGNI.

YAGNI (You Ain’t Gonna Need It) – This is a term used by Russ Olsen in Design Patterns in Ruby. In this book, Russ Olsen does a fantastic job of explaining design patterns. He also explains when not to use certain patterns and the problems that come with using each pattern – After all, software is about trade off’s.

Russ Olsen explains that we should not write code or implement patterns in anticipation of a requirement that might emerge. Firstly that requirement may never surface, and if it does it may have a stipulation which makes the pre-emptive solution void. It is safer to code to meet the current and real requirements of the system. To drive this point home, here is a delightful paragraph from Russ Olsens’ Design Patterns in Ruby:

“Look at it this way: Barring a sharp blow to the head, as you stand here today you are as dumb as you ever will be. We are all learning, getting smarter every day. This is especially true in software projects: You can be sure that you will have a better grasp of the requirements, technology, and design of any software that you work on at the end of the project then at the beginning. Whenever you put in a feature before you really need it, you are guilty of programming while stupid; if you wait until you really do need the thing, you are likely to have a better understanding of what you need to do and how you should go about doing it.”

The moral is: Don’t be guilty of programming while stupid

But, then it occurred to me that there is really another difference, I’ll try to describe my thought by raising the question to you and your development team: What are you actually selling to your customer, how do you create value?

I’ll give you three alternatives: 1) are you selling a plan, 2) are you selling a team or 3) are you selling a product.

Selling the plan

Selling a plan, to me,  is not agile or lean at all. Let’s say that the customer would like to have some kind of software development, and the intentions, business case etc. are all valid and correct. The customer contacts development company MasterPlan.Net and they will create a plan describing how they are going to create the value needed.

The assumption here is that if we follow the plan, then something valualble will be created. Let’s look at the other part:

Selling a team

When selling a team; the customer can choose to use the team in any way he like, if it is to develop a piece of software, build a house etc. in theory it does not matter.

If the customer has a vision, a product backlog and uses agile methods such as Scrum, then there will most likely come something valuable out from the team every 2-4 weeks.

The cost for producing the software is based on the cost of the team and the number of iterations needed by the team to create a version that meets the need from the customer.

This scenario is much more agile and will respond to change much better than the scenario with the plan. On the other hand, what about…

Selling a product

Let’s assume that there is a software system, and we call it a product. Each feature in the system flows through a value creating process including actions such as development, testing, and deployment.

The challange here, is to transform business needs into business values as efficent as possible. And the development process is a description of the current best way of doing that.

To do this efficiently, a motivated, complete team is created. With the ability to move items through the process. The team uses the concept of Scrum to work in iterations. The team tries to minimize the number of things in process the size of the items in the process so that the speed is maximized.

In the perfect world, features are delivered just-in-time, and development teams pulls work from the input queue. Or even better, it can be seen as a Kanban system… where the order is added last in the process… that’s where the motivations for test-driven development is really clear.

So what’s the point?

Maybe, I’ll really need to describe this in a better way. But for me this states the real difference between Scrum and Lean. Do you see the difference?

Scrum is the framework for helping the team collaborating, to make things flow through the development process.

Lean Software Development, is about creating value, make things flow in the value stream from business need to business value.

It is possible to use waterfall project methods to push things thorugh the value stream, but what you’ll see then is a number of large queues between for example development and test etc. Ie. waste in the lean world… so that is something that we don’t like.

SO, again, what’s the point, Agile methods and Scrum are useful practices to enable Lean Software Developent. For me, that complements the perspective of Lean being a foundation of principles guiding software development.

So, is Lean Software Development applicable when you are developing software in a non-product line environment? Yes, I’ll believe so but more on that in another post. What do you think?

cheers!

1. Extra features – I cannot see any direct correlations here
2. Partially done work – If code that is not refactored is not done, then skipping refactoring will be valid here. But I don’t think it is that obvious. Based on that argument, we can actually motivate allmost everyting by this kind of waste
3. Relearning – Does not help me that much here, or?
4. Task switching – Nope
5. Handoffs – Nope
6. Delay – Code that is not refactored will result in delay and will take longer to maintain and make changes to. So, Delay can be an effect, but is not the cause.
7. Defects – Not really, but as with delay, sloppy code will most likely result in more defects or increased cost.

In the book Lean Software Development – From concept to Cash by Tom and Mary Poppendieck, the topic complexity is one of the first topics discussed when waste is discussed. Technical debt will result in increased complexity. Adding complexity into the code is – what I can see – increasing the technical debt.

So, my current standpoint is that putting technical debt into the code base is waste. Removing technical debt is neccessary waste that should not be needed if the sloppy code was not there from the first place.

Then the next question is how we can develop code with minimal technical debt fast enough? Well, write less code, build quality into the software via Test-Driven-Development and constant refactoring might be a good start.

Do you have the guts to create slack to increase productivity?

Go lean !

What is true for a company standard? Is it:
1) These standards should be used throughout the whole company
2) We cannot have local customizations of that standard, because then it would not be a company standard
3) If someone finds a better way of working ie. improving the standard, that that change should be communicated throughout the whole company.

If these three statements are true; then I don’t think company standards are something to strive for, for a couple of reasons:

1) It feels like, this standard is the best way to do things, just follow the process and the work will be done..
2) We remove local creativity
3) We as a company get really slow, every improvment turns into a large change in way of working for a lot of people.
4) Instead of enable standards for improvement and efficiency, the standards are things that will prevent people for being productive and creative.

So, how should we think about standards then? I thik the concepts from Lean and Kaizen can guide us here:

Standards are there to describe our current best way of doing things. The teams can choose to follow the standards. The team is accountable and responsible for achieving the result of thier work, if they decide to not follow our current best way of doing things, it is their call.

Let’s assume that it is not the culture of the company to delegate responsibility to the teams. Then the people deciding what the team should do must have a tool to commuicate the directions for the team. Let us also assume that these directions are called company standards. Because these standards describe how we are doing things in this company…

If the team is not responsible for what they are doing, where is the motivation for improvements? I think respect for people and delegating responsibility as close to the work as possible is the key to continous improvements.

So, is a company wide standard a failure? Well, it depends on how the company thinks about the people in the enterprise.

Looking at the value stream and the principles of lean, the lean organization aims at reducing the cycle time from the need/request of feature until it is in production. Moving focus to features (and we only do the features that provide the highest value right?) we reduces a lot of waste partially done work, extra features etc etc.

In a larger enterprise, with many different processes and also many different sysetms, the need comes from the business in the process improvements rather than in syftem improvements. Well this is just plain old SOA, or?

Looking at SCRUM; you have the product owner and the product backlog. Where the product backlog is an prioritized list of the most valuable featuresets for that solution.

From a business perspective, moving from a system backlog to a process backlog, and the development company/partner/department/team is responsible for eliminating backlog items as effectively as possible. Software development can be a more natural part in business improvements instead of just delivering software.

For me, this aligns very well with what I consder to be Lean.

So my question are, are there any process backlogs available? Anyone having experiences of this? I

/ Johan Nordin