Category: Agile

Learning organization & agile software development

Peter Senge

Unfortunately most of the organizations do suffer from many learning disabilities. According to Peter Senge, one good example is “enemy is out there“. What it means is that when things fails, organizations find outside thing(s) to blame. For example when sales are lower, companies blame it to the current economic conditions (outside factor) over inefficient selling process (internal factor). Making the sales process efficient could have still yielded more results at the given economic situation. The enemy out there factor limits opportunity to correct the selling process. In generic terms these learning disabilities limits the “opportunity to learn“.

We need to overcome these learning disabilities to truly find the organization that learn!

The concept of “Learning organization” is an emerging concept and it will take time to mature. But the potential of what it offers is promising to explore further. Learning organization is just a term and there is not much value in looking at the definition. What is interesting is to explore the five discipline of a learning organization. Therefore a good definition for the learning organization is an organization that recognizes and practices the following five disciplines.

Note: The discipline here means an area of practice. For example psychology is a discipline so as quantum physics.

Disciplines of a learning organization

  1. Systems thinking: Ability to see the whole
  2. Personal mastery: The process of life long learning
  3. Shared vision: A vision that shared & committed by everybody
  4. Mental model: Generalized & assumed view of how the world works
  5. Team learning: The process of developing  the ability to create the desired results as a team

What is in it for us?

What it store for us is the simple fact that these disciplines touches the heart of agility. Agile software development is based on empiricism and therefore does not offer a consistent framework for practice. The learning organization disciplines provides a good foundation to truly practice agile software development. In other words agile movement guides us on what we need to do and learning organizations disciplines provides ways to how to practice. 

Let’s explore how it works;

Systems thinking

What systems thinking tells us is to take a holistic approach rather than a component or event oriented view. The second important fact is that we are just part of the system & components in the system regularly exchange feedback. System archetypes provides a structure to understand the systems in operation and how to take the best view. In summary it is the system structure that causes its behavior. A good read on systems thinking can be found here. http://www.thwink.org/sustain/glossary/SystemsThinking.htm

If you think further on this line, you will realize that whole Agile movement is all about taking the systems thinking view.

In prior agile era, we had many phase gates and we had different teams such as business analysis, designers/architects, developers, etc. to perform a single operation of software development. At that time the whole software engineering was a disintegrated & event driven operation.

What agile movement brought in to software engineering was the systems thinking. No longer we have separate events like specification sign-offs, etc. Now we have same team performing different components in software engineering. Today we optimize & find solutions to the whole. For example we invite business people for the retrospectives and to demos. We optimize the feedback mechanisms and increases the frequency of feedback to single iteration. So the agile movement changed the whole structure of the software engineering thus creating a total different behavior. Simply we as agilists ride on top the systems thinking.

So now the question is: Are we done with systems thinking by inventing agile? Not really. Systems thinking discipline can take us much further than what we are today due to many reasons.

First reason is that today many organizations applies agile methods simply as a process and does not understand what agile thinking brings to the table. In this case the discipline of systems thinking can bring them back to some light.

Second reason is that use of systems archetypes can make the organizations to better understand how software development system works and to find the optimal way to operate. Remember every organization is different.

There are many other concepts inside systems thinking discipline such as point of high leverage, reinforcing/balancing feedback, etc. which can help an organization to find the true agile organization or take it to the next level.

Personal mastery

Personal mastery is the process of life long learning to achieve results we truly desire in our life. It is about being master of the what we do. The key to personal mastery is to find the a concept called “Creative tension” which will drive us forward. A further reading on personal mastery: https://www.solonline.org/?page=PersonalMastery

Agile software teams are self managed and consisted of motivated individuals who seeks excellence in whatever they do. Ceremonies such as retrospective look in to what learning which team take forward. Targets such as sprints goals clarifies teams of what they need to achieve. To take out few lines from Agile manifesto principles;

“Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done”

“Continuous attention to technical excellence and good design enhances agility”

The discipline of personal mastery provides a framework to team members to improve their skills which requires in Agile software development.

Shared vision

Shared vision is a vision which the whole organization is committed. Shared vision touches heart of the organization and everybody in the organization shares the same vision.

The vision that is shared is by whole organization aligns it towards the excellence. When you look at the agile manifesto, aligning to business vision is the core focus of agile software development. Again to take few lines from agile manifesto

“Responding to change over following a plan”

“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

“Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.”

Software development teams need to share the vision in order to see the value of welcoming changes even in late in development. When teams share the same vision as business, the highest priority would be to satisfy customer demands would happen automatically.

So the discipline of shared vision provides a framework to help software engineering teams to create, commit & share the organizational shared vision in the true agile organization.

Mental model

Software development is all about making abstractions about real world. Take an example of an order processing software. Order processing happens physically in a factory and software development teams needs to model it successfully to create a valuable piece of software. Object Oriented Programming concepts helped a great deal for software engineering to come to the place where we are today.

In definition Mental models are the abstractions which we make about real life situations. Mental models exists in minds of people and they exists below the level of conscious so people doesn’t realize the existence of mental model. Human forms mental models for everything. For example we build an abstract model in our mind regarding an upcoming version of Windows or year end Agile conference. We form these models based on limited information we tend to hear or find from somebody or somewhere. There is no guarantee that our mental are right and mental models are never accurate. What is important to be aware of mental models and test them.

Extending above example of order processing, let’s look how the mental models works. When the teams looks for requirements, they talk to people who does the order processing. Order processing people describe the process using abstractions. I.e. they describe their mental model of order processing. Then software developers creates another mental model inside their mind.

What agile movement did was to align with the formation of mental models and make them visible as much as possible. For example we use user stories over lengthy documentations so we don’t fix a wrong mental models. (Note: According Mike Cohn the abstraction of user stories provoke the discussion – which is a very good way to test the mental models)

We use personas to model the real users and we use “As a … I want to … So that …” format to model the real user scenarios & importantly understand the story behind the story. Agile architecture is about modeling the software based on what we know and keep opportunity to improve it as we go rather than fixing a wrong mental model. Concepts like sprint planning, sprint reviews, fast feedback, etc. provides opportunities to explore the mental models further. In waterfall we fix the model and it will be very hard to change.

Although agile movement touches the concept of mental models but the discipline of mental models offer far fetched benefits than what we use today. In my view we can use the discipline of mental model to create a new way of working to review & test the mental models which we form over user needs or requirements.

Team learning

Team learning is about learning as a team to create the results. Set of individual heroes does not always produce results unless they make a good team. The important fact to create a good team is that “team is able to learn as a team not only as individuals”.

Agile software development is based on teams not on individuals. We measure team velocity not individual velocity. We perform team retrospectives not individual performance reviews.

Agile manifesto specifies the need of the team but does not specify how to learn as a team (rightly so). There are many books written & many literature about how to create high-performing teams. But the discipline of team learning provides a different perspective based on systems thinking to practice how to work best as a team.

Conclusion

Agile organization is the one we all are dreaming to work for. Learning organization is what it helps to create the true agile organization.

As noted earlier, agile movement provides guidelines regarding what to achieve & learning organization disciplines provides how to do that.

And the true agile organization is just one benefit of the learning organization.

Further reading

http://newtechusa.net/agile/agile-gateway-drug-to-the-learning-organization

http://www.amazon.com/The-Fifth-Discipline-Practice-Organization/dp/0553456342

Advertisements

Managing queue length

Queue-2012-12-11Optimizing the software engineering is no easy task. The intangibility of software assets in combination that most of the engineering happens at the human brain makes it difficult to see the whole picture. Optimization requires visibility. This is where the lean manufacturing provides a good foundation to optimize and improve the software engineering as a whole. The pioneers like Mary Poppendieck & many others have already paved the way.

Taking a leaf out of the lean engineering book, one of the important concepts that comes along with SAFe is “Managing the queue length”. The longer the queue, higher the waiting. In lean manufacturing, waiting is  one of the seven wastes. Therefore managing queue length is important.

What is a queue in software development?

At the first glance, we don’t see any queues in software. But if you make a deeper look in to the definition of the queue, you will see many long queues. According to merriam-webster queue is a “line of people who are waiting for something”.

This can be easily mapped to “List of features which are waiting to be developed”. Going further, product backlog is the list which contains the features (user stories). Does this means the product backlog is a queue?

Is the product backlog a queue?

The product backlog interestingly is not a queue. The justification lies on the fact that queues cannot be easily re-arranged and typically works on FIFO basis. If you try to change the queue at the bus or train station, you will not have happy results.

In this meaning, you can change the order (prioritization) of the backlog so it is not a queue.

Waterfall is primarily about long & costly queues

Everything is planned and scheduled at the beginning in a waterfall project. This is essentially a long queue & it defines who will work on what even at the last week. The problem with the queues are multi-fold. It creates long waiting time for new features. The most dangerous fact is changing the queue (schedule in this case) is very costly.

For example if there going to be a change to the requirement, the whole schedule needs to be adjusted. If there are external dependencies such as delivering a component to another team, etc. then it requires lot of effort to change the queue (schedule).

This is the main reason why changes are primarily considered bad in waterfall projects, no matter what benefits they brings to the business. There are lengthy & costly procedures to pass a change in to the development.

Managing queue length

Any experienced Agile practitioner would tell you that they only plan next two iterations. This is essentially keeping the queue length short. Only the current sprint is committed (SAFe even reduces this commitment by introducing a concept called stretch goals) and it is only the queue we have in agile software development in a way.

All the other stories in the backlog are not planned & kept untouched. This comes from the fourth value of the agile manifesto “Responding to change over following a plan” as well as from the second principle of the manifesto “Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

Why we should have shorter queues?

Benefits of the shorter queues are many.

  1. The main benefit is that it facilitate the agility by facilitating changes even late in to development.Also changing is less costly.
  2. Secondly it makes the waiting time predictable. If there is an urgent change to be introduced, everybody knows it is maximum of one iteration which they have wait. 
  3. Ultimately it supports the business by allowing early realization of business value by reducing the waiting time

Summary

In software development, queues are the stories or features scheduled to be taken in to development.

In conclusion, the queues are bad due to cost of change is significant but change is essential. Agile software development by design limits the queue length and therefore supporting businesses to realize benefits quickly.

The takeaway is to limit the queue size by only planning current sprint and next sprint.

Photo: http://blog.hqc.sk.ca/wp-content/uploads/2012/12/Queue-2012-12-11.jpg

Iterative thinking

768px-Screwtop_spiralTHE “WATERFALL” VS “AGILE” BATTLE SEEMS TO BE OVER BY NOW AND AGILE HAS EMERGED AS THE WINNER.  Still the current software development landscape is far from the order. It is filled with many tools, processes, frameworks, methods, methodologies, best practices, procedures (Be my guest …)

This drives many software development teams in to confusion. Outcome of this confusion is that teams have been following processes, methods, etc. just as a recipe. It can be scrum, XP, Lean Start up or even SAFe but you will see many teams use them without understanding the underlying thinking. Probably this is one reason why the (almost) movement “Be agile” had to be born.

It is obvious that teams should not use “Agile recipes“. There is no better reason than the first value of Agile manifesto to show why any team should not use a recipe . The first value states ” INDIVIDUALS & INTERACTIONS over processes and tools”. What has most value is the team itself and the interactions around it, not the use to tools. Teams should understand the basic underlying thinking of these Agile methods in order to be successful.

So the first question is what is the underlying thinking and the second question is whether is there any similarity of this underlying thinking? There is very simple answer to both the questions. All these Agile methods are based on a simple concept of  “Iterative thinking” (at least what it should be). Applying any Agile method just as a tool will not guarantee the success of the project but if one can find the success if he/she understands and applies iterative thinking. In my view, there is no need to use ANY tool or method if one can master and use iterative thinking in a project.

So the big question is what is “Iterative thinking”?

Iterations
Iterative delivery

 

Iterative thinking is connected to all the areas of software development. Iterative planning, Iterative design, Iterative development, etc. are examples. Iterative thinking can be simplified in to “it is all about slicing the customer need and deliver the solution in a progressive piece meal manner“. One important factor here is that each delivery should ensure that it is build on top of previous delivery and the future deliveries can be build on top of it.

The outcome of the “Iterative thinking” is the “Iterative delivery” and it is an extremely important concept when it comes to complex domains, scopes, etc for the success. When it comes simple software problems, the iterative thinking becomes less visible and just using a recipe can lead to success.

The “Iterative thinking” can be illustrated in an example. Let’s take typical user management feature:

steering-iterative-and-incremental-delivery-with-jeff-patton-13-728

  • The first iterative delivery can be a simple screen where users can be added to the system and the added users are listed in a crud list. There are no validations for the user creation form. Not much thought given for the design of the user management feature. The main objective is to get the initial thinking in sync between the development team and the problem owners. This delivery can be called as “User management Spike”.
  • The next delivery can be “Basic user management” where more features are beefed up on top the the user management spike. It can be that user creation form has more fields and validations are added, passwords are encrypted, etc. Also the basic design of the the user management has emerged.
  • This can be extended to further iterations till the problem owner is satisfied with the given solution.

(This should not be confused with incremental delivery. See this http://goo.gl/MSWRMI)

 

When it comes to complex software problems (such as automated vehicle guidance systems, etc.) there will be many more iterations than a simple user management feature.

  • The first iterations can be that nobody touches the code and it is all about presenting (by PPT) team’s understanding on the requirements.
  • The next iterations can be further strengthen on design aspects of the system (if team feels a certain up-front design is required).
  • These iterations can be further worked on through providing basic integration tests.
  • This can continue from the first delivery to final user acceptance.

The magic behind the “Iterative thinking/delivery is rapid & constructive feedback. When a team follows the iterative delivery, the feedback would be specific (such as saying “I don’t need the “middle name” field here, add a new field called “Date of birth”, move the “address” to the bottom, show the last login date/time, etc.) This specific feedback would speed up the team thus resulting rapid value delivery which fits to the purpose.

So in conclusion what can be noted is that the important fact in Agile software development is not to use the tools, processes, frameworks, methods, methodologies, best practices, procedures, etc. as recipes but to understand the concept of Iterative thinking. The simple use of Iterative thinking can lead the successful project delivery even ignoring all the norms and prescribed methodologies such as Scrum.

References:

http://www.mountaingoatsoftware.com/blog/the-one-true-way-to-be-agile

http://en.wikipedia.org/wiki/Iterative_design

What is waterfall?

thinking_man_by_the_waterfall_by_nayr90-d38nj4o

Recently I have been involved in a discussion what is waterfall (of course in software development perspective).

The very first thing comes to the mind is that waterfall software development is a phased approach (over iterative Agile). You have got defined phases and boundaries to cross like “Requirement phase, Design phase, etc.”

If you stop for a moment and look in detail, you will realize that this is more of a description of the “Execution model”. Or how the Waterfall model is executed. But there is much more deeper conceptual definition if you look at from a business perspective.

When you look at software engineering from the problem vs solution concept, you can see it. Software is a solution for a business problem. And software engineering is all about articulating the solution.

waterfall

So this is how the Waterfall approaches;

  • You have the “Business problem” and
  • You have a layer to “Specify the problem” (requirement),
  • Another layer to “Specify the solution” (architecture and design),
  • Another layer to “Develop the solution” (development),
  • Another layer to “Confirm the solution” (testing)
  • Final layer to “Apply the solution” to the problem (Deployment).

Through this you can transform the horizontal phases of waterfall to vertical layers and by doing so you can see how the waterfall development is really structured.

I don’t need to mention the waste and the confusion this causes when you have so many layers between the Problem and the Solution.

But Agile software development provides a much more compact approach which is more effective than Waterfall. In Agile, you just have two layers representing the Problem and the Solution. Problem is represented by Product owners (if you consider Scrum) and Solution is represented by Development team. By reducing the unwanted layers in the middle, Agile has reduced the waste and the confusion.

But as a note this doesn’t mean that you can avoid the needed steps like design, testing, deployment, etc. It is just that all are done within one layer

Images;

http://goo.gl/TDVuri

Relative estimates and bubble sort algorithm

While reading Mike Cohn’s article, I came across this comment http://goo.gl/KU6cA What a cool idea when it comes to relative estimation techniques.

If you are to get the benefits of relative estimates, you need to get the relative sizing right (which is a challenge itself). But most of the teams struggle on this. Let me give you an example to illustrate this.

To illustrate this difficulty;

  • Team has 20 stories to estimate
  • They go through the backlog and establish the base story and mark that as 2 story points
  • Then they start from the beginning of the backlog and start estimating the relative size
  • 1st story is easy and you compare effort for that against the base story and mark it as 5 story points.
  • When it comes to 2nd story, you now needs to compare it with 2 other stories (base story and 1st story)
  • This adds on when you go further down the backlog
  • What happens is that you loose the sense of stories in the top of the backlog when you come towards the bottom of the backlog. Instead of having one common base story you tend to have localized bases since as humans we don’t have much RAM

There are options to avoid this (such as Triangulation, etc.) but what bubble sort algorithm offers is simple and effective than those.

How you can use is to first go through the backlog and give it a bubble sort. You don’t need to worry about what is the relative size of the story but just compare whether the current story is bigger or smaller compared to next story in the list.

See further http://en.wikipedia.org/wiki/Bubble_sort

If you do this as the start, it will not have the problem which I mentioned above example. Further this will act as a good ice breaker for the estimation session.

Images are taken from http://goo.gl/Qrnfp and http://goo.gl/7uUlI

Unknown unknown

qpij9xffjouhq09qjsfs

Question 1: Do you know the problem? Yes. Do you know the solution? Yes. Then go and use waterfall.
Question 2: Do you know the problem? Yes. Do you know the solution? No. Then use Agile.

 

Question 3: Do you know the problem? No. Do you know the solution? No. Then use Learn startup.

 

Reference: David J Bland “Lean startups is not only 4 for start ups

Photo credit: http://goo.gl/EPb0O