Category: Project Management

Homogeneous & heterogeneous queues – Why it is important in software engineering
Photo credit

From very early age, software engineering has been borrowing heavily from manufacturing industry. We as software professionals has learned a lot from manufacturing systems/concepts such as TPS. No doubt, this has helped a great extent to the success of software engineering profession.

But there is one fundamental issue with borrowing blindly from manufacturing. That is manufacturing queues and software development queues are fundamentally different.

Do we have queues in software engineering???

Queue in this context is the line up of work to be completed. In software engineering, our queues are  sprint or product backlog. In manufacturing, queues are the jobs to be performed in a production line.

Manufacturing systems

All (almost) the manufacturing systems are based on “homogeneous queues“. What that means is that each piece of work (job) produced is similar or homogeneous. For example consider a Toyota factory. There will be multiple production lines but one line can produce only one type of vehicle at a time. If the line required to produce another type of vehicle, a considerable downtime is required to for the change.

Software development systems

In software engineering, we can equate a “production line” to the software team and queue to the sprint/product backlog.

Software development teams are never get similar (or homogeneous) type of work.  The first user story (or task) is totally different from the second user story. Therefore the software development queues are “heterogeneous queues” as one task is different from another.

What does this mean? Why this difference is important?

This simple difference in queues makes the DIRECT application of some of manufacturing concepts such as six sigma to software engineering is fundamentally flawed. The concepts which work in homogeneous queues do not necessarily work in heterogeneous queues.

For example, six sigma relies on the variance to identify problems in the system. Variance can only be tracked when the work is repetitive. I.e. when somebody produces the same thing over and over in the same way. We never get this repetition in software development.

If Toyota produces the software in their factory;

Imagine a situation where Toyota has been asked to produce software in their production line.

Photo credit

Engineers will set up the production line for story 1 – Let’s say to develop “User creation” feature. We will have BA workstation specifying the requirements, to be passed to UX workstation to produce prototype, etc. Finally it goes to deployment slot after passing QA.

When the second story (This is a bug to fix in “Global search”) comes to first workstation, that workstation requires a total change of set up. This require downtime for change of set up.

With this amount of variability, most of the manufacturing concepts/tools will be  ineffective.


One needs to be careful and understand this fundamental difference before directly applying manufacturing concepts & tools to software development.

Many misses this important difference between the types of queues which results in pressurizing development teams to deliver software like cars coming out of Toyota production line.

Obviously if the software teams are asked to develop same task over and over again, there will not be any problems

A big cautionary note

A big cautionary note here is that this does not mean that there is no use of adopting concepts from manufacturing or every manufacturing concept is not valid. For example lean concepts such as “inventory as a waste”, “eliminating waste”, etc. are fundamentals and they are still valid fully in software engineering.

My argument here is that as software professionals, we should not blindly adopt ALL the manufacturing concepts in to software development.


Importance of controlled failures


We try to avoid failures as much as possible. But there are certain situations where failures can be beneficial.

For example take a situation where things are not going according to the best possible path but it it is beyond your control to take it back to the correct path. There are few alternatives in such situations.

  1. One option is to fight all out and try to get it back to the track.
  2. Second option is to let it go & brace for the disaster.
  3. The third option is to led the way towards a small & controlled failure.

The purpose of this discussion is to explore the third option. First let’s look at the first & second options. If you use the first option & fight with all guns blazing, it would require significant amount of energy and time. After all it may be that you are fighting a losing war.The second option can lead to a total disaster & the chance of revival can be slim.  

What is a “Controlled failure”?

If you look at the third option next, there are many advantages. My definition of controlled failure is that you let certain variables to fail. Those variables are carefully selected as well as limited.

For example consider a situation where a project team is pushed to deliver under unrealistic deadlines over and over. You are trying to explain to the decision makers about the risk but you hear back every-time “our customers demands it and without meeting those demands company will not survive“. There are many variables to this situation. Examples are “Code quality”, “Functional quality”, “Developer burnout”, “Penalty of missed deadlines”, etc.

So in a controlled failure you open only the cards (variables) you want to play. For example it can be that you select the “Code quality” variable and let a internal deadline to be missed. (Normal option would be to ask the developers to work in another weekend). This can lead to decision makers to realize the importance of adjusting unrealistic deadlines and start working on them. 

The importance

The important aspect of controlled failure is the word “Controlled“. The failure should be such that it does not lead to significant loss and should be possible to control the situation through non-failed variables.

Some further examples for situation where controlled failure may help

  1. The request to improve code quality being ignored for a period of time
  2. Request for full regression cycle being ignored
  3. Certain teams are not preparing for the sprint demo & review & come up with all the excuses
  4. In personal life example, kids are ignoring the instruction to go to bed early

The natural tendency of  a leader is to work with the situation and get it back to track. It takes some considerable energy and situations where there are multiple of such scenarios. It requires leader’s heroic effort & it is not a sustainable strategy.

So setting up a controlled fire can help the situation!

For example connecting to above situations

  1. Let the next change request to over-run (than working late to finish on time) due to brittle code
  2. Let the staging release (before the production release) to have bugs which are noticed by the customer
  3. Let an important demo to fail
  4. Let the kids to take their own time to go to bed but get them to wake up exactly to the time you want next day

Concluding thoughts

Failures are good on long run if you learn the lesson out of it. A controlled failure will help to learn the lesson and “Control failures” are an important tool in management.

Learning organization & agile software development

Peter Senge

Unfortunately most of the organizations do suffer from many learning disabilities. According to Peter Senge, one good example is “enemy is out there“. What it means is that when things fails, organizations find outside thing(s) to blame. For example when sales are lower, companies blame it to the current economic conditions (outside factor) over inefficient selling process (internal factor). Making the sales process efficient could have still yielded more results at the given economic situation. The enemy out there factor limits opportunity to correct the selling process. In generic terms these learning disabilities limits the “opportunity to learn“.

We need to overcome these learning disabilities to truly find the organization that learn!

The concept of “Learning organization” is an emerging concept and it will take time to mature. But the potential of what it offers is promising to explore further. Learning organization is just a term and there is not much value in looking at the definition. What is interesting is to explore the five discipline of a learning organization. Therefore a good definition for the learning organization is an organization that recognizes and practices the following five disciplines.

Note: The discipline here means an area of practice. For example psychology is a discipline so as quantum physics.

Disciplines of a learning organization

  1. Systems thinking: Ability to see the whole
  2. Personal mastery: The process of life long learning
  3. Shared vision: A vision that shared & committed by everybody
  4. Mental model: Generalized & assumed view of how the world works
  5. Team learning: The process of developing  the ability to create the desired results as a team

What is in it for us?

What it store for us is the simple fact that these disciplines touches the heart of agility. Agile software development is based on empiricism and therefore does not offer a consistent framework for practice. The learning organization disciplines provides a good foundation to truly practice agile software development. In other words agile movement guides us on what we need to do and learning organizations disciplines provides ways to how to practice. 

Let’s explore how it works;

Systems thinking

What systems thinking tells us is to take a holistic approach rather than a component or event oriented view. The second important fact is that we are just part of the system & components in the system regularly exchange feedback. System archetypes provides a structure to understand the systems in operation and how to take the best view. In summary it is the system structure that causes its behavior. A good read on systems thinking can be found here.

If you think further on this line, you will realize that whole Agile movement is all about taking the systems thinking view.

In prior agile era, we had many phase gates and we had different teams such as business analysis, designers/architects, developers, etc. to perform a single operation of software development. At that time the whole software engineering was a disintegrated & event driven operation.

What agile movement brought in to software engineering was the systems thinking. No longer we have separate events like specification sign-offs, etc. Now we have same team performing different components in software engineering. Today we optimize & find solutions to the whole. For example we invite business people for the retrospectives and to demos. We optimize the feedback mechanisms and increases the frequency of feedback to single iteration. So the agile movement changed the whole structure of the software engineering thus creating a total different behavior. Simply we as agilists ride on top the systems thinking.

So now the question is: Are we done with systems thinking by inventing agile? Not really. Systems thinking discipline can take us much further than what we are today due to many reasons.

First reason is that today many organizations applies agile methods simply as a process and does not understand what agile thinking brings to the table. In this case the discipline of systems thinking can bring them back to some light.

Second reason is that use of systems archetypes can make the organizations to better understand how software development system works and to find the optimal way to operate. Remember every organization is different.

There are many other concepts inside systems thinking discipline such as point of high leverage, reinforcing/balancing feedback, etc. which can help an organization to find the true agile organization or take it to the next level.

Personal mastery

Personal mastery is the process of life long learning to achieve results we truly desire in our life. It is about being master of the what we do. The key to personal mastery is to find the a concept called “Creative tension” which will drive us forward. A further reading on personal mastery:

Agile software teams are self managed and consisted of motivated individuals who seeks excellence in whatever they do. Ceremonies such as retrospective look in to what learning which team take forward. Targets such as sprints goals clarifies teams of what they need to achieve. To take out few lines from Agile manifesto principles;

“Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done”

“Continuous attention to technical excellence and good design enhances agility”

The discipline of personal mastery provides a framework to team members to improve their skills which requires in Agile software development.

Shared vision

Shared vision is a vision which the whole organization is committed. Shared vision touches heart of the organization and everybody in the organization shares the same vision.

The vision that is shared is by whole organization aligns it towards the excellence. When you look at the agile manifesto, aligning to business vision is the core focus of agile software development. Again to take few lines from agile manifesto

“Responding to change over following a plan”

“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

“Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.”

Software development teams need to share the vision in order to see the value of welcoming changes even in late in development. When teams share the same vision as business, the highest priority would be to satisfy customer demands would happen automatically.

So the discipline of shared vision provides a framework to help software engineering teams to create, commit & share the organizational shared vision in the true agile organization.

Mental model

Software development is all about making abstractions about real world. Take an example of an order processing software. Order processing happens physically in a factory and software development teams needs to model it successfully to create a valuable piece of software. Object Oriented Programming concepts helped a great deal for software engineering to come to the place where we are today.

In definition Mental models are the abstractions which we make about real life situations. Mental models exists in minds of people and they exists below the level of conscious so people doesn’t realize the existence of mental model. Human forms mental models for everything. For example we build an abstract model in our mind regarding an upcoming version of Windows or year end Agile conference. We form these models based on limited information we tend to hear or find from somebody or somewhere. There is no guarantee that our mental are right and mental models are never accurate. What is important to be aware of mental models and test them.

Extending above example of order processing, let’s look how the mental models works. When the teams looks for requirements, they talk to people who does the order processing. Order processing people describe the process using abstractions. I.e. they describe their mental model of order processing. Then software developers creates another mental model inside their mind.

What agile movement did was to align with the formation of mental models and make them visible as much as possible. For example we use user stories over lengthy documentations so we don’t fix a wrong mental models. (Note: According Mike Cohn the abstraction of user stories provoke the discussion – which is a very good way to test the mental models)

We use personas to model the real users and we use “As a … I want to … So that …” format to model the real user scenarios & importantly understand the story behind the story. Agile architecture is about modeling the software based on what we know and keep opportunity to improve it as we go rather than fixing a wrong mental model. Concepts like sprint planning, sprint reviews, fast feedback, etc. provides opportunities to explore the mental models further. In waterfall we fix the model and it will be very hard to change.

Although agile movement touches the concept of mental models but the discipline of mental models offer far fetched benefits than what we use today. In my view we can use the discipline of mental model to create a new way of working to review & test the mental models which we form over user needs or requirements.

Team learning

Team learning is about learning as a team to create the results. Set of individual heroes does not always produce results unless they make a good team. The important fact to create a good team is that “team is able to learn as a team not only as individuals”.

Agile software development is based on teams not on individuals. We measure team velocity not individual velocity. We perform team retrospectives not individual performance reviews.

Agile manifesto specifies the need of the team but does not specify how to learn as a team (rightly so). There are many books written & many literature about how to create high-performing teams. But the discipline of team learning provides a different perspective based on systems thinking to practice how to work best as a team.


Agile organization is the one we all are dreaming to work for. Learning organization is what it helps to create the true agile organization.

As noted earlier, agile movement provides guidelines regarding what to achieve & learning organization disciplines provides how to do that.

And the true agile organization is just one benefit of the learning organization.

Further reading

Managing queue length

Queue-2012-12-11Optimizing the software engineering is no easy task. The intangibility of software assets in combination that most of the engineering happens at the human brain makes it difficult to see the whole picture. Optimization requires visibility. This is where the lean manufacturing provides a good foundation to optimize and improve the software engineering as a whole. The pioneers like Mary Poppendieck & many others have already paved the way.

Taking a leaf out of the lean engineering book, one of the important concepts that comes along with SAFe is “Managing the queue length”. The longer the queue, higher the waiting. In lean manufacturing, waiting is  one of the seven wastes. Therefore managing queue length is important.

What is a queue in software development?

At the first glance, we don’t see any queues in software. But if you make a deeper look in to the definition of the queue, you will see many long queues. According to merriam-webster queue is a “line of people who are waiting for something”.

This can be easily mapped to “List of features which are waiting to be developed”. Going further, product backlog is the list which contains the features (user stories). Does this means the product backlog is a queue?

Is the product backlog a queue?

The product backlog interestingly is not a queue. The justification lies on the fact that queues cannot be easily re-arranged and typically works on FIFO basis. If you try to change the queue at the bus or train station, you will not have happy results.

In this meaning, you can change the order (prioritization) of the backlog so it is not a queue.

Waterfall is primarily about long & costly queues

Everything is planned and scheduled at the beginning in a waterfall project. This is essentially a long queue & it defines who will work on what even at the last week. The problem with the queues are multi-fold. It creates long waiting time for new features. The most dangerous fact is changing the queue (schedule in this case) is very costly.

For example if there going to be a change to the requirement, the whole schedule needs to be adjusted. If there are external dependencies such as delivering a component to another team, etc. then it requires lot of effort to change the queue (schedule).

This is the main reason why changes are primarily considered bad in waterfall projects, no matter what benefits they brings to the business. There are lengthy & costly procedures to pass a change in to the development.

Managing queue length

Any experienced Agile practitioner would tell you that they only plan next two iterations. This is essentially keeping the queue length short. Only the current sprint is committed (SAFe even reduces this commitment by introducing a concept called stretch goals) and it is only the queue we have in agile software development in a way.

All the other stories in the backlog are not planned & kept untouched. This comes from the fourth value of the agile manifesto “Responding to change over following a plan” as well as from the second principle of the manifesto “Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

Why we should have shorter queues?

Benefits of the shorter queues are many.

  1. The main benefit is that it facilitate the agility by facilitating changes even late in to development.Also changing is less costly.
  2. Secondly it makes the waiting time predictable. If there is an urgent change to be introduced, everybody knows it is maximum of one iteration which they have wait. 
  3. Ultimately it supports the business by allowing early realization of business value by reducing the waiting time


In software development, queues are the stories or features scheduled to be taken in to development.

In conclusion, the queues are bad due to cost of change is significant but change is essential. Agile software development by design limits the queue length and therefore supporting businesses to realize benefits quickly.

The takeaway is to limit the queue size by only planning current sprint and next sprint.


What is waterfall?


Recently I have been involved in a discussion what is waterfall (of course in software development perspective).

The very first thing comes to the mind is that waterfall software development is a phased approach (over iterative Agile). You have got defined phases and boundaries to cross like “Requirement phase, Design phase, etc.”

If you stop for a moment and look in detail, you will realize that this is more of a description of the “Execution model”. Or how the Waterfall model is executed. But there is much more deeper conceptual definition if you look at from a business perspective.

When you look at software engineering from the problem vs solution concept, you can see it. Software is a solution for a business problem. And software engineering is all about articulating the solution.


So this is how the Waterfall approaches;

  • You have the “Business problem” and
  • You have a layer to “Specify the problem” (requirement),
  • Another layer to “Specify the solution” (architecture and design),
  • Another layer to “Develop the solution” (development),
  • Another layer to “Confirm the solution” (testing)
  • Final layer to “Apply the solution” to the problem (Deployment).

Through this you can transform the horizontal phases of waterfall to vertical layers and by doing so you can see how the waterfall development is really structured.

I don’t need to mention the waste and the confusion this causes when you have so many layers between the Problem and the Solution.

But Agile software development provides a much more compact approach which is more effective than Waterfall. In Agile, you just have two layers representing the Problem and the Solution. Problem is represented by Product owners (if you consider Scrum) and Solution is represented by Development team. By reducing the unwanted layers in the middle, Agile has reduced the waste and the confusion.

But as a note this doesn’t mean that you can avoid the needed steps like design, testing, deployment, etc. It is just that all are done within one layer


Today’s Quality and Tomorrow’s Quality

Software Quality has been a much discussed topic in software community. Quality has been categorized in to “Quality Assurance (QA)” and “Quality Control (QC)”. In the following article, Jim Highsmith, provide a totally different categorization.  (Jim Highsmith is one of founder signatories of Agile Manifesto) I found it to be a really interesting way of looking at the Quality.

The beauty of this categorization is that  it combines the Quality Assurance and Quality Control remarkably well within the single scope of Quality. This gives a very easy definition for the Quality without needing to go through two separate areas to understand the Quality in its entirety.

“Today’s Quality and Tomorrow’s Quality” – 

In simple terms, Today’s Quality ensure that the product is usable today. In other terms, the software does not contain any critical bugs, or it is built to suit the users’ needs, etc. By looking from the traditional Quality Assurance and Quality Control view, achieving Today’s Quality would need both QA and QC activities to be performed.

Whereas “Tomorrow’s Quality” refers to ensuring that software will perform in the future as today. This means ensuring code readability, technical documentation, following accepted design patterns, keeping technical debt low, etc. This also would need both QA and QC activities to be performed.

This is very strong but simple definition of Quality

Do you follow Agile?

Do you follow Agile? How sure are you?

Well, this is not a new question at all. This was there for a long time, may be from the very beginning of Agile. Over the period this was discussed again and again. Even there was some terms created out of this. Take a look at terms like “Scrum BUT”, “Mini Waterfall”, etc. These words are there in our vocabulary because of people are saying they follow Agile but not doing so. That is bad, isn’t it?

I don’t want here to write another post here on this traditional question. Instead what I want to do here is to explore whether there is some simple tool which I can used to get a quantifiable answer. It should be something which can be used by anybody who is having some basic idea about Agile.

If you Google, you will find any resources where you can use to measure yourself. Some of them are listed below.

The above are very useful resources and I don’t have any question on that. But they did not carry the simplicity and transparency which I am looking for. As I said, I want something simple and it should be very transparent. When talking about transparency and if you take one of those questionnaires from above, I don’t have a clue why those questions were picked in the questionnaire, what is the basis for them. I wanted a solution which is much more Crystal than those.

When I was thinking further, I ended up going back to the basics. I mean to very basic. If you have a method or a tool which compare what you are doing against the Agile Manifesto, then you can’t be wrong. Yes this is it. If you follow the Agile Manifesto, then you gotta be doing Agile.

When I go to manifesto and look at that, there still was a hurdle to cross. There are two parts of the Manifesto (Agile Values and Principles). What do you use in your small evaluation tool? Is it Values, is it Principles or is it both? I like to use the Principles only.

Why? There are few reasons

  1. When you go to and go to Principles, you will see that the topic is “Principles behind the Agile Manifesto”. So these Principles are behind everything. I mean everything when it comes to Agile J Therefore there is no better thing to use than the Agile Principles in my Agile evaluation.
  2. Since the Agile Values are derived out of the Principles, the four Agile Values are at a higherlevel compared to something you want to use in an evaluation. For example “Individual and Interactions” would mean many different meanings which can lead to many ambiguous interpretations when used in the evaluation. This can deteriorate the accuracy of the results.
  3. Due to number 2 reason above, we should not use both Values and Principles to evaluate. Further using both would complicate the evaluation process which is not what I want

So what is my proposal?

There gotta be few simple steps

  1. Appoint a coordinator
  2. Get all 12 Agile Principles in a spreadsheet such as MS Excel
  3. In front of each Principle, get the team to indicate whether they are following the Principle correctly or not. It is important that the team understand each of these principles clearly.
  4. I suggest each member in the team has their independent evaluation sheet and then consolidate them at the end. If you feel that this would complicate the process, then get the whole team in and go through each Principle and agree on a result
  5. This should be it.

On conclusion, I have use this way to evaluate few projects and I found it not that easy to do so. But in my view there is no other better way to check whether you are doing Agile or not than evaluating against the Agile Principles.

Finally, I found the following article which has the same thinking.