Homogeneous & heterogeneous queues – Why it is important in software engineering

Photo credit

From very early age, software engineering has been borrowing heavily from manufacturing industry. We as software professionals has learned a lot from manufacturing systems/concepts such as TPS. No doubt, this has helped a great extent to the success of software engineering profession.

But there is one fundamental issue with borrowing blindly from manufacturing. That is manufacturing queues and software development queues are fundamentally different.

Do we have queues in software engineering???

Queue in this context is the line up of work to be completed. In software engineering, our queues are  sprint or product backlog. In manufacturing, queues are the jobs to be performed in a production line.

Manufacturing systems

All (almost) the manufacturing systems are based on “homogeneous queues“. What that means is that each piece of work (job) produced is similar or homogeneous. For example consider a Toyota factory. There will be multiple production lines but one line can produce only one type of vehicle at a time. If the line required to produce another type of vehicle, a considerable downtime is required to for the change.

Software development systems

In software engineering, we can equate a “production line” to the software team and queue to the sprint/product backlog.

Software development teams are never get similar (or homogeneous) type of work.  The first user story (or task) is totally different from the second user story. Therefore the software development queues are “heterogeneous queues” as one task is different from another.

What does this mean? Why this difference is important?

This simple difference in queues makes the DIRECT application of some of manufacturing concepts such as six sigma to software engineering is fundamentally flawed. The concepts which work in homogeneous queues do not necessarily work in heterogeneous queues.

For example, six sigma relies on the variance to identify problems in the system. Variance can only be tracked when the work is repetitive. I.e. when somebody produces the same thing over and over in the same way. We never get this repetition in software development.

If Toyota produces the software in their factory;

Imagine a situation where Toyota has been asked to produce software in their production line.

Photo credit

Engineers will set up the production line for story 1 – Let’s say to develop “User creation” feature. We will have BA workstation specifying the requirements, to be passed to UX workstation to produce prototype, etc. Finally it goes to deployment slot after passing QA.

When the second story (This is a bug to fix in “Global search”) comes to first workstation, that workstation requires a total change of set up. This require downtime for change of set up.

With this amount of variability, most of the manufacturing concepts/tools will be  ineffective.

Conclusion

One needs to be careful and understand this fundamental difference before directly applying manufacturing concepts & tools to software development.

Many misses this important difference between the types of queues which results in pressurizing development teams to deliver software like cars coming out of Toyota production line.

Obviously if the software teams are asked to develop same task over and over again, there will not be any problems

A big cautionary note

A big cautionary note here is that this does not mean that there is no use of adopting concepts from manufacturing or every manufacturing concept is not valid. For example lean concepts such as “inventory as a waste”, “eliminating waste”, etc. are fundamentals and they are still valid fully in software engineering.

My argument here is that as software professionals, we should not blindly adopt ALL the manufacturing concepts in to software development.

Importance of controlled failures

We try to avoid failures as much as possible. But there are certain situations where failures can be beneficial.

For example take a situation where things are not going according to the best possible path but it it is beyond your control to take it back to the correct path. There are few alternatives in such situations.

1. One option is to fight all out and try to get it back to the track.
2. Second option is to let it go & brace for the disaster.
3. The third option is to led the way towards a small & controlled failure.

The purpose of this discussion is to explore the third option. First let’s look at the first & second options. If you use the first option & fight with all guns blazing, it would require significant amount of energy and time. After all it may be that you are fighting a losing war.The second option can lead to a total disaster & the chance of revival can be slim.  

What is a “Controlled failure”?

If you look at the third option next, there are many advantages. My definition of controlled failure is that you let certain variables to fail. Those variables are carefully selected as well as limited.

For example consider a situation where a project team is pushed to deliver under unrealistic deadlines over and over. You are trying to explain to the decision makers about the risk but you hear back every-time “our customers demands it and without meeting those demands company will not survive“. There are many variables to this situation. Examples are “Code quality”, “Functional quality”, “Developer burnout”, “Penalty of missed deadlines”, etc.

So in a controlled failure you open only the cards (variables) you want to play. For example it can be that you select the “Code quality” variable and let a internal deadline to be missed. (Normal option would be to ask the developers to work in another weekend). This can lead to decision makers to realize the importance of adjusting unrealistic deadlines and start working on them. 

The importance

The important aspect of controlled failure is the word “Controlled“. The failure should be such that it does not lead to significant loss and should be possible to control the situation through non-failed variables.

Some further examples for situation where controlled failure may help

1. The request to improve code quality being ignored for a period of time
2. Request for full regression cycle being ignored
3. Certain teams are not preparing for the sprint demo & review & come up with all the excuses
4. In personal life example, kids are ignoring the instruction to go to bed early

The natural tendency of  a leader is to work with the situation and get it back to track. It takes some considerable energy and situations where there are multiple of such scenarios. It requires leader’s heroic effort & it is not a sustainable strategy.

So setting up a controlled fire can help the situation!

For example connecting to above situations

1. Let the next change request to over-run (than working late to finish on time) due to brittle code
2. Let the staging release (before the production release) to have bugs which are noticed by the customer
3. Let an important demo to fail
4. Let the kids to take their own time to go to bed but get them to wake up exactly to the time you want next day

Concluding thoughts

Failures are good on long run if you learn the lesson out of it. A controlled failure will help to learn the lesson and “Control failures” are an important tool in management.

Learning organization & agile software development

Unfortunately most of the organizations do suffer from many learning disabilities. According to Peter Senge, one good example is “enemy is out there“. What it means is that when things fails, organizations find outside thing(s) to blame. For example when sales are lower, companies blame it to the current economic conditions (outside factor) over inefficient selling process (internal factor). Making the sales process efficient could have still yielded more results at the given economic situation. The enemy out there factor limits opportunity to correct the selling process. In generic terms these learning disabilities limits the “opportunity to learn“.

We need to overcome these learning disabilities to truly find the organization that learn!

The concept of “Learning organization” is an emerging concept and it will take time to mature. But the potential of what it offers is promising to explore further. Learning organization is just a term and there is not much value in looking at the definition. What is interesting is to explore the five discipline of a learning organization. Therefore a good definition for the learning organization is an organization that recognizes and practices the following five disciplines.

Note: The discipline here means an area of practice. For example psychology is a discipline so as quantum physics.

Disciplines of a learning organization

1. Systems thinking: Ability to see the whole
2. Personal mastery: The process of life long learning
3. Shared vision: A vision that shared & committed by everybody
4. Mental model: Generalized & assumed view of how the world works
5. Team learning: The process of developing  the ability to create the desired results as a team

What is in it for us?

What it store for us is the simple fact that these disciplines touches the heart of agility. Agile software development is based on empiricism and therefore does not offer a consistent framework for practice. The learning organization disciplines provides a good foundation to truly practice agile software development. In other words agile movement guides us on what we need to do and learning organizations disciplines provides ways to how to practice. 

Let’s explore how it works;

Systems thinking

What systems thinking tells us is to take a holistic approach rather than a component or event oriented view. The second important fact is that we are just part of the system & components in the system regularly exchange feedback. System archetypes provides a structure to understand the systems in operation and how to take the best view. In summary it is the system structure that causes its behavior. A good read on systems thinking can be found here. http://www.thwink.org/sustain/glossary/SystemsThinking.htm

If you think further on this line, you will realize that whole Agile movement is all about taking the systems thinking view.

In prior agile era, we had many phase gates and we had different teams such as business analysis, designers/architects, developers, etc. to perform a single operation of software development. At that time the whole software engineering was a disintegrated & event driven operation.

What agile movement brought in to software engineering was the systems thinking. No longer we have separate events like specification sign-offs, etc. Now we have same team performing different components in software engineering. Today we optimize & find solutions to the whole. For example we invite business people for the retrospectives and to demos. We optimize the feedback mechanisms and increases the frequency of feedback to single iteration. So the agile movement changed the whole structure of the software engineering thus creating a total different behavior. Simply we as agilists ride on top the systems thinking.

So now the question is: Are we done with systems thinking by inventing agile? Not really. Systems thinking discipline can take us much further than what we are today due to many reasons.

First reason is that today many organizations applies agile methods simply as a process and does not understand what agile thinking brings to the table. In this case the discipline of systems thinking can bring them back to some light.

Second reason is that use of systems archetypes can make the organizations to better understand how software development system works and to find the optimal way to operate. Remember every organization is different.

There are many other concepts inside systems thinking discipline such as point of high leverage, reinforcing/balancing feedback, etc. which can help an organization to find the true agile organization or take it to the next level.

Personal mastery

Personal mastery is the process of life long learning to achieve results we truly desire in our life. It is about being master of the what we do. The key to personal mastery is to find the a concept called “Creative tension” which will drive us forward. A further reading on personal mastery: https://www.solonline.org/?page=PersonalMastery

Agile software teams are self managed and consisted of motivated individuals who seeks excellence in whatever they do. Ceremonies such as retrospective look in to what learning which team take forward. Targets such as sprints goals clarifies teams of what they need to achieve. To take out few lines from Agile manifesto principles;

“Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done”

“Continuous attention to technical excellence and good design enhances agility”

The discipline of personal mastery provides a framework to team members to improve their skills which requires in Agile software development.

Shared vision

Shared vision is a vision which the whole organization is committed. Shared vision touches heart of the organization and everybody in the organization shares the same vision.

The vision that is shared is by whole organization aligns it towards the excellence. When you look at the agile manifesto, aligning to business vision is the core focus of agile software development. Again to take few lines from agile manifesto

“Responding to change over following a plan”

“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

“Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.”

Software development teams need to share the vision in order to see the value of welcoming changes even in late in development. When teams share the same vision as business, the highest priority would be to satisfy customer demands would happen automatically.

So the discipline of shared vision provides a framework to help software engineering teams to create, commit & share the organizational shared vision in the true agile organization.

Mental model

Software development is all about making abstractions about real world. Take an example of an order processing software. Order processing happens physically in a factory and software development teams needs to model it successfully to create a valuable piece of software. Object Oriented Programming concepts helped a great deal for software engineering to come to the place where we are today.

In definition Mental models are the abstractions which we make about real life situations. Mental models exists in minds of people and they exists below the level of conscious so people doesn’t realize the existence of mental model. Human forms mental models for everything. For example we build an abstract model in our mind regarding an upcoming version of Windows or year end Agile conference. We form these models based on limited information we tend to hear or find from somebody or somewhere. There is no guarantee that our mental are right and mental models are never accurate. What is important to be aware of mental models and test them.

Extending above example of order processing, let’s look how the mental models works. When the teams looks for requirements, they talk to people who does the order processing. Order processing people describe the process using abstractions. I.e. they describe their mental model of order processing. Then software developers creates another mental model inside their mind.

What agile movement did was to align with the formation of mental models and make them visible as much as possible. For example we use user stories over lengthy documentations so we don’t fix a wrong mental models. (Note: According Mike Cohn the abstraction of user stories provoke the discussion – which is a very good way to test the mental models)

We use personas to model the real users and we use “As a … I want to … So that …” format to model the real user scenarios & importantly understand the story behind the story. Agile architecture is about modeling the software based on what we know and keep opportunity to improve it as we go rather than fixing a wrong mental model. Concepts like sprint planning, sprint reviews, fast feedback, etc. provides opportunities to explore the mental models further. In waterfall we fix the model and it will be very hard to change.

Although agile movement touches the concept of mental models but the discipline of mental models offer far fetched benefits than what we use today. In my view we can use the discipline of mental model to create a new way of working to review & test the mental models which we form over user needs or requirements.

Team learning

Team learning is about learning as a team to create the results. Set of individual heroes does not always produce results unless they make a good team. The important fact to create a good team is that “team is able to learn as a team not only as individuals”.

Agile software development is based on teams not on individuals. We measure team velocity not individual velocity. We perform team retrospectives not individual performance reviews.

Agile manifesto specifies the need of the team but does not specify how to learn as a team (rightly so). There are many books written & many literature about how to create high-performing teams. But the discipline of team learning provides a different perspective based on systems thinking to practice how to work best as a team.

Conclusion

Agile organization is the one we all are dreaming to work for. Learning organization is what it helps to create the true agile organization.

As noted earlier, agile movement provides guidelines regarding what to achieve & learning organization disciplines provides how to do that.

And the true agile organization is just one benefit of the learning organization.

Further reading

http://newtechusa.net/agile/agile-gateway-drug-to-the-learning-organization

http://www.amazon.com/The-Fifth-Discipline-Practice-Organization/dp/0553456342

Managing queue length

Optimizing the software engineering is no easy task. The intangibility of software assets in combination that most of the engineering happens at the human brain makes it difficult to see the whole picture. Optimization requires visibility. This is where the lean manufacturing provides a good foundation to optimize and improve the software engineering as a whole. The pioneers like Mary Poppendieck & many others have already paved the way.

Taking a leaf out of the lean engineering book, one of the important concepts that comes along with SAFe is “Managing the queue length”. The longer the queue, higher the waiting. In lean manufacturing, waiting is  one of the seven wastes. Therefore managing queue length is important.

What is a queue in software development?

At the first glance, we don’t see any queues in software. But if you make a deeper look in to the definition of the queue, you will see many long queues. According to merriam-webster queue is a “line of people who are waiting for something”.

This can be easily mapped to “List of features which are waiting to be developed”. Going further, product backlog is the list which contains the features (user stories). Does this means the product backlog is a queue?

Is the product backlog a queue?

The product backlog interestingly is not a queue. The justification lies on the fact that queues cannot be easily re-arranged and typically works on FIFO basis. If you try to change the queue at the bus or train station, you will not have happy results.

In this meaning, you can change the order (prioritization) of the backlog so it is not a queue.

Waterfall is primarily about long & costly queues

Everything is planned and scheduled at the beginning in a waterfall project. This is essentially a long queue & it defines who will work on what even at the last week. The problem with the queues are multi-fold. It creates long waiting time for new features. The most dangerous fact is changing the queue (schedule in this case) is very costly.

For example if there going to be a change to the requirement, the whole schedule needs to be adjusted. If there are external dependencies such as delivering a component to another team, etc. then it requires lot of effort to change the queue (schedule).

This is the main reason why changes are primarily considered bad in waterfall projects, no matter what benefits they brings to the business. There are lengthy & costly procedures to pass a change in to the development.

Managing queue length

Any experienced Agile practitioner would tell you that they only plan next two iterations. This is essentially keeping the queue length short. Only the current sprint is committed (SAFe even reduces this commitment by introducing a concept called stretch goals) and it is only the queue we have in agile software development in a way.

All the other stories in the backlog are not planned & kept untouched. This comes from the fourth value of the agile manifesto “Responding to change over following a plan” as well as from the second principle of the manifesto “Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.“

Why we should have shorter queues?

Benefits of the shorter queues are many.

1. The main benefit is that it facilitate the agility by facilitating changes even late in to development.Also changing is less costly.
2. Secondly it makes the waiting time predictable. If there is an urgent change to be introduced, everybody knows it is maximum of one iteration which they have wait. 
3. Ultimately it supports the business by allowing early realization of business value by reducing the waiting time

Summary

In software development, queues are the stories or features scheduled to be taken in to development.

In conclusion, the queues are bad due to cost of change is significant but change is essential. Agile software development by design limits the queue length and therefore supporting businesses to realize benefits quickly.

The takeaway is to limit the queue size by only planning current sprint and next sprint.

Photo: http://blog.hqc.sk.ca/wp-content/uploads/2012/12/Queue-2012-12-11.jpg

What is waterfall?

Recently I have been involved in a discussion what is waterfall (of course in software development perspective).

The very first thing comes to the mind is that waterfall software development is a phased approach (over iterative Agile). You have got defined phases and boundaries to cross like “Requirement phase, Design phase, etc.”

If you stop for a moment and look in detail, you will realize that this is more of a description of the “Execution model”. Or how the Waterfall model is executed. But there is much more deeper conceptual definition if you look at from a business perspective.

When you look at software engineering from the problem vs solution concept, you can see it. Software is a solution for a business problem. And software engineering is all about articulating the solution.

So this is how the Waterfall approaches;

• You have the “Business problem” and
• You have a layer to “Specify the problem” (requirement),
• Another layer to “Specify the solution” (architecture and design),
• Another layer to “Develop the solution” (development),
• Another layer to “Confirm the solution” (testing)
• Final layer to “Apply the solution” to the problem (Deployment).

Through this you can transform the horizontal phases of waterfall to vertical layers and by doing so you can see how the waterfall development is really structured.

I don’t need to mention the waste and the confusion this causes when you have so many layers between the Problem and the Solution.

But Agile software development provides a much more compact approach which is more effective than Waterfall. In Agile, you just have two layers representing the Problem and the Solution. Problem is represented by Product owners (if you consider Scrum) and Solution is represented by Development team. By reducing the unwanted layers in the middle, Agile has reduced the waste and the confusion.

But as a note this doesn’t mean that you can avoid the needed steps like design, testing, deployment, etc. It is just that all are done within one layer

Images;

http://goo.gl/TDVuri

Today’s Quality and Tomorrow’s Quality

Software Quality has been a much discussed topic in software community. Quality has been categorized in to “Quality Assurance (QA)” and “Quality Control (QC)”. In the following article, Jim Highsmith, provide a totally different categorization.  (Jim Highsmith is one of founder signatories of Agile Manifesto) I found it to be a really interesting way of looking at the Quality.

The beauty of this categorization is that  it combines the Quality Assurance and Quality Control remarkably well within the single scope of Quality. This gives a very easy definition for the Quality without needing to go through two separate areas to understand the Quality in its entirety.

“Today’s Quality and Tomorrow’s Quality” – http://jimhighsmith.com/beyond-scope-schedule-and-cost-the-agile-triangle/ 

In simple terms, Today’s Quality ensure that the product is usable today. In other terms, the software does not contain any critical bugs, or it is built to suit the users’ needs, etc. By looking from the traditional Quality Assurance and Quality Control view, achieving Today’s Quality would need both QA and QC activities to be performed.

Whereas “Tomorrow’s Quality” refers to ensuring that software will perform in the future as today. This means ensuring code readability, technical documentation, following accepted design patterns, keeping technical debt low, etc. This also would need both QA and QC activities to be performed.

This is very strong but simple definition of Quality

Do you follow Agile?

Do you follow Agile? How sure are you?

Well, this is not a new question at all. This was there for a long time, may be from the very beginning of Agile. Over the period this was discussed again and again. Even there was some terms created out of this. Take a look at terms like “Scrum BUT”, “Mini Waterfall”, etc. These words are there in our vocabulary because of people are saying they follow Agile but not doing so. That is bad, isn’t it?

I don’t want here to write another post here on this traditional question. Instead what I want to do here is to explore whether there is some simple tool which I can used to get a quantifiable answer. It should be something which can be used by anybody who is having some basic idea about Agile.

If you Google, you will find any resources where you can use to measure yourself. Some of them are listed below.

http://www.dragile.com/index.php/survey/startSurvey/index

http://agileassessments.thoughtworks.com/online-assessments/agile-self-evaluation/management-practices

http://comparativeagility.com/

http://www.leadingagile.com/agile-assessment/

The above are very useful resources and I don’t have any question on that. But they did not carry the simplicity and transparency which I am looking for. As I said, I want something simple and it should be very transparent. When talking about transparency and if you take one of those questionnaires from above, I don’t have a clue why those questions were picked in the questionnaire, what is the basis for them. I wanted a solution which is much more Crystal than those.

When I was thinking further, I ended up going back to the basics. I mean to very basic. If you have a method or a tool which compare what you are doing against the Agile Manifesto, then you can’t be wrong. Yes this is it. If you follow the Agile Manifesto, then you gotta be doing Agile.

When I go to manifesto and look at that, there still was a hurdle to cross. There are two parts of the Manifesto (Agile Values and Principles). What do you use in your small evaluation tool? Is it Values, is it Principles or is it both? I like to use the Principles only.

Why? There are few reasons

1. When you go to agilemanifesto.org and go to Principles, you will see that the topic is “Principles behind the Agile Manifesto”. So these Principles are behind everything. I mean everything when it comes to Agile J Therefore there is no better thing to use than the Agile Principles in my Agile evaluation.
2. Since the Agile Values are derived out of the Principles, the four Agile Values are at a higherlevel compared to something you want to use in an evaluation. For example “Individual and Interactions” would mean many different meanings which can lead to many ambiguous interpretations when used in the evaluation. This can deteriorate the accuracy of the results.
3. Due to number 2 reason above, we should not use both Values and Principles to evaluate. Further using both would complicate the evaluation process which is not what I want

So what is my proposal?

There gotta be few simple steps

1. Appoint a coordinator
2. Get all 12 Agile Principles in a spreadsheet such as MS Excel
3. In front of each Principle, get the team to indicate whether they are following the Principle correctly or not. It is important that the team understand each of these principles clearly.
4. I suggest each member in the team has their independent evaluation sheet and then consolidate them at the end. If you feel that this would complicate the process, then get the whole team in and go through each Principle and agree on a result
5. This should be it.

On conclusion, I have use this way to evaluate few projects and I found it not that easy to do so. But in my view there is no other better way to check whether you are doing Agile or not than evaluating against the Agile Principles.

Finally, I found the following article which has the same thinking. http://neverletdown.net/2012/05/if-you-arent-agile-then-what-are-you/

Movies and Project Management

Well making a movie is no doubt a project and needs careful management to make a good movie. But let us look at movies and project in a different angle.

Time for the background. The October 2011 edition of the PM Networks (the PMI magazine) covers a story about the lessons learned of the great rescue on the Chilean Coal Mine disaster. That single page article questions whether we have learn our lessons from that. Below the main text, a highlighted text area mentions how challenging the management of the rescue project was.

I was then wondering when somebody will would come up with a film on that thinking on how dramatic of series of events of the rescue. Thinking further on the same thought, I was thinking why we like great movies. Specially the action movies. Recall ‘Matrix’, ‘Day after tomorrow’, ‘Anaconda’, Independence Day’, ‘etc. Do they have anything common and what those films to do with Project Management?

Lets park project management a bit and see what is there in common in those films. There is a character or group of characters (lets call them heroes) that is trying to rescue or destroy or achieve something. Be it the world or any other thing.

Let me recall something. What is the definition of a Project? According to PIMBOK version 4 – It “is a temporary endeavor undertaken to create a unique product or service”. It has a start and  end.

Do you smell something here? What is there in the Independence Day? It is to destroy the aliens – “a temporary endeavor” and by destroying the aliens, the word is saved. – “a unique service”. Are we talking about a project here?

Let’s look at few more details here.

• Do we have a schedule here? Yeah very tight schedule. Take Independence day. Capton Miller has only few hours
• Aren’t they having technological, human resources limitations? Plenty. Take independence day
• Aren’t they having conflicts in the team? Always, otherwise the movie will not be that interesting
• The list goes on and on and all those are characteristics of real project constraints and parameters

Let’s look at these stories in these films in a different angle. What are those Super Hero characters like Captain Steven Hiller in Independence Day do? And who are they?

Aren’t they ‘Project Managers’ and don’t they manage ‘Projects’? Be it “saving world”, “protecting a kid” or “killing Anaconda”, “Saving Private Ryan” or anything.

These heroes are high performing Project Managers who manage their Projects in extreme conditions and with many unknowns including Aliens. They are super-efficient in getting their team to achieve the objectives. They always have bad luck to start with but ended up having luck in their side. I wonder what if those super project managers sit for PMP certification.

We love those movies where the project (story) of the movie is really challenging. More the challenge, more we love the movie. We love the movie when the Project Manager is performing under tremendous difficulty. More the difficulty, more we love. I still wonder as project managers, why we really love those projects in the silver screen but not enjoy that much when we face with real projects in much better conditions. Answer may be the human nature.

The Bad Side of Prototyping

You may have heard many good things about prototyping. How much money it has saved, how much time it saved, how much effort it saved, etc. Those are absolutely true,  nothing wrong with them.

But think a second, is everything is good about prototyping? Do we have any downside of it? Well may be. Lets look at that further and focus only on the throwaway prototypes here.

It Kills Thinking

What? Should the prototypes suppose to generate ideas? Yes it is true but;

Imagine that you have a busy customer. You discuss the requirements with him (may be in high-level) and step by step, you come up with the prototype. You will get the feedback on the Prototype. But when time passes (and  if the customer gets busy) he tend to agree with what you come up without much thought from his end. The main reason for that is customer knows that you have got hold of the requirements and therefore he takes easy and pass the responsibility of requirements to you. Good to be a customer. 🙂

Secondly what about the development team? Isn’t it common that when you have a prototype, they tend to follow  exactly as in the prototype rather than thinking what is correct? Need not to mention that Prototype can be wrong. One common excuse on this lack of thinking is that the prototype has being discussed many times and agree with the customer so that is the ultimate truth and why change it. This happens specially if the person who did the prototype is not a permanent member of the team.

Effort of Maintenance and Level of Confusion

Once you build the Prototype, you need to maintain it. If not, it can lead to many confusions.  There are two situations where a half baked prototype can lead to confusions.

• In the Middle of Requirement Elaboration – In this case, you start building the prototype to keep the requirement elaboration flowing. Then after each round of discussion, you keep on updating the prototype based on the feedback. After sometime, you get tired of updating and most importantly now you are familiar with the  logic of the system. These two factors lead not to update the Prototype. Sometimes you tend to refer to the meeting notes. The consequence of this is that you are nowhere with the requirement documentation. The Prototype is not complete as well as there is no proper requirement document.
• During Development – Secondly even if you lucky to sign off a complete prototype at the end of the requirement elaboration, but there will be many further discussions during the product development. These updates to the requirements get rarely reflect in the Prototype. This can lead to problems when the acceptance testing is done.  When the Prototype is used as a requirement document, then you will find it not updated and confusion happens on what is the correct requirement.

Therefore it will become mandatory to maintain the Prototype if you start building it and to maintain a Prototype throughout the Project Life Cycle itself is a project of its own.

No Documentation

Isn’t it good? May be in certain times, but not always.

The reliance of Prototype will eliminate or reduce the requirement documentation. This can lead to many different kind of issues. To list a few

• Difficulty in freezing the scope –  Many customers would require a requirement sign-off. This is difficult when you have a Prototype.
• Inconclusive Requirements – Certain requirements may not be conclusive. For example, it is difficult to describe the different kind of data which will be listed in a grid, through a Prototype. Therefore the developer will not have an understand what sort of data and what characteristics of them to be expected (Domain of the data). To explain further, lets say you have a Grid and you are worried about the alinement, width of the column, etc. A comprehensive document can give inside into those unlike a Prototype.
• Typical Issues of Lack of Requirements Documents – For example, product not confirming to what customer wants, bugs, etc. etc.

What to do with Business Logic

It is true that a Picture is worth of  1000 words but there are things which Picture can not describe. For example, it is very difficult to describe a workflow using a Prototype. In general terms, the Prototype is a very good tool to explain the User Interface but it cannot describe the Business Rules of a system in the same effectiveness.

 

Loss of Focus

On a blog post in a different line of topic in Prototype, David Bernstein describes that users focus on what is missing rather than what is there in a prototype.

Conclusion

I am a strong advocate of prototype and in which I have seen many benefits (not to mentioned that sales team uses Prototypes win projects as well). I just wanted to list down few points where the Prototyper should be careful so that he can avoid those negatives and come up with great results.

Doing Agile- Product Owner

Have you ever heard of the following comments?

“You are the expert, you tell me what to do”

How about these comments?

• “I will not be able to participate the meeting today and if you have any questions, pass them via mail”
• “I have given you all my requirements. Now you can run the project”
• “I will be little bit busy this week as I am working on an important proposal. Since you have good understanding about what I want, let me know if there is anything you can’t resolve“

OK, let’s discuss some details.

Although Agile was first introduced in early nineteen nineties, there are many organizations that haven’t done its first agile project yet. Refer M&T 2008 survey.

There are many issues that an Agile team faces when they tries to use Agile Methodology with a customer organization that does not have previous experiences in Agile.

Let’s take Scrum and start with Product Owner.

Who is Product Owner?

Let’s try to look in to the definition in order to start the discussion. Let’s see what Mountain Goat has to say about the definition.

According to their Redistributable Scrum presentation, PO,

• Define the features of the product
• Decide on release date and content
• Be responsible for the profitability of the product (ROI)
• Prioritize features according to market value
• Adjust features and priority every iteration, as needed
• Accept/reject work

The definition is pretty straight forward and let’s sees how a typical Scrum project would happen for a customer organization that does not have previous experiences in Scrum or Agile.

What is happening?

Now let’s run the project in fast forward mode. The main character is a person who is trying to set up the project. Be it a Project Manager, Scrum Master, Senior Team Member or Head of PMO.

Before starting

• You request that the Product Owner (PO) role should be played by a person from customer’s organization. The response?  You will get a name of a person immediately.
• Then you will explain the process, roles, responsibilities, etc. to the Product Owner and everything is perfectly understood by the PO.
• Then you will put your demands to the table. For example, the PO should be there in Sprint Planning Meeting, Daily Scrum, etc. Response? Every demand is accepted.

So with everything going pretty well, you are pretty happy and excited about the new project and its delivery.

Sprint 0?

• Now you ask the PO to create the User Stories.
• Then after couple of days, you will get a “2-3 page document” that describes the system in high level.

Opps!!!

• Then you will send your User Story template and ask him to document the requirement according to that format.

Let’s discuss some details now.

Typical issues when you have customer PO

1. User Stories –  What is that? Never understands the User Story template and usefulness it (well “never” is bit of exaggerate but it is very rare this understanding is there)
2. Product Backlog – OK, I will do. But never creates the Product Backlog (PB) correctly
3. Requirement ownership; That is me – But the requirement ownership is pushed to the project team when the project progresses
4. Agile process – What is that? Never understand the process related to Scrum/Agile
5. Sprint Planning meeting – Good to have.  This meeting is a meeting where the development team explain their version of requirements to the PO.
6. Scrum Meetings –  I will be there. If you find the guy, you are lucky.
7. Release Demo – This is what I was looking for. Yes you will find the guy attending. He will be mostly happy with what the team has produced. But he will always has the question why functionality X is not there in the release – Where the X functionality is never even planned at the release.
8. Agility- OK, I can change anything. Misunderstanding of Agility. When you initially talk about agile process, the response would be “yeah I have heard about that and it’s good that we are using it”. But what are the thoughts behind those words? Ha, we can change everything.
9. Retrospective- Everything is fine. He will provide very good feedback about working with the team – Of course the team deserves it as they have taken the burden of PO and come up with great results. But availability of PO, commitment of PO, etc. never discussed in the retro.
10. When do you finish –  My demand. Always wanting to know when we can finish the development. Exact date, not possible in Scrum (at least in the start of the development)

Implications to the Project

Let’s see how the above points would affect the project.

Requirement Ownership

If you look at the problems related to requirements in the project, (a proper PB will never come from customer and requirement ownership slowly pushes to the team as project progress) what would typically happen is that the project team gets the requirement ownership. This means we would have

• Missing requirements
• Wrong requirements
• Incomplete requirements

This kills the foundation of Scrum and there is a great risk of wrong product being developed before somebody catches it.

Also this would require deploying shadow resources such as “Proxy PO”, “BA”, “Project Manager”, etc. This makes the cost of development higher than usual.

Lack of understanding of Scrum process

When there is no understanding of the process, there will be no understanding about the activities, tools, techniques, etc. which goes with it. For example, what is sprint burn-down is and the benefits it would give.

This would lead to two major problems

1. Customer is always curious about what is going to happen and how the team would work to achieve the final milestone.  This would lead to a situation where customer is not certain about the things going on with the development team and the bonding between the customer and team is not that cohesive.
2. There will be demands to the project team from the customer for things that are not possible to satisfy from the Agile model. For example when the team will finish the project, etc. This leads to a situation where the team has to revert to non-Agile tool and techniques to satisfy those demands as there is pressure from the management to satisfy those in order to keep the customer happy. For example, how many of you have seen, teams preparing MS project plan to find out the end date in addition to the scrum tools?

This situation will lead to

• Duplicate of work
• Lack of trust towards Scrum process by both the team and customer
• Extra pressure to the project team due to the demands that cannot be satisfied

Lack of participation to the meetings and lack of enthusiasm

• Lack of guidance to the project team resulting project team moving in not so correct direction
• Taking un-necessary effort for the tasks that should not be there in an ideal Scrum. For example requirement elaboration, documentation, etc. This lead to a situation where the team is being distracted from their main tasks in order to perform above mentioned supportive tasks.
• Need to deploy shadow resources such as “Proxy PO”, “BA”, “Project Manager”, etc. This makes the cost of development higher than usual.

Miss-interpretation of Agility?

This miss-conception of anything can be changed anywhere is deadly dangerous to the project. There are many situations that during the Sprint, the requirements are changed, release dates are changed, etc.

• When requirements are changed in the middle, those are not well thought of and never properly planned. This will make wrong functionality being developed, regression issues, etc.
• Also this will lead to requirements being conflicted with each other.
• With this it is no possible to find out team’s real velocity.

Impact of biased retrospective

Answer is pretty simple.  The improvements needed from customer are never done and none of the above discussed issues are ever corrected.

Reasons for the above Situations

1. Customer is the King – Typical customer supplier relationship makes customer to pass everything (especially boring tasks) to supplier. In most cases, the supplier team is not strong enough say no to that. Even if there are courageous personalities in the team who can do that, those people are normally killed by management of supplier stating the importance of customer, etc.
2. Conceptual difference – Scrum and Agile are conceptually different from traditional approach which customers are used to.  Therefore making it little difficult to understand the concepts and specially usefulness of them.  For example, you don’t talk about team velocity, you don’t estimate work in story points. Instead in traditional world you estimate man hours and give them a fixed finish date.
3. Time to enjoy – For some customer PO, working on a supplier project is an opportunity to relax from his tough day today work. Why? In a failure, there is supplier project team to take the blame no matter whose fault is.

What should you do?

• Project is the king. No matter what happened, it should be both customer’s and supplier’s responsibility to ensure that the project is successful. Therefore customers should learn to be more flexible and suppliers should learn to be stronger.
• Education. If you find a customer who does not have previous exposure to Scrum/Agile, it should be a part of the project process to run a workshop to educate them as well as if possible to ask them to go on a professional training. May be it is a good idea to have at least one person from customer to be certified as a certified scrum master.
• There should not be any compromise to process. Unfortunately the process is as good as people it runs. But the project teams should always try to adhere to proper process and should not find alternatives. For example, project teams should not employ “Proxy BAs” and instead should push customer to do the work properly.