Tag: productivity

Estimating software projects

Couple weeks ago I came across a great article about software estimation by Pawel Brodzinski which made me think about how my approach to estimating software projects has evolved over time.

At the moment I work in product development where Agile approach is used to deliver new increments of software. We have stable teams and we use Story Points and Velocity to estimate and plan our sprints and releases. But I used to work in project-based companies where a reliable process for estimating projects was a key factor for our long-term success. It wasn’t important only from project perspective, but the estimates were used to forecast our revenue, they influenced recruitment, resorucing plans, etc. They had to be more or less right.

Estimating software projects is difficult, error-prone and time-consuming. Of course, one may say that Agile approach is a good solution to this problem, but unfortunately not all the clients are willing to follow it. In your response to many RFPs you’re obliged to provide 2 numbers (I call them “RFP twins”) – cost and delivery date – before you even have a real chance to talk the client.

My personal experience confirms that spending more and more time on estimating doesn’t improve the quality of an output you get. It’s even more misleading because adding more details to the scope definition doesn’t make the final estimates better, but it makes you biased by feeling more certain about what you have just come up with.

Years ago I thought that giving lot of time to senior developers to analyse the scope, break it down into tasks and estimate them, would give us an accurate project schedule and minimise a risk of failure. Project life showed, however, that it didn’t work as expected and that our estimates were usually wrong regardless of how much time we invested in getting them.

Over time my approach to estimating software projects has changed and I started using a different process for getting estimates. The first step is to define a team that is going to work on the project (something like 1 full-time visual designer, 2 developers and QA specialist, etc.) and then involve experts (that team, preferrably) to say how many days/weeks/months (units depend on an overall project size) that project is going to take. The scope is no longer split into fine-grained tasks, but it’s rather broken down into a couple of a high-level modules/functionalities/deliverables. The whole process is time-boxed to prevent digging into too many details and to minimise the effort that is invested.

The second step is to compare the results with projects of a similar type and size that were done in the past. I don’t have a precise definition of what “similar type and size” means, but the teams usually don’t find it difficult (it can be a project that a similar team worked on or a project with similar functionalities, etc.). This approach is far less “scientific” than the one described by Pawel in his article, but the goal and reasoning remain the same – to use historical data to verify the estimates and to avoid being biased.

Finally, the estimates should be provided as ranges (like “this project/functionality requires 2-4 weeks of a given team effort”) – bearing in mind the huge uncertainty creative software projects are faced with you should be aiming for a right order of magnitude rather than a precise number you’re never going to meet.

Last but not least, all the assumtions that were made during the process should be documented and stored along with the estimates. I even suggest that a thorough, generic list of assumtions is built over time so that it can be used as a checklist and therefore save time and make you not forget about “something obvious”. Moreover, assumptions usually enable you to come up with some changes to project scope that can be presented to the client and that could lower the costs of the whole endavour. From my experience I can say that such suggestions are usually a good starting point for more detailed discussions with the client (especially when the client was initially reluctant to do so) and makes it easier to convince the client to have some workshops with you before final cost/schedule commitments are done.

 

To summarise, I recommend using the following ideas when estimating software projects:

  • use historical and empirical data as much as possible to avoid biases
  • if only possible, organise workshops with the client to get a better understanding of the client’s needs
  • combine different approaches (i.e. statistical forecasting + sanity check by experts)
  • don’t spend too much time on estimating – it won’t improve your numbers after all
  • provide estimates in ranges (the project will take from n to m days/weeks/months)
  • invest in accuracy, not precision
  • document your assumtions

 

 

Limiting Work in Progress

In many organisations being busy is valued more than delivering the results. At first sight, it may sound quite reasonable – after all, we’re all paid for having hands full of work, aren’t we?

Unfortunately, it doesn’t work that way. In a vast majority of cases, the more tasks we try to complete, the less we really accomplish. And pushing yourself to work even harder doesn’t change anything.

Why is that?

The scenario described above is quite common for many teams and it’s usually caused by an excessive Work in Progress (WiP). WiP is anything that is started but not yet completed. It’s a User Story that was started, but hasn’t been Done yet – it’s either blocked, or awaiting QA, or resides in any other halfway state.

Excessive Work in Progress is harmful for several reasons that, among others, include the following:

  • It leads to mistaking activity to accomplishing valuable work.

Limiting WiP introduces slack time which is regrettably not acceptable in far too many organisations that blindly follow “100% utilisation” approach. The truth is that fair amount of slack time is desirable; it happens when any team member, for whatever reasons, is “blocked” because of WiP limits and cannot do regular work. I’ll cover slack time and 100% utilisation myth in a separate blog post.

  • It diverts our attention from what’s of the highest priority to less important tasks.

When we get blocked it’s much easier to move on to something else rather than bend over backwards to get an original task or blocker resolved. Unfortunately, in most cases it implies an implicit, ad-hoc prioritisation instead of following an original order of work (after all, an original task was of a higher priority – otherwise you wouldn’t be working on that, right?).

  • It’s exhausting mentally.

Please bear in mind that anything you have started but not finished occupies some part of your available mental capacity. And the more WIP, the more mental energy is consumed with trying to keep track of it all. Although one can argue that even with many tasks started people work only on one of them at the time, unfortunately, it’s not true – Zeigarnik Effect suggests is that our brains will be switching context despite what we consciously think it would do.

  • Context switching is a cost.

Context switching prevents us from getting into flow which is a key to being highly productive. Usually, it’s exemplified by talking about distractions and interruptions (instant messages, phone calls, etc.), but having too many tasks in progress causes the same problem – it works counter to flow. We’re most efficient when allowed to complete one thing uninterrupted. Switching between tasks requires context change and it takes time we need to get at full speed back again. Multitasking  harms our productivity.

Now, as you know that Work in Progress should be limited, the question arises how it can be achieved? The solution may differ from team to team, but the common idea is to disallow to have more stories in “in-progress-like” status (In Progress, Awaiting QA, Awaiting Code Review, Blocked, etc.) than an upfront agreed threshold. It basically means that none of the team members can take on a new item if a given number of tasks are already being processed.

This approach is usually backed with a simple guidance that whenever you finish work on an item you should check if there aren’t any unassigned tasks starting from the rightmost part of the board (that’s closest to the Done column). And you either pick one or you move one column to the left (towards To Do column) and keep repeating the whole process until you find something to work on.

Agile is not about being busy, but about delivering value to the clients. Delivering it fast to foster learning and outperform competitors. And Work in Progress limits encourage a culture of “done”. Moreover, WiP limits make impediments, blockers and bottlenecks visible and enable the team to swarm around them to get them resolved as soon as possible. WiP limits smooth the flow and increase overall team’s productivity.

So, please stop starting things and start finishing them!