In parts 1-3 I've introduced a simple framework for dealing with common challenges in software development. This framework is based around an agile concept called Card Conversation Confirmation and is focussed on giving a team an instant agile software development lifecycle that can be rolled out gradually with relative ease to take them away from an ad-hoc existence. In part 4 we're looking at Key Performance Indicators (KPI) that can be recorded and reported in the early stages to illustrate the framework's success and promote agile as the way forward. In the next article I'll discuss how versatile this framework is to illustrate how it can be adopted by any organisation in any situation. So, parts 1-3 supply the framework in principle and 4-5 will offer a means to communicate its value and gain the courage to establish that first stepping stone to becoming agile.

If a team wants to become agile it must have a reason for wanting to do so - there must be something not going to plan - something broken about the process - or just a desire to perform better. Regardless of the reason the status quo must surely present some indicators that prompt this move into the agile world - maybe the development team consistently receives requirements that are poorly briefed and this is followed by poor communication leading to delays in getting started - or maybe the build is consistently buggy and the testing isn't comprehensive enough so poor quality code is delivered into production - or maybe the business is trying to deliver so much so quickly and the team just doesn't seem to have the bandwidth. I say doesn't seem to have the bandwidth because without recording some KPIs and making some changes no one will know for certain. A successful software development lifecycle needs some measurements relating to its throughput. These are not necessarily the kinds of KPI that the business may immediately be interested in – they may be more interested in “how quickly can the team deliver?” but that question doesn’t have a straightforward answer – “it depends on several factors!” The KPIs we’re looking at here provide information about these factors that isn’t technical and can easily be used to reinforce answers to the questions the business asks and will bring about a greater understanding of the dynamics of the software development lifecycle across the extended team.

Lets take that as an example - suppose for argument's sake the business is making 8 new requests per week on average but the team is only deploying 5 a week on average - where are the other 3? One might suppose that there is a stack of un-started requests and if there is then that's one KPI we can start to work on. Often its the case that such a stack if it were to be prioritised contains only a handful of useful requests leaving the majority as nothing more than nice to have's. This can happen for a number of reasons but two of the most critical to deal with could be:

1. The requests may be being submitted by several stakeholders and of that group some have more influence than others and some may carry more commercial weight than others. Requests submitted by less wieldy stakeholders are not to be diminished simply by that token but that is probably exactly what is happening - these are the items that are stacking up but could add significant value.

2. Not everything that is being delivered is necessarily what needs to be delivered now. Some requests come in weeks or even days before their delivery is required but others come in months in advance. Is work load being prioritised and scheduled properly? There is a risk that less valuable requests are being delivered ahead of more valuable ones.

By measuring this aspect right at the beginning of the process remarkable improvements can benefits can be realised very quickly. The overflow of requests and the two problems listed above are major contributors to challenges such as late introduction of change within a sprint - which is arguably the biggest challenge to any ad-hoc development situation. To change this for good its important to keep up the measurements as part of that change so that everyone can see that its still works 1year down the line.

This is where Gate 1 comes in (from Part 3) - getting Cards onto the backlog - if we consider for now that each request is a card (each of which may have sub-cards) then not all of them necessarily deserve to be on the backlog - they can remain on what I'll refer to as the demand log - and once they have been put on the back log the next step is to get them scheduled.

So every week between 6 and 10 new requests come in from the business and these should be placed on the demand log. This requires the demand log to be re-prioritised immediately to include these new items in the correct order - this is so important that no item can be added to the demand log without assigning it a priority relevant to everything else in the list - the reason being that we need to start talking about it - asking questions like "Why is this in the demand log?" - the answers are surely going to involve statements like "well it adds more value than item number 7" - so set the priority first - it might change at any point but set it first.

The team then needs to decide whether it can or should go on the backlog. Each organisation will have its own criteria for making this decision and that may change from one request to the next - so its useful to record why each item is allowed onto the backlog - part of that criteria should certainly be "the team understands the request" and maybe "the request has been submitted in the correct format and all information has been provided". Remember - the business doesn't want the team to waste its energy estimating or even reading requests that are not ready or important enough at the time. Suppose a problem occurs with a particular request and this becomes disruptive to the whole process - a member of the senior management team might consider it to be a low priority and question why its in the backlog - if its been recorded the answer can be given immediately - all these little things help. It also becomes quite useful if the backlog begins to become unmanageable - if the number of requests sitting on it unscheduled is building up the team needs to know whether to remove some items if they really shouldn't be there or bring in some help to get them cleared down - the reason why they got there in the first place is key to that decision.

When it comes to deciding whether to schedule a request care must be taken to ensure the business has enough confidence that their work is in the pipeline but this balanced against the risk of scheduling something too early and setting too many unrealistic expectations as items get shunted down the priority ladder as new more important ones arrive further down the calendar.

The KPIs the team records against these events in the lifecycle can be recorded in anyway that is deemed suitable for their situation - I wouldn't want to be prescriptive - this article is just to help illustrate the importance of KPIs. Here are some important ones to consider and this is how the KPI log might look:

The Demand Log figures are the ones the team was worried about in the first place and these still exist as they did before - maybe this is the first time they've been written down though. The team can now have a conversation about how it’s organising that demand log better than it did before by measuring how many requests move from it into the product backlog and ultimately get delivered – note that the demand log count is steadily increasing but the backlog is maintaining a steady pace. This could mean several things – maybe the criteria for getting onto the backlog is working perfectly – maybe its too strict not letting enough value through – maybe the team are deliberately imposing limits to artificially set the backlog pace. Hopefully its not the latter but what ever the reason the team wouldn’t be able to focus on the situation without having the data to show that it exists.

The business will be able to ask its self "are we putting too much stuff on the demand log?" - this may lead to an improvement in the business which gives them better clarity over what they want to deliver and the trend may go down - on the other hand it may stay the same or increase coupled with an increase in the value of the requests which could only be a good thing for everyone.

The information collected about priority movements is useful in those discussions where there is a poor value or poor quality delivery and questions are asked - if there were an unusually high level of priority movements at any stage this may have been a contributory factor. Also useful if these numbers are consistently high and delivery is consistently below expectations - the team could suggest that the business support a reduction in priority movements for a trial period to see if this helps - helping to sell the benefits of agile to the whole team.

The final two I've mentioned in the table above are useful in indicating that the backlog is or is not as the case may be - being delivered at a steady pace. I would strongly recommend monitoring the team's velocity in terms of what it delivers but also the velocity being demanded by the product backlog. This is maybe a stage further on for some teams than just monitoring the number of requests on each but is is a much more relative and therefore correlated comparison.

As with the rest of this process the team needs to take it slowly - don't try to measure too many KPIs all at once - start with the measurements that are going to help the most and let them run for a while. Make sure the team has a strategy for improving on that number and don't add any more KPIs unless they are necessary and have a similar strategy. Too many KPIs will spoil the benefit - it will be impossible to decide which measurement is responsible for what and arguments will ensue - about a dozen or so is probably enough. If you think you need more - try to create a window of focus on just a few for any given period and once improvements in that area have been made move the window to focus on some other areas that may have been suffering in the meantime.