I am often asked how do you measure the performance of software development teams.
Its a good question, and its surprising that I don’t have any clear answers. Recently I’ve been reading Hard Facts, Dangerous Half-Truths And Total Nonsense: Profiting From Evidence-Based Management. Which talks a lot about evidence based management and defining what real results are.
First and foremost, I expect my team to deliver something. Since our projects are mostly iterative based we make real deliverables to customers every 1-2 weeks. That right we have something real to show off after a one or two weeks. If a team can deliver something, anything, something has seriously gone wrong.
Second I look at story velocity. This measurement is usually aggregated across a single group, although sometime I peek at contractor story velocity, and compare it to the average to see if they are worth keeping. This measurement is only valid if the team stays together. If the team member keep moving around then the measurements get all screwed up.
Which brings me to my next metric. I look to see the number of people moving in and out of the project. Within an iteration the number should be zero. If a person leaves at the end of an iteration, they shouldn’t come back, so we measure that too.
Finally, I look at the severity and the number of bugs filed post launch. This includes the number of rollbacks, and outages caused by poor release planning. Any bugs which are severe enough to fix within two days or less, and filed after launch are a bad sign. Either, the requirements were not communicated correctly, or the level of quality was poor. Either way these critical post launch bugs are a huge time sink. Bugs filed after launch which must be fixed in one-two weeks or less are also measured.
Lastly something that I would like to measure, but have a hard time doing it is the amount of idle time for engineers. A poor manager doesn’t know how to keep his/her staff occupied. A poor manager will let their staff be blocked for long periods of time.
For non-agile projects I don’t have as many good measurements. With planned projects I make sure milestones are hit, and that a milestone exists every week or two. I also keep an eye out for heroes and cowboys. A cowboy or a hero is a bad sign, because it means one person is running too much of the project and their is a lack of peer review and team accountability. Either means a manager will never get the true status of a project.