Don't overpromise and foreclose on your project

Performance as money

I first was introduced to the idea of performance as money through watching an MIT lecture series on algorithms. In the opening lecture, Charles Leiserson talks about performance as currency:

A good way I think about performance, and the reason it’s on the bottom of the heap, is sort of like performance is like money, it’s like currency. You say what good does a stack of hundred dollar bills do for you? Would you rather have food or water or shelter or whatever? And you’re willing to pay those hundred dollar bills, if you have hundred dollar bills, for that commodity.

when he talks about performance being on the bottom of the heap, what he’s talking about is how other things are more important than performance; features, security, ease of use and so on.

Performance constraints as the debt limit

I’ve posted before on non functional requirements every application should have, and first and foremost amongst those constraints is performance. The deal you need to make with yourself and your users is how much of that performance stack of bills are you willing or able to spend to get there? all of the decisions you make, from language choice to functionality to visual design are informed by this decision. Setting your performance constraint is just like setting your household budget (or the government’s debt ceiling).

And the credit rating?

To extend the metaphor a little bit further than is natural, the credit rating is akin to the likelihood that your project will fail overall. We don’t have a good measure for that in software development, but I do think drawing the parallel is interesting. If you read the S&P downgrade overview, you will see that the decision was in part due to the rising debt burden (technical debt?), but more significantly it is due of the infighting and lack of a clear plan forward:

More broadly, the downgrade reflects our view that the effectiveness, stability, and predictability of American policymaking and political institutions have weakened…

This sounds so much like entrenched behavior of stakeholders on a poorly managed software projects (and its effect) that I couldn’t pass on the opportunity to use it. Simply put, if you have a lack of alignment and clear plan about what is acceptable, what is important and how you are going to spend your resources (or generate income) things aren’t going to be plain sailing.

Performance—Avoid foreclosure

O.K. I really am stretching it here, but I wanted to finish up with how I think you best avoid getting mired in software rot or simply just building something that no-one wants to use. And it’s really similar to being financially prudent:

• Understand that almost everything you do bears some cost to performance
• Decide what is an acceptable level of performance…
• …and spend within your means
• Don’t go into debt(let performance drop below acceptable level)
• But sometimes it’s going to happen. If it does, get a plan in place quickly to address it.

The post Performance – spend it wisely and never raise the debt ceiling appeared first on Eyefodder.

The estimation app in action

The application

The application is really simple. You just break down your project into a few tasks and enter three estimates for each—Optimistic, Nominal and Pessimistic (To get what I mean about these terms, check out this post). What it then does is calculate the expected duration of the tasks as well as a high and low end of a range that you can have 95% confidence in. On top of that, it consolidates all of your estimates into a rolled up view that tells you how long you can expect your full project to take. See, there can be some science behind software estimation after all (if you want the details behind how the estimates are worked out, they’re covered in a previous post).

The sparklines

One of the things I wanted to do over and above a spreadsheet was to make something that was clean and simple to read yet packed a lot of information into each row. What originally was six data points has been reduced to two thanks to using sparklines. I could have possibly also done away with the low and high end estimates and had them as rollovers (like in the main result chart) but I just found it to be too overloaded and hard to read. I’ve found that there has been some really nice outcomes from using the sparklines:

• I can see which tasks have the most uncertainty in them (those with the broadest range)
• I can see how different tasks stack up against one another by comparing the sparklines vertically (they’re all on the same x axis scale). I can do this much quicker than if it were a numeric table
• I can present a hierarchy of information. The ranges are most important, then the expected time (which you can see on rollover), followed by the variance and the shape of the estimate which you can infer from the shape of the chart

The results script

One of my big frustrations is that when people come up with software estimates, they give you values that imply a far greater precision than is reasonable. I’ve seen software estimates for a project in terms like “3042.5 hours” What this does is imply that the estimate is between 3042.45 and 3042.55 hours. Far more likely is that you have added up a number of estimates and at best you might be precise to within ten, twenty or even fifty hours (tough to say without knowing more about the estimate, but you get the drift). With the results script, I’ve tried to represent this by using natural language to lessen the implied precision. A few examples:

• Anything less than 1 is represented as “A few hours” “less than a day”
• Up to a week you have “A couple of days” or “A few days”
• Up to a month you have “About a week, a couple of weeks, three weeks
• More than that, time is measured in months

While looking into this, I also found this article talking about the psychology of the zero. In essence it says that people are more likely to ‘accept’ a number with implied precision than one without. If you want to try this out and use the calculated numbers, rolling over the dots on the results plot will give you them. I also left in the calculated numbers for the individual tasks so you can see what’s going in to the final estimate.

What’s next?

Well, I’m hoping you’ll tell me. Right now, all estimates are in days and part days, but I thought I could add an option so you could put estimates in hours for smaller tasks. Maybe some export options or something. If you use this app and get some value from it, let me know if there’s anything else you’d like to see it do. So go ahead and try it out and let me know what you think!

The post Try this, my mini-application for software estimation appeared first on Eyefodder.

How long would it take you to build “the free world’s most formidable weapon”?

The real reason why your project will take longer than you think

What the Navy realized is that when you give a baseline estimate, the number you tend to give is how long you think it will take in ordinary circumstances; the outcome that will happen most of the time. This isn’t however the average amount of time the task will take. How so? Consider this extract from Bob Martin’s excellent book The Clean Coder where he describes a typical software estimation conversation:

Mike: “How likely is it that you’ll be done in three days?

Peter: “Pretty likely.”

Mike: “Can you put a number on it?”

Peter: “Fifty or sixty percent.”

Mike: “So there’s a good chance that it’ll take you four days.”

Peter: “Yes, in fact it might even take me five or six, though I doubt it.”

Mike: “How much do you doubt it?”

Peter: “Oh, I don’t know … I’m ninety-five percent certain I’ll be done before six days have passed.”

Mike: “You mean it might be seven days?”

Peter: “Well, only if everything goes wrong. Heck, if everything goes wrong, it could take me ten or even eleven days. But it’s not very likely that so much will go wrong.”

As Uncle Bob goes on to explain, what’s going on here is that Mike is describing a probability distribution, something like this:

What is happening is that when you describe how long you think it will take, you are describing the mode – what will happen most often. However because there is more chance of running over than under, you need to take that into account. What the Navy did was find a really quick simple way of finding the mean by looking at three simple estimates for the duration of a task…

Three little estimates

They figured out that if you can get people to estimate these three possible outcomes, you can calculate the expected duration for a task:

Nominal (ψ)

This is the standard issue estimate, what people are used to giving—the amount of time you would expect a task to take under normal circumstances. In the example above it would probably be 3.

Nukem (ω)

This is the longest it could take if pretty much everything went wrong. This is the one in a thousand worst case scenario, the Nukem (for more on this read my last post). Mike’s Nukem is 11.

Mu-nicorn (μ)

I took extreme license with naming this one, but bear with me. This is the your estimate of how long this would take if the stars aligned, unicorns pair programmed with you and generally everything went so well you have to check yourself.

Super Simple Math

Now you have these estimates, you can calculate the expected duration of the task using this formula:

T_e = (μ + 4ψ + ω)/6

You can also get the standard deviation of the estimate, which gives a measure of how uncertain the task is:

σ = (ω – μ )/6

Now, you can tell people that you are pretty confident that your task will take between T_e-2σ and T_e+2σ. This statement is not strictly correct, and I will explain at the end why this is so, but to keep it simple, I would use this.

Putting it to practice

In order to put this into on your project I would recommend breaking your app up into somewhat smaller, more manageable pieces for estimating. In theory, this helps because the law of large numbers implies that your errors cancel out. In practice, it’s also just plain easier to understand your app in small pieces. When you have your estimates for a number of tasks, you can simply add the expected times together to get the expected duration for the project. To get the standard deviation, you have to calculate the square root of the sum of the squares:
σ _project = √ ∑ σ ²

Conclusion

There is nothing like real development data and experience to understand how long things are going to take. Unfortunately you will often have a situation when you don’t have that information available and you are still called upon to give a estimate. This technique can help you do that and have solid reasoning to back you up. I hope you find this useful, and if you want to look into estimation more, I highly recommend both Bob Martins book The Clean Coder mentioned above and Mike Cohn’s book on Agile Estimating and Planning

Post-script – there (probably) be dragons down here

This gets a little involved down here, so if you want to keep it simple, don’t worry about this stuff. As you are usually at the beginning of a project when you do this and are wildly imprecise anyway it shouldn’t make much difference.

The more astute amongst you will have seen an issue with how we worked out our range by simply adding / subtracting two standard deviations from the estimated duration of the project. That assumes that its equally likely we run over as it is that we run under—that is, it’s a normal distribution like this:

Trouble is, as we saw in the example, there is a tail off to the right. This is what’s known as a beta distribution (or at least an example of one), something more like this:

If you look at these overlaid onto one another, you see that the range estimate when using normal distribution actually underestimates the duration a bit. If you want to keep things simple you are probably ok, but it’s worth knowing:

The shape of the distribution is called a beta distribution, and it makes the math for working out the range quite a bit more complicated. You need to calculate a couple of variables that describe the shape of the curve: α and β. These are a bit complicated to work out, but for the sake of completeness, I hunted them down for you here. They are:

α = ((mean-min)/(max-min)) * ((((mean-min)*(max-mean))/σ²)-1)

β = (max-mean)/(mean-min)*α

Once you have these, you can use the Excel BETAINV function to get your minimum and maximum ranges. For example, if you want to calculate an duration as a range you can have 95% confidence in:

low end of range = BETAINV(0.025,α,β,min,max)

high end of range = BETAINV(0.975,α,β,min,max)

The post Software Estimation — A good simple way courtesy of the Navy and the cold war appeared first on Eyefodder.

Duke Nukem Forever Launched after 14 years development; what can this big chested antihero teach you about your project?

Why it overran

There are a number of reasons why the Duke took so long to get his politically incorrect ass back out in the wild. As an adoring public were salivating at the thought of a new Nukem, the development team and its management were stuck in a pattern of events that pretty much ensured a death march of a project:

Scope Creep

Anyone who has ever been on a development project will be familiar with this problem. After the success of Duke Nukem 3D, George Broussard—the game’s creator—kept seeing opportunities for ever more awesome features to put in the experience:

Broussard simply couldn’t tolerate the idea of Duke Nukem Forever coming out with anything other than the latest and greatest technology and awe-inspiring gameplay. He didn’t just want it to be good. It had to surpass every other game that had ever existed, the same way the original Duke Nukem 3D had.

Snow scenes? You got it. Online Multiplayer? Of course! These sorts of things may seem price-of-admission now, but bear in mind that these features were being added over ten years ago. And – I’m not saying that these were bad ideas or unnecessary ideas, just that the goalposts kept moving, which makes it kind of tough to finish.

Technology Death Spiral

When Duke Nukem 3D came out, it was built on a game engine named “build” by a child prodigy called Ken Silverman. In order to keep up with ever increasing needs of the project (see above), Broussard swapped out for a new game engine not once, but twice. This, again, may have been necessary, but it meant that the development team were always playing catch up.

Too much cash, not enough business

Here’s the thing: Duke Nukem 3D was a runaway success. So much so, that unlike before, Broussard could basically fund development themselves. Creative freedom is great, but without the business reminding you that what your building is for people to use and enjoy you run the risk of getting stuck into some sort of artwank vortex.

Fear of Success—gilding the lily

I’m not a console game developer, but when I heard it takes up to a year from “locking a game down” to launch I really felt sorry for those guys. The projects I work on have some level of locking down code to close out the last remaining bugs, but if it took a year, I can see how it would be an almost paralysing decision to make.

You’re no different from Nukem

To some extent or another, your project will have some of these issues. Or the server will blow up. If you don’t have complete freedom and never know when to finish the business will be on your ass. Or the IT department. Or someone else’s sucky APIs will kill you slowly as you realise you’re the first person to use them. Or you will get a fundamental set of assumptions so wrong that you need to reset. Or you hire someone that turns out to be worse than useless (and yeah – I know you will fire them quickly, but it will still suck energy and time from the team).

Once you realize that some or all of these things will happen to your project, you can start to use this information to your advantage…

Duke Nukem Forever is at the limit of how long things will take

See here’s the thing: Duke Nukem is pretty much at the outer limit of how long you can get away with a project overrunning. Any longer and it would have been be laid to rest. If it weren’t for the cult-like following DNF’s only achievement would have been 3 Vaporware of the year awards (and a lifetime award to boot). Gearbox Software, the company that picked up development after the original developer went bankrupt probably realized that they were better off launching a mediocre game and starting from scratch than suffering yet another rewrite. This gives them the opportunity to reset the ‘development time’ clock.

Enter the Nukem as a unit of development time

With all this in mind, I propose that we consider using the lesson of Duke Nukem Forever to give us a new unit of software estimation:

A Nukem: The amount of time a project or part thereof will take when pretty much anything that can go wrong does.

So – how long will your next project take?

I can’t be precise, but I can be accurate. This is how long your next project will take longer than you expect it to to and less than a Nukem. Or, in mathematical terms:

ψ < x < ω

Where:

ψ = the amount of time you expect it to take

x = the amount of time it will actually take

ω = a Nukem

What’s interesting here is that many of the issues that Nukem had weren’t really down to code or developers, yet developers are the ones who get asked to estimate project duration and get held to it. Oh well, tough shit, that’s what you get for being a creator. Broussard’s stock response of “screw you publisher, it’ll be ready when it’s ready” actually ended up killing his baby. Of course if you go to someone who is paying the bills with this range, you’ll probably not get vey far. But there is hope. In my next post I’ll show you how you can use this information to arrive at a range that is more palatable to them, has some science behind it, and allows the business to make some decisions based on your assumptions.

The post Your project will take longer than you think and less than a Nukem appeared first on Eyefodder.