Predictions and Promises

I have been pondering the Great Dichotomy between the two offices of NITI a lot lately. We have one office in Toronto (sales, marketing, support, manufacturing, etc), and another in Montreal (R&D), and their cultures are very different. Neither is exactly wrong, but people from one culture have a lot of trouble understanding the other.

I think I now at least partly understand the difference. And, surely to the joy of the more technical readers here, I can explain it in terms of task scheduling algorithms.

In Montreal we use a system called Schedulator that I originally "designed" (it has since been rewritten). It basically takes Joel Spolsky's Painless Software Schedules essay and automates it.

Schedulator, and Joel's original essay, is about schedule prediction. It helps you predict when your project is going to reach various milestones. Used under the right conditions, it can work very well.

Now what happens if you get some surprise bugs in an earlier release? Joel didn't say anything about that. Well, what happens is your schedule will slip, because high-priority tasks insert themselves in front of everything else. Schedulator manages this automatically, and each day you can look at a pretty graph of your project schedule and watch the end date slip further into the future.

Inside R&D, there are other mini-deadlines as well. We have this concept of a Zero Bug Bounce (yes, stolen directly from Microsoft), where the idea is that all tasks are either complete, found too recently to be reasonably fixed, or shovable into a future release. And it's someone's job to predict when the next bounce will be, then make sure you get done on time.

Hey, what's this "make sure" business? Schedulator updates its predictions in real time, right? So the bounce date might slip, but only for a good reason, right? So there's nothing we can do, right?

Sort of. The problem is, if the last release-critical bug just doesn't seem very important, something more important will always come up, the new thing will jump to the front of the schedule, and the bounce will never get done. Thus, we have to introduce a form of deadline-based prioritization as we get very close to a bounce date. In the deadline-based system, you convert your prediction into a promise: that is, Schedulator tells you, as of right now, when you think you can get it done. You add a bit of time, just in case. And then you promise that you will absolutely have it done by that time, no matter what.

The same overall method applies for software final release dates (sooner or later, you just stop fixing bugs - even important-seeming ones - in the old version so you can finally just finish the new one). This method feels wrong; we're raising the priority of obviously lower-priority tasks above obviously important ones. Developers want things to be simple; they like prediction, but they don't like keeping promises that force them to break their priority scheme.

Here's my big insight. Marketing people are exactly the opposite. They don't care about predictions at all. They only care about promises. You can predict bug fix times, bounce dates, or anything else, but in the end, they want you to commit to a release date, and they want to do their work, confident that you'll be done by the date you promised. Exactly what that date is isn't very important; that you guarantee it is what's critical, because otherwise they can't properly do their thing.

And Marketing-type people carry their concern for promises above prediction way too far. Joel's article, above, talks about how Microsoft Project is useful, but not for writing software. I now understand why: it's because the type of work is very different. Some tasks a salesperson might have to do - like arranging a meeting - can take two weeks, but only, say, 5% of their effort during that time. Software developers work mostly linearly (one 100% task at a time), while salespeople do a lot of tasks in parallel. So it's easy for a salesperson to promise, "I'll have this meeting set up within about two weeks, no problem." Even if something new and "more important" comes up, they don't delay the previous task; they just do one more thing in the same amount of time. But after a person gets sufficiently heavily loaded with 5% tasks, each new task does delay the work, the same way as it would for a developer. That's the exception, however, not the rule, because simply not overloading your salespeople can dodge the problem.

A couple of weeks ago, the CEO told me, "I think we need a Schedulator for the rest of the company." This was true, in a way: we have people in sales and marketing who are having trouble making and keeping their due date promises. Schedulator, in contrast, was created to help developers, who were having trouble making predictions. It doesn't help them keep promises (although better predictions are one necessary part of saner promises); in fact, unless you're careful, it gives them an excuse for breaking promises when the predictions change. The design we came up with for a "marketing schedulator" is actually totally different from the original Schedulator. It involves a central, top-down plan from Microsoft Project, and only three bits of information fed back from individuals: notes about each task, % complete, and a checkbox: "in danger of missing the due date." Despite your best intentions, you might still have to sometimes break promises, but when this is (albeit rarely) the case, now you have to check a checkbox and have the CEO get angry at you.

I suppose the ultimate Schedulator of the future could combine the two concepts. It would predict dates, then help you promise them by auto-increasing the priority and moving tasks around so that future predictions always leave your existing promises intact. Then everyone on both sides of the fence could use it. Meanwhile, I think we need two systems, which will be strangely perpendicular to each other.