The Regenerate Web

It's been a little quiet from me the past couple of weeks.  (Well, many of you were on spring break, so I doubt you missed me all that much... after all - tequila shots and warm breezes were calling.)

The past month has been fun for me.  As I mentioned in an earlier post, I'm now contracting full time as a project manager.  Yes, it is fun... this is the kind of stuff that gets my adrenaline going.

While the schedule has been an adjustment, the activity is just like riding a bicycle.  Project plans, status reports, meeting minutes, issues logs, risk management.  You never forget.

Some have asked me why I took a detour from the speaking and writing to go back to a full-time cubicle-dwelling contract for a few months.  (The reality is that it isn't much of a detour as I still have a speaking schedule, and I'm in talks about my next book, but I digress.)  The biggest reason I agreed to take on this contract can be summed up in two words:  "street cred."

It's the same reason a successful actor agrees to do an independent film at a reduced rate, or why an athlete will join in a pick-up game of ball.  As a project manager, I never want to get too far away from my roots.  I don't want my expertise to be academic.  As a Chief Accomplishment Officer, I'm wired to DO, to PERFORM, and to ACCOMPLISH.

So for a few months (as long as my client and I agree that I'm adding value to the project and to the organization), I'll stick around.  I'll share some knowledge.  I'll learn a few new tricks myself.  And more importantly, I'll maintain my street cred.

What are YOU doing to maintain your credibility in your field?

Suppose one of you wants to build a tower. Will he not first sit down and estimate the cost to see if he has enough money to complete it? For if he lays the foundation and is not able to finish it, everyone who sees it will ridicule him saying, "This fellow began to build and was not able to finish." - Luke 14:28-30

So why then do some projects, their sponsors, they managers and the developers of those projects utterly fail to comprehend the fact that planning is a critical success factor for a project's success?

We do they delve feet first into development, code writing, and releasing products when they have not the slightest notion of what done looks like, how long it will take to get to done, and how much done costs?

I have been thinking about this topic of whether actions add up to produce results for a while and recorded this eight minute audio post. Check it out.

Prescription Instructions

1. The magic pill must be taken, by managers, once per week, on Monday.
2. The magic pill has no effect on the manager during the week until 40 work hours have been logged.
3. Once 40 work hours have been logged, the magic pill prevents the manager from thinking about work activities.
4. Blackberries and remote email are considered work activities by the magic pill.
5. If the manager persists in thinking about work activities, the magic pill will shut down conscious thought and make the subject sleep for a temporary period (naptime).
6. In most cases, the magic pill has been shown to change the work habits of managers, who now know they must be effective within 40 work hours per week.
7. In clinical trials, in some cases, side effects of the magic pill have improved family and social activities.

Think about this magic pill. If you took the magic pill, what habits would you change to become more effective?

The use of 3 point estimates is fraught with statistical integrity problems. Don't elicit estimates in this way.

While many will have anecdotal evidence of this working for them personally on small self contained projects, the issue of estimating impacts enterprise, multiple supplier, and integrated project teams (IPTs).

Summary of the Three Point Estimate Approach to Cost and Schedule

• The estimates are biased values
• The estimates are not anchored to the underlying probability distributes
• The estimates are not correlated with the other estimates in the project network or cost structure model

This approach starts with two primary sources. Effective Risk Management, Dr. Edmund Conrow, AIAA, 2003 and "Judgment Under Uncertainty: heuristics and Biases," Tversky and Kahnemanm Science, Volume 185, 27 September 1974.

People often rely on a limited number of heuristic principles that reduce complex tasks of assessing probabilities and predicting values to simpler judgmental operations. These heuristics can lead to biased assessments of probability. Three such heuristics, first discussed by Yversky and Kahneman, include adjustment and anchoring, availability, and representativeness.

The broader discussion of these biases can be found in the full length discussion in Judgment Under Uncertainty: Heuristics and Biases, Cambridge University Press 1982. This is a primary source work. Conrow and others have derived the impacts from this material. Primary impacts can be found in Complex estimating problems in software development, large construction, oil & gas capacity estimating (field production capacities), and economics.

As stated by Tversky and Kaheman: "In many situation people make estimates by starting from the initial value that is adjusted to yield the final answer." The judgment heuristic is called "adjustments. Adjustments are typically insufficient - different starting points yield different estimates which are biases toward the initial values (hence the term anchoring).

The consequences of adjustment and anchoring lead to an underestimation bias of potential minimum and maximum values associated with the likelihood of an event, a duration, or the probability distribution representing these activities.

So Now the 3 Point Estimate Problem - Part 2

Many of the probabilistic questions (what is the likely duration of the task?) with which people are concerned belong to one of the following types:

• What is the probability of an object A belonging to a class B?
  
• What is the probability that process B will generate event A? 

In these cases, people often rely on representativeness in which the probabilities are evaluated by the degrees to which A is representative of B, that is, by the degree to which A resembles B.

This is what happens when you ask a developer or planned to define the Most Likely duration for work and then ask what is the upper and lower limits of that duration. A Naval Research study and studies in the oil & gas field production estimating domain  have clearly shown that the order in which you ask those question results in statistically significant difference in the estimated values.

But There is More

The heuristic above is called representativeness and it affects risk related decisions (duration estimates are actually risk estimates - the risk of completing on or before a specific date). This risks include:

1. Insensitivity to prior probability - the Bayesian statistical impacts spoken about by K in Bayes Theorem for Project Managers.
2. Insensitivity to sample size - this is the classic anecdotal evidence of "it works for me on my sample projects, so it must work in general."
3. Misconceptions of change - the drawing of a triangle or any arbitrary curve, placing to end points and marking the peak of the curve and drawing conclusions in the absence of any confirm that this pseudo-probability distribution has any connection what so ever with reality. Or worse, reverse engineering a probability distribution curve to fit the data for one project and then generalizing that process to all projects in the universe. This latter is how the Beta curve, with fixed symmetric standard deviations resulted in the PERT formula. PERT can have up to 27% unfavorable bias in the presence of coupling between tasks.
4. Insensitivity to predictability - again a Bayesian problem.
5. The illusion of validity - no calibrated variances.
6. Misconceptions of regression.

It's the first of these heuristics that is the source of the exclusion of the 3-point process as practiced by many - asking the subject matter expert for the three values.

When faced by ambiguity or uncertain information, people have a tendency to interpret information that confirms their beliefs; with new data they tend to accept information that confirms their beliefs but to question new information that conflicts with them.

Improving Risk Communication, National Research Council, National Academy Press, Washington DC, 1989.

Conclusion

Making the 3 point estimate process the basis of project cost and schedule estimating is very sporty business. For personal projects, projects where you are the personal lead, have a small group of friends working the project, or the project is essentially self contained with your small group of friends, the risk is lower.

If your project is being implemented in a larger context, by essentially strangers - contractors, teams beyond your direct experience and control (other staff teams, development teams from other locations, subcontractors, 3rd parties, COTS providers) - you're taking on risk and may not know it, be able to quantify it, or even characterize this risk profile.

This is a primary source of project failure - there is no credible source of cost and schedule estimates when you simply ask some "what is the most likely, upper, and lower limits of cost or duration." You've planted the seeds of late and over budget and may not even know it.

"I know planning is important, but I have so much to do today," Lauren explained, hoping I would let her off the hook.

I nodded my head. "I know you have a lot to do, today. How much of what you do today will be effective?" I asked.

"What do you mean? I have phone calls to return, emails to answer, meetings to go to. I have a couple of employees I have to speak to about things they were supposed to take care. I have two projects that are behind schedule. A lot of things piled up over the past week."

"How much of what you do today will be effective?" I repeated.

"Well." Lauren stopped. "I know some things are more important than other things."

"And, how do you make that decision? How do you know what you do is effective? How do you know what you do is important?" Lauren's posture shifted. She backed off the table between us. She was listening. "I will venture that 80 percent of what you do today will be wasted time and only 20 percent of what you do will be effective. How will you know you are working on the 20 percent?"

A while ago Jeff and I had Eric Sink on the Stack Overflow Podcast, and we were yammering on about version control, especially the trendy new distributed version control systems, like Mercurial and Git.

In that podcast, I said, “To me, the fact that they make branching and merging easier just means that your coworkers are more likely to branch and merge, and you’re more likely to be confused.”

This is what Taco looks like nowWell, you know, that podcast is not prepared carefully in advance; it’s just a couple of people shooting the breeze. So what usually happens is that we say things that are, to use the technical term, wrong. Usually they are wrong either in details or in spirit, or in details and in spirit, but this time, I was just plain wrong. Like strawberry pizza. Or jalapeño bagels. WRONG.

Long before this podcast occurred, my team had switched to Mercurial, and the switch really confused me, so I hired someone to check in code for me (just kidding). I did struggle along for a while by memorizing a few key commands, imagining that they were working just like Subversion, but when something didn’t go the way it would have with Subversion, I got confused, and would pretty much just have to run down the hall to get Benjamin or Jacob to help.

And then my team said, hey you know what? This Mercurial bug-juice is really amazing, we want to actually make a code review product that works with it, and, and, what’s more, we think that there’s a big market providing commercial support and hosting for it (Mercurial itself is freely available under GPL, but a lot of corporations want some kind of support before they’ll use something).

And I thought, what do I know? But as you know I don’t really make the decisions around here, because “management is a support function,” so they took all the interns, all six of them, and set off to build a product around Mercurial.

I decided I better figure out what the heck is going on with this “distributed version control” stuff before somebody asks me a question about the products that my company allegedly sells, and I don’t have an answer, and somebody in the blogo-“sphere” writes another article about me junking the sharp.

And I studied, and studied, and finally figured something out. Which I want to share with you.

With distributed version control, the distributed part is actually not the most interesting part.

The interesting part is that these systems think in terms of changes, not in terms of versions.

That’s a very zen-like thing to say, I know. Traditional version control thinks: OK, I have version 1. And now I have version 2. And now I have version 3.

And distributed version control thinks, I had nothing. And then I got these changes. And then I got these other changes.

It’s a different Program Model, so the user model has to change.

In Subversion, you might think, “bring my version up to date with the main version” or “go back to the previous version.”

In Mercurial, you think, “get me Jacob’s change set” or “let’s just forget that change set.”

If you come at Mercurial with a Subversion mindset, things will almost work, but when they don’t, you’ll be confused, unhappy, and unsuccessful, and you’ll hate Mercurial.

Whereas if you free your mind and reimagine version control, and grok the zen of the difference between thinking about managing the versions vs. thinking about managing the changes, you’ll become enlightened and happy and realize that this is the way version control was meant to work.

I know, it’s strange... since 1972 everyone was thinking that we were manipulating versions, but, it turned out, surprisingly, that thinking about the changes themselves as first class solved a very important problem: the problem of merging branched code.

And here is the most important point, indeed, the most important thing that we’ve learned about developer productivity in a decade. It’s so important that it merits a place as the very last opinion piece that I write, so if you only remember one thing, remember this:

When you manage changes instead of managing versions, merging works better, and therefore, you can branch any time your organizational goals require it, because merging back will be a piece of cake.

I can’t tell you how many Subversion users have told me the following story: “We tried to branch our code, and that worked fine. But when it came time to merge back, it was a complete nightmare and we had to practically reapply every change by hand, and we swore never again and we developed a new way of developing software using if statements instead of branches.”

Sometimes they’re even kind of proud of this new, single-trunk invention of theirs. As if it’s a virtue to work around the fact that your version control tool is not doing what it’s meant to do.

With distributed version control, merges are easy and work fine. So you can actually have a stable branch and a development branch, or create long-lived branches for your QA team where they test things before deployment, or you can create short-lived branches to try out new ideas and see how they work.

This is too important to miss out on. This is possibly the biggest advance in software development technology in the ten years I’ve been writing articles here.

Or, to put it another way, I’d go back to C++ before I gave up on Mercurial.

If you are using Subversion, stop it. Just stop. Subversion = Leeches. Mercurial and Git = Antibiotics. We have better technology now.

Because so many people dive into Mercurial without fully understanding the new program model, which can leave them thinking that it’s broken and malicious, I wrote a Mercurial tutorial, HgInit.

Today, when people ask me about that podcast where I dissed DVCS, I tell them that it was just a very carefully planned fake-out of my long time friend and competitor Eric Sink, who makes a non-distributed version control system. Like that time he started selling bug-tracking software, and, to punish him, we sent him a very expensive Fog Creek backpack with a fake form letter that made it look like we were doing so well that expensive backpacks were the standard Christmas gift we were sending every FogBugz customer.

I seem to have run out the clock on this site. It has been an extreme honor to have you reading my essays over the last ten years. I couldn’t ask for a greater group of readers. Whether you’re one of the hundreds of people who volunteered their time to translate articles into over 40 languages, or the 22,894 people who has taken the time to send me an email, or the 50,838 people who subscribed to the email newsletter, or the 2,262,348 people per year who visited the website and read some of the 1067 articles I’ve written, I sincerely thank you for your attention.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

A recent post at PM Hut describes the process of capturing 3-Point estimates for schedule. This is an example of Yogi's quote in action

In theory there is no difference between theory and practice. In practice there is. - Yogi Berra

While it may be appropriate in the classroom to teach about the 3 point estimate process - minimum, most likely, and maximum - it is not appropriate in practice. This does not mean there are not gobs of people out there doing this. It doesn't matter how many people teach this, how often it is said, it's simply not good practice.

Here's why:

• If you ask someone to estimate how long something will take by having them think about the Most Likely duration, they will for the most part actually think about the "average" duration. The most likely duration is the value that appears most often if you were to ask a group of people the same question. The Most Likely is the Mode of the samples of that question. It is the value that occurs most often.
• Now when you ask someone the estimate what the least value the duration might be. Or ask them what the most value the duration might be, you'll get statistically significant different values, depending on the order you ask that question. This understanding has been developed by many organizations ranging from the US Navy to petroleum engineers estimating the contents of gas reserves.

So What's the Point Here?

Do not, I repeat Do Not ask for 3 point estimates of duration and cost. Instead as for "variances" of the probability distribution around the Most Likely number. Worse case ask for variances around the Mean (the average). But care must be taken for the measure of the Mean. Since the mean is a value formed by adding all the possible measures, it itself is subject to variance. Most Likely - the Mode - is a simple counting of the most recurring observed value - it is ordinal.

Why Is This Important?

When you ask for the Most Likely, High and Low, you can get up to 27% estimating bias on the estimates. This bias can be favorable or unfavorable. No matter, it is a bias.

The way to do this for duration and cost is to construct a variance ranking processes. Here's a sample table.

Now there are several important things about this table:

• The risk level classification is not a number, it is a letter. This avoids the natural tendency to do arithmetic on the risk ranking. This is a common failure mode for anyone using the PMI style risk management. Take the probability of occurrence and multiply it by the consequential outcome to get a risk rank. NOT.

Both the probability of occurrence and the consequential outcomes are probability distributions, represented by integral equations. Multiplication is not an operator on integral equations - except in their Laplace Transform representations.

• The descriptions of the risk classification represent the operational aspects of the project for a specific technology or process. This means that the words in the classification column can change depending on the problem domain. For manned space flight a Risk Classification D, may have different words, and certai9ntly have different variance ranges than a risk variance D for an Oracle ERP system roll out.

What we need is a table like this for every major risk category.

A similar table can be built for the consequential outcomes. Then and ONLY then can the risk matrix be constructed.

The final advise is risk ranking in terms of variance should be geometric.

The reason you want to do this is that the separation of differences is not linear, it needs to be geometric. The scale 1, 2, 3, 4, 5 means that the difference between 1 and 2 is 100%. The difference between 4 and 5, is 20%.

Here's a Live Example

Connecting the dots - from NOT doing three point estimates - to doing probabilist impacts for the probability of occurrence and the consequential outcomes is shown in the picture below.

So:

• No three point measures asked to the engineers - only variance estimates for each class of risk
• No arithmetic between occurrence and impact
• Narratives of the measures (in the NASA example above they use numbers - not good - use letters)

I will be sending out the next edition of my newsletter, called Lead Well, this coming Monday. I try to make the content of the newsletter unique to what's on the blog because I know many of you subscribe to both. If you are not currently getting the newsletter, click here to sign up. We do not use your email to spam you, we just send the quarterly newsletter.

This quarter's newsletter is focused on Organizational Agility. Articles will offer a way to look at what agility IS and IS NOT and how it differs from change management and change acceptance. I am shairng a list of questions you can use to assess your organization's agility - and how agile YOU are as an individual leader and manager. It is a meaty edition of Lead Well and I hope you sign up to receive it.