The Regenerate Web

John Goodpasture has a post on estimates from Fred Brooks. Fred's quote, which I'll repeat for effect, from John's site is

It is very difficult to make a vigorous, plausible, and job-risking defense of an estimate that is derived by no quantitative method, supported by little data, and certified chiefly by the hunches of the managers.

In many domains, estimating is performed in ways similar to Fred's description. Here's some phrases found in RFP's from the Defense Department:

• The estimate must be easily traceable from the lowest level at which the Offeror’s estimate was substantiated to the CLIN.
• When using historical data, the Offeror should describe why the system is comparable to the proposed program. This includes a functional and technical comparison explaining the differences as well as similarities between the historical and the proposed system. Also include an explanation of the relationship between the analogous element cost and the total program cost.
• Adjustments to derive the proposal estimate relate to reasons and justification for any adjustments made to programmatic, technical and actual cost data for the historical system. The Offeror should provide the basis and document any adjustments applied to the historical data (e.g., complexity factors and normalization methods) that reflect the characteristics of the proposed system. This includes an audit trail sufficient for the Government to reconstruct the proposed estimate and judge its credibility.

Now these are heavy duty words, probably not applicable outside the procurement of similarly heavy duty weapons systems.

But as John points out through Brooks' quote, to do otherwise is complete nonsense on anything but a trivial project.

So wjy does this continue to be the practice in many areas? Some would say, it's because IT projects are software projects and you really can't estimate software projects. To this I say frankly - BS. The program we work int he defense are predominately software development projects. Software development with emerging requirements, changing requirements, state of the art technology, all the things IT project managers encounter.

My personal opinion is

Project estimating, and project management is hard work. Software development is fun work. Still hard but fun. Project Management is hard work, with almost no fun.

So why work on something hard if it's not fun? Secondly, there is no real downside for missing estimates in the commercial IT world. Oh sure, there is hell to pay, people get fired, the project gets canceled. But the company rarely goes out of business, people never die, launch dates are rarely missed. I'm not talking about silly commercial product launch dates, I'm talking about launch dates to Mars, launch dates for NRO (National Reconnaissance Office) launch dates. Those kind of launch dates.

What's missing is the discipline - at appropriate levels of course - to "do the right thing." That's the root cause of most problems in the commercial world - pure missing discipline.

It's the Micky Rooney school of management. Where Micky and Judy Garland are in the alley behind a NYC theater, when they say to each other and their friends...

Hey guy, I have an idea, let's put on a show

It's that, let's get started and see where this goes, kind of approach to project management. Where it goes many times is straight into the ditch.

I spoke again this year Carnegie Mellon University on the topic of project management in the software development world.

Not matter what method you use to develop the software needed by the project, for success these principles must be in place

• moon bases
• jet packs
• 200 mph cars 

but has no...

• cell phones,
• Internet,
• and no laptop computers

Over the long haul, dramatic things happen to change the equation.

Sunil Paul, quoted in The Atlantic, “Better Luck This Time,” July/August 2009.

Via Wayne Abba's PMI Baltimore Chapter address, September 17, 2009. A former senior contract performance management analyst in the Office of the Secretary of Defense from 1982-99.

Forecasting the impact of the latest gadget, process, or method is very sporty business.

For every complex problem there is an answer that is clear, simple, and wrong. - H. L. Mencken

I'm attending a National Defense Industry Association conference on Program Management. I took a break to read some emails - you can only take so much of Earned Value, probabilistic cost modeling, a DCMA self assessment audit guidelines.

There's a discussion going on about how agile software development methods can be applied to safety critical systems. First I earn my living working on programs that contain safety critical system. As well I designed and deployed a safety critical system - the Tricon a classic 80's product name.

The discussion from the agile point of view is typical agile - we've got this really cool way of doing things, you can use it on your problem too.

In the absence of a domain and context, probably not. I spoke for two nights this week at Carnegie Mellon University, San Jose on the 5 immutable principles of project management. My co-speaker is a senior manager at Pay Pal, which make very good use of Scrum in the development and maintenance of its code base.

One of the learning from that series of lectures is that a software development method needs to have a domain and context for it to be effective. If we say a development method is valuable in the absence of that domain and context, we are following Mencken axiom

There is one of those semi-heated discussions on an agile forum around CMMI, Agile and the confusion between them. Here's a summary from CMMI DEV V1.2. Note that the software development activities live in Engineering.

An Update

I'm presenting to a graduate class at Carnegie Mellon West Monday and Tuesday of this week. As well I'm in a loop discussion on an agile forum about CMMI and agile.

It's breathtaking how many people confuse CMMI with a software development method. Much in the same way there is confusion between PMBOK and a project management method.

The chart above - if you in fact take CMMI as a framework for software based product development maturity assessment - shows where "development" activities live and where other activities live. CMMI says you need all these process areas to increase the probability of success.

So agile software development provides methods to fulfill some of these process areas. Specifically the one in the Engineering Process Group. But there are process area where agile has little our nothing to say.

So Agile is not project management in the sense used by those defining the processes of project management. If you redefine the process needed to increase the probability of success of the software project, then maybe you can call agile project management.

"Things are managed; people are led" - Capt. Grace Hopper

Thanks to John Goodpasture for the reminder. I once had the privilege of "trying" to carry Captin Hopper's bag at the Orange County airport for an ACM meeting she was speaking at. She wouldn't let anyone "give her a hand."

While responding to a post on Herding Cats, I came across a $75 Outlook based PM tool MissingLink Project Management

So minus the software interpreted process of Andrew's tool, what's the difference here? Outlook is ubiquitous, possibly on every desktop and cell phone in some form. It's highly configurable and like Andrew is fond of saying "as easy as email." It is email.

We get brilliant results from average people managing brilliant systems. Our competitors get average results from brilliant people working around broken systems

– Fujio Cho, Chairman Toyota Motors

People are needed. But the "system" is the critical success factor. Tools implement the "system," but tools must support the processes defined by the "system." If not the tools are worthless and the people cannot perform their job, so the whole thing falls apart.

A smooth sea never made a skilled mariner - English Proverb

Experience comes from "doing." Doing in difficult situations, with difficult projects builds experience that can be applied to the next project which has difficult situations.

Dan Ward - a favorite Blogger and Author in Defense AT&L - has a nice little post about the principles of the US Navy Seals.

• Humans are more important than hardware
• Quality is better than quantity
• Special Operations Forces cannot be mass produced

Connection with Project Management and Project Controls

I'm home for awhile from a 9 week stint on a large US Army program, rebaselining the next increment of the contract.

• When we don't have the right people in the right roles all the technology - planning and cost tools - provide little value to our efforts. The Control Account Managers needed to be "really good" to get their Budget Change Requests in shape for baselining within the allocated time period. No tool is going to fix a poorly formed BCR and Work Package flow.
• Having all the numbers line up and the Work Package duration be "credible" in the right sequence to maintain the needed schedule margin and cost margin is a critical success factor. This margin is calculated using Monte Carlo simulation, per DID 81650.
• The core cadre of PP&C specialist is rare these days. The training, experience, and skills needed to pull the Performance Measurement Baseline together does not come from books. It comes for Training, Skills, and Experience. Hand On "doing" of the work of PP&C. Same for being a program manager. Being a Control Account Manager is not learned from a book either.

What this means is:

• Tools are 3rd place every time
• People and Process are synergistic. You have have good people and a poor process and be in the ditch. You can have a good process and poor people and be in the same ditch.
• Good people can over come poor process b y making up their own processes, but that process is not scalable.
• Good people and a good process can over come the lack of tools for awhile, but cannot be sustained.

We us a quote around tools and process.

Did it once, and I'll do it by hand. Do it 2 to 3 times and I'll write a VBA macro. Do it 50 times and I buy a "real" tool

In all case though it starts with competent, trained, skilled, experienced, and most of all motivated people.

All success flows from there.

For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled
- Richard P. Feynman

When we hear about the latest "gadget" or "process" that is replacing the current "gadget" or "process," in the project management world, let's stop and ask does this "next new thing" provide new or more informative answers to these questions:

1. Where Are We Going?
2. How Do We Get There?
3. Do We Have Enough Time, Resources, And Money To Get There?
4. What Impediments Will We Encounter Along The Way?
5. How Do We Know We Are Making Progress?

The amount of "restating the obvious" in project management literature, blogs, and commercial web sites is growing. PMI is part of this as are the numerous commercial sites, mostly aimed at managing IT projects.

I'm preparing an abstract for the upcoming EVM World 2010. My session editor is a principle at a Washington DC consulting firm, specializing in EV and Cost management. The leaders of that firm are household names in the cost analysis and forecasting business fro NASA and DoD.

My editor is very strict in pushing me to move beyond the obvious. My topic of "getting back to GREEN" is a critical activity on every aerospace and defense program we work. If everything was going right, they would not need to hire us. Rarely does everything go right of course.

Don't Restate the Obvious

We all know people skills are needed. We all know we need a plan and a schedule. We all know we must have a risk management plan integrated with the schedule. Blah, blah, blah.

These are all necessary but not sufficient conditions for increasing the Probability of Program Success (PoPS). But these are rarely executed properly.

What's the reason for project's being "off GREEN?" Here's my opinion of the bowels of a large DoD contract where I'm working the re-baselining efforts.

1. We don't think about project management from the system engineering point of view. Program Architecture is equally as important as Technical Architecture. The successful programs have a programmatic process flow that describes how the technical components will increase in maturity, how risk will be retired, and how the program as a "system" will recognize that progress and being made.
2. We are missing the direct accountability for Work Package performance as a group. We have weekly Earned Value, we have weekly and sometimes daily stand ups, we have every tool know to man for communicating, visualizing, testing, developing, and doing anything else to the product we need to do. But they rarely operate as a "system of systems." In the successful programs (that don't need us), all these elements are integrated. Sometimes seamlessly, sometimes the stitches are BIG along the seams. More Frankenstein like than plastic surgeon like.
3. We fail to predefine measures of progress are needed to recognize physical percent complete is being reached. The post hoc assessment of progress is the "kiss of death" for any non-trivial program. "How will I recognize success when it walks through the door?" The plan has to tell me this or I won't recognize it.

“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.”

?  Mark Twain

It has become appallingly obvious that our technology has exceeded our humanity
?  Albert Einstein