The Regenerate Web

There are unlimited opinions floating around about the role of management. Many come from voices that would have management be removed from the process - the extreme agile point of view.

For those looking to understand how management has and is evolving in the "post industrial" age, here's a nice article A NEW ROLE FOR MANAGEMENT IN TODAY’S POST-INDUSTRIAL ORGANIZATION

The Ivey Business Journal is free Business Journal from the Ivey School of Business, University of Western Ontario.

- Bilal M. Ayyub, Peter G. Prassinos, and John Etherton

PMI has moved to Virtual Communities, away for the standalone organizations of the past. What ever their reasoning, it doesn't matter. I voted against the VC move for College of Performance Management, but we went there anyway.

Now that VC's are starting, I', signing up for their Blogs.

This process is of course broken and lame. No Goggle Reader link, when you navigate to the RSS feed page, and copy the URL to Google Reader it can't find a "read" feed, but provides a "watch" process. This is new for Google Reader and applicable to those sites that are clueless about RSS and Blog Feeds.

Experience with the VC To date - not impressed.

I had the pleasure of speaking at the Fort Worth Texas PMI Chapter meeting and two sessions of the PMI Symposium this June.

Here's the material

Fort Worth Chapter Meeting

Symposium: Immutable Principles of Project Management

Symposium: Establishing the Performance Measurement Baseline

Ben Snyder has several posts on PM Hut about the demise of the "Iron Triangle" in PMBOK® Version. For some reason PMI decided to remove this concept, just when those of us in the defense and space business are starting to use the phrase "Ponzi Scheme."

Just as a reminder, the three invariant variables in any project are, Cost, Schedule, and Technical Performance. Technical Performance includes the quality measures from the previous version of PMBOK®. Technical Performance Measures describe the behavior of the product or service in units meaningful to the buyer.

The relationship between these three variables is coupled, in likely non-linear ways, but they are coupled and this coupling is inseparable. There have been those that suggest this coupling can be broken. It can't.

Just so we don't forget - since PMI removed the reminder - making trades between Cost, Schedule, and Technical Performance and NOT expecting unfavorable impacts is a Ponzi Scheme. Treat anyone suggesting those trades can be made with no impact in the same way you'd treat Charles. Run Away.

There are no points for predicting rain, only building lifeboats

-Warren Buffet’s Noah Principle 

When we hear about the "principles" of project management, good principles for sure, we need to ask the next question how can we cause actionable outcomes

When someone tells me they have a schedule, they have a budget, they've got a set of requirements. One starting response is:

Are these schedules, budgets, and requirements credible?

What are the units of measure of credible? How would we recognize credible if it walked in the door?

Let's Look At The Attributes of a Credible Schedule

The National Defense Industry Association (NDIA) has quarterly meetings of the Industrial Committee for Program Management (ICPM). These meetings are a gathering of the industry and government program management community.

During the February 2010 meetings there was a presentation of the Program Planning and Scheduling Subcommittee (PPSS) Initiative, by Ms Lil Vayhinger and Ms Rebecca Davies. One slide in a presentation listed essential elements of the "generally accepted" scheduling principles. These are:

A valid schedule provides reasonable and credible information based on realistic logic, durations, and dates. This information is:

• Complete: The schedule captures the entire discrete, authorized project effort from start through completion.

• Traceable: The schedule logic is horizontally and vertically integrated with cross-references to key documents & tools. 

• Transparent: The schedule provides visibility to assure it is complete, traceable, has documented assumptions, and provides full disclosure of program status and forecast. 

• Statused: The schedule has accurate progress through the status date. 

• Predictive: The schedule provides meaningful critical paths and accurate forecasts for remaining work through program completion.

An effective schedule is useful, helps align time-phased resources, and is built and maintained using a controlled process. This  information is:

• Usable: The schedule is an indispensable tool for timely and effective management decisions and  actions. 

• Resourced: The schedule aligns with actual and projected resource availability.

• Controlled: The schedule is built, baselined, and maintained using a stable, repeatable, & documented process. 

These attributes are the basis of a credible schedule and the costs associated with the work delivered by the work elements of the schedule. They are independent of the domain or context of the project. They are independent of the size of the project or program. They are the basis of a "credible" schedule and therefore a "credible" project.

If you have a project that does not have these attributes, you're likely in trouble and you may not know it.

The discussion of estimating - cost and schedule - has produced references usable in a variety of domains. As a background here's the guidance in the domain I work.

This "mandates" an Estimate At Completion with a Best Case, Worst Case, Most Likely. As well a Schedule Risk Analysis (SRA) is mandated.

And just for clarification for those favoring "hand built" 3 point estimates, the 3 points estimates stated in DID-81560 above are generated using the "ordinal range" method. The outcome from this processes are the percent ranges - upper and lower - of the probability distribution (Triangle) used for the Monte Carlo tool (Risk+ and @Risk for Project). So we have the three points, but they were not gathered by asking the engineers what they should be. The Systems Engineering, Control Account Managers, and Technical Leads, plus historical data, plus some statistical analysis produced the ordinal ranges for 5 classes of programmatic risk. That is then applied to each class of work, and then loaded into the baselined Integrated Master Schedule.

The point of all this is that producing credible estimates is part of the culture of the work we do. It's still just as hard, just as complex, and just as annoying as any other domain. But doing it, and doing it right is what is expected. It's part of being credible. So for those gathering variance ranges by hand, if the credibility - statistical confidence, auto-correlation, assessment of all the biases and influences discussed in The Psychology of Risk and Sources of Estimating Error can be addressed, then you're on the path to credibility.

Eisenhower had to deal with as

"fractious and dysfunctional a group of egomaniacs as any war had ever seen."

Ike's greatest achievement was keeping the allied command focused on killing the Nazi's not each other.

- From a speech delivered by Secretary of Defense, Robert M. Gates, Abilene, KS, Saturday May 08, 2010.

Mike Clayton pointed me to a post on the The Psychology of Risk. The book in his post Risk: The Science and Politics of Fear, looks like the next reading assignment after our book club finishes Against The Gods.

Mike's post lists four biases for developed by Tversky, Kahnemann, and Slovic.

This book, Mike's post, and the materials in the previous post on Sources of Estimating Error, should put an end to the naive and simple minded approach to gathering estimates on projects.

Please, please, please do not apply three point estimates to anything without first having read the materials referenced here and the Tversky and Kahnemann materials. You will simply be setting the stage for disappointment all around.

I was surfing some presentations looking for good layout ideas, when I came across a Jazz - A Metaphor for Agile Management. I'm not a real jazz fan, but I do enjoy certain artist. Sitting in the Chautauqua,  performance hall last week for the Brahmas 1st Piano Concerto and thinking about Jazz as a management metaphor, I'm struck by several things:

• How many players can you have in the performance before you need a conductor?
• If I want to perform the 1st Piano Concerto, can it be done without a conductor, individual section music?
• Of course the pianist, Peter Serkin, played without  music, but he "knows" the piece inside outside. The orchestra struggled a bit since they had only rehearsed for two days.
• When you listen to the penultimate performance, I would guess it was rehearsed many dozens of times, Ashkenazy, and Haitink studied the piece and the orchestration for months if not years, and were both at the peak of their performance capabilities before laying down the tracks. 

Finally:

• When the metaphor is applied to software development, are there people (the customer) willing to pay others (the developers) to improvise in the way some Jazz groups do?
• At what point does "spending other peoples money" need oversight in the same way the orchestra needs a conductor.
• What are the limits to the size and complexity of the music or the software that requires external oversight - project management or a conductor.
• What is the customer doesn't like Jazz?

Starting with Tversky & Kahnemann (1974), "Judgment under Uncertainty: heuristics and Biases," there are sources of error that should lay the foundation for abandoning the simple and many times naive 3 point estimate approach to work duration and effort.

• Representativeness - occurs where probabilities are evaluated by the degree to which A is representative of B.

The last time we did this, we say durations that looked like this. Or, We've asked the guys down Florida and they think we should be able to get this done in 1,400 hours, that what it took them the last time they did it.

• Insensitivity to prior probability of outcomes - we ignore the past statistical behavior of our work efforts. This is ignorance of Bayesian statistics.

I know they took 1,400 hours, but we've learned a lot from their mistakes, so we can do it for less. We don't really have evidence, but our gut feel is they were not very good at their job and took too long.

• Insensitivity to sample size - this is a very common error, where the estimator makes a forecast independent of the sample size.

My students - all 23 of them - answered a survey I wrote about a behavior of problems with Earned Value, and they came up with answers I'll apply to a broad set of situations. I sampled the people I know and they came up some interested statements of what works and doesn't work. We hand picked 12 managers in our firm, gave them a survey we put together from some outside suggestions and we'll be making changes based on their answers.

• Misconceptions of Chance - the belief that random processes represent the core behaviors of a process, while observing a short sequence of outcomes.

I've seen this happen 3 or 4 times in other places, it's got to be the same thing happening here. It's happening in all kinds of places, so it's got to be the same here. "For every up there is a down," let's just wait and we'll be back on track pretty soon.

• Insensitivity of predictability - the future can be predicted intuitive predictions

I've seen this happen before, it's got to be the cause of what's happening this time too.

• Illusion of Validity - the observations we're seeing are a good fit with what we're expecting.

We interviewed a list of people selected by management, and they support the what management thinks is going wrong here. Management has a target budget and schedule in mind, and when we asked the developers - after hearing the management numbers - they came up with about the same numbers.

• Misconceptions of Regression - all those random processes add up to a average we can live with.

When we get all the estimates in, we can find the average variance and use that to forecast the work we're going to be given in the future. This concept is actually mis-represented in a government cost estimating guidebook. Adding a sample set of distributions results in a "normal" distribution.

• Biases due to the effectiveness of a search set -when asked to recall some data or event, the one they come up with is what is most familiar to them.

• Biases due to the retrievability of instances - the data I have at hand is the data I used for my forecast.

We had all the data from the projects they did in Dallas, and used that as our sample data for foresting our performance.

• Adjustment and Anchoring - the estimate is based on the starting value and is then adjusted to get the final answer.

Management gave us a starting point for the cost and schedule, so we need to improve that estimate.

• Biases in the evaluation of conjunctive and disjunctive event - in conjunctive events probabilities are over estimated.

Our rule of thumb has served us well in the past - turns up to be biased

• Anchoring in the assessment of subjective probability distribution - confidence intervals are over confident.

These topics are from Micheal Axelsen's review of of Tversky & Kahnemann 1974 paper, "Judgment under uncertainty: heuristics and biases."

These topics will hopefully put an end to the simple minded 3-point estimates extracted from the people tasked with doing the work. They certainty have a contribution to the estimating process. But serious problems arise when you take those numbers and start making management decisions. In exactly the same way you create serious problems when you take management directive and start makes estimates "anchored" on the bounds of cost or schedule they what to have you produce to.

The standard approach to project taskl duration estimating is the PMBOK® 3 point estimate. Josh has provided several approaches to the discussion of the issues.The top two issues are:

• The anchoring and adjusting error occur when humans make estimates in the presence of uncertainty.
• Failure to understand the underlying statistical distributions produce unfavorable confidence in the estimates

Stochastic process models of task durations appear in other domains - shop floor scheduling, interval estimates, and constrained resource management - are examples. Bayesian networks are another. These paradigms offer significant improvement in the confidence of the estimates needed for credible schedule and cost estimates for projects in all domains.

The issues with the traditional - and many times naive - approaches include:

• Unrealistic assumptions of probabilistic independence.
• Failure to take into account the "estimating in the presence of uncertainty" described in the anchoring and adjustment research.
• The stochastic nature of all project work, that is ignored in non-statistical estimating processes.
• Failure to assess the probabilistic critical path

It is Time to Move Forward

For a project to be successful it must balance schedule, cost, and the technical performance measures. The influence on these three elements are probabilistic rather than deterministic. The outcomes from these elements and their relationships are a function of these probabilistic distributions.

Because of this, we must move forward in our understanding of the behaviors of the project...

• No more single point estimates.
• No three point estimates without acknowledgment of their limitations.
• No naive definition of the critical path.
• No more ignoring the complexities of schedule and cost.
• No using simple minded process for complex projects

He who can handle the quickest rate of change survives - Col John Boyd USAF (Ret)

Boyd is the author of the Observation, Orientation, Decision, Action loop.

The OODA loop is a model for decision making that combines theoretical constructs from biology, physics, mathematics, and thermodynamics applied to air combat. OODA is applicable is any domain where rapid adaption to an emerging situation is needed for survival.

There are four process areas (from "Performance Based Leadership Preparation I - American University, Pat Barker instructor, Senior Program Manager Certificate Program, US DoD):

1. Take information pushed to you, or information you pull from the environment.
2. Comprehend what that information means
3. Decide what to do
4. Do it

I'm headed to Fort Worth to speak at the PMI Chapter meeting and conduct a workshop. The topics start with the Immutable Principles of Project Management. Many authors have their lists of principles and practices. Here's what has emerged over the decades for me.

The five immutable principles of project management are:

1. Know where you are going by defining “done” at some point in the future. This point may be far in the future – months or years from now. Or closer in the future days or weeks from now.
2. Have some kind of plan to get to where you are going. This plan can be simple or it can be complex. The fidelity of the plan depends on the tolerance for risk by the users of the plan.
3. Understand the resources needed to execute the plan. How much time and money is needed to reach the destination. This can be fixed or it can be variable.
4. Identify the impediments to progress along the way to the destination. Have some means of removing, avoiding, or ignoring these impediments.
5. Have some way to measure your planned progress, not just your progress. Progress to Plan must be measured in units of physical percent complete.

Josh from PMStudent.com has a post about estimating. There is a discussion of using 3 point estimates.

Some time ago, I came to the conclusion that making 3 point estimates is a bad idea. Instead use confidence intervals for risk ranges and drive the variance from those bands with a Monte Carlo Simulator.

This is a critical topic that goes to the credibility of any estimate no matter the domain.

The starting point is to understand the concept of anchoring and adjustments to the estimate and their impact on false optimism (or pessimism). The paper to start with is "Anchoring and Adjustment in Software Estimation," Aranda and Easterbrook.

When you read this, you may rethink the approach that asks the estimator the Most Likely estimate and Optimistic and Pessimistic estimates.

The original work in this area is from Tversky and Kahneman and their "Prospect Theory." A good place to look for some details (besides their work) is Against The Gods, Peter Bernstein, in Chapter 16, The Failure of Invariance."

It's time to move on from the classic "simple" approach found in places like PMBOK(r) to the processes used in other places that depend on probabilistic and statistical analysis - engineering, finance, medicine, physics.

Here's the binders for the EADS proposal for the KC-X tanker. The award is worth $35B over its period of performance. Photo source, The DEW Line

This is probably the largest award in recent years - no matter who wins (Boeing or EADS). Somewhere in those 17 volumes and 8,000 pages is the IMP/IMS and the Basis of Estimate content.

This concept is almost always missing in enterprise IT. Outside of the technical solution, the proposal needs to answer - How is the program going to be executed? What is the confidence that the plans, schedules, costs, and technical performance will turn out in the way the customer wants?

We're back to the core questions that must be answered for ANY project, no matter the method, the domain, context, or even the size:

• How much will this cost?
• When will we be done?
• How can we tell we're making the progress as we planned to make?
• What's going to be the impediments along the way and how are we going to handle them?

From experience on Crew Exploration Vehicle, the writing of the proposal was a $10M project in itself.

Can you say - Program Planning and Controls?

The most successful method of "project controls" is Earned Value Management (EVM). There are 32 criteria for a fully compliance Earned Value System in the government contracting world. But these 32 criteria are not needed for success when applying Earned Value outside a formal contracting environment. For government contracts greater than $20M (that is pretty low in the government world), EV can be just as effective without all the formality.

Here's the 10 activities you need.

Let's look at each one and pretend we're not speaking about Earned Value, just good project management performance measurement activities. And let's look at these from a general product development point of view. From Scrum to a PMBOK process group and knowledge area.

Here are the numbered items from the 32 Criteria that should be present in any project performance management system independent from calling it Earned Value. This list has been worked on on the LinkedIn Earned Value item, led by Wayne Abba and other colleagues.

Mr. Abba led the EVM initiatives during his public service career in the Department of Defense and continues to lead through his membership in the National Defense Industrial Association and the Project Management Institute’s College of Performance Management.

• 1. Define authorized work elements - we need to know what work we are authorized to do. It's the customer's money and we must spend it wisely. This can be a pile of sticky notes on the wall, that are moved from one column to another in the agreed order. Or it can be a full blown Integrated Master Schedule, with Work Packages authorized to be released for execution.

• 2. Identify project organizational structure - who's doing what during the execution of the project?

• 5. Provide integrated planning, scheduling, budgeting, work authorization, and cost accumulation processes - you got to keep all the moving parts connected in some way.

• 6. Schedule the authorized work in a sequential manner that identifies the significant task dependencies - if I do work out of order is confuses people. This is the fundamental principle of "value flow." Do the work in the order that creates the highest value to the project, business, or user.

• 7. Identify physical products and organizations - show me what you're building by building it and showing me the outcomes. The physical, tangible evidence that you did what you said you were going to do.

• 8. Establish and maintain time–phased budget baseline - who's paying for this? How much money do we need next week to keep working? How much is this going to cost when we're done? Is there enough in the bank to complete the project. Questions like this and their answers are asked on any project.

• 16. Record direct costs consistently in a formal manner - even if we're spending at a fixed rate (level of effort), we need to know the rate at which we're spending.

• 23. Periodically generate project metrics - stop, take a break, look at the progress that has been made. Is it at a rate that we'll be happy with? Is it at a rate that will get us to the end at the expected time?

• 25. Develop revised cost estimates–at–completion based on performance to date - if not, then what is the rate we need to run at to get there on time? If we can't change that, what time well we be done? What will it cost when we're done?

• 28. Incorporate authorized changes in a timely manner- when we revise all this information, keep track of these revisions, so we aren't confused in the end when someone asked "why is this different from what I thought is was?"

“Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them.” - Laurence J. Peter

When there is mention of the complexity of something like Earned Value, it makes me think about the context of project where EV is most appropriate:

• ERP
• Enterprise IT Integration
• Large Construction
• The obvious defense and space programs, with software intensive components
• Telecommunications systems
• Wireless infrastructure software systems
• Embedded software systems

So when it's mentioned that things like EV and probabilistic risk, cost modeling, technical risk modeling, etc. are heavy on math, I'm wondering what those developers think about the tons of code that is being developed? Is that "too heavy" to  get their mind around.

Wicked Problem Domain

Most of the software systems we work -  majority are defense and space but also enterprise class IT - can be classified as wicked in some way. Horst Rittel defined a wicked problem as:

• You don't understand the problem, until you've developed the solution.
• Wicked problems has no stopping rule
• Solutions to wicked problems are not right or wrong
• Every wicked problem is essentially unique and novel
• Wicked problems have no alternative solution
• Every solution to a wicked problem is a "one shot"  solution

In my career experience, most the work has been on wicked problems, or something close to that.

• Radar systems
• Fault tolerance process control
• Embedded real time control (flight, turbines, nuclear emergency shutdown)
• Engineering design systems
• Programmatic controls for wicked problems

Earned Value and its value to NON-Wicked Problems?

So I'd be interested to hear a bit about non-wicked style?

Is value added to non-wicked problems by processes like EV?

If not, then maybe those challenging EV have a point. EV is not applicable to their problem.