When there is a discussion around PERT, the Central Limit Theorem, estimating durations, efforts, and costs, or any probabilistic or statistical aspect of a project, there is usually a fundamental flaw in the thought process.

When we estimate, we are ALWAYS subject to anchoring and adjustments. It is built into our nature. No matter how well you rationalize the processes for capturing estimates, you cannot avoid anchoring and adjusting. Recognition and correction for anchoring and adjusting is the only way out. The estimates are still biased, they are now recognized as biased.

The core of the Kahneman and Tversky thesis is the contention that people frequently form estimates by starting with a given, easily available reference value — which could be arbitrary — and adjusting from that value. An estimate, therefore, would be "anchored" to that value.

Most estimators are in fact "irrational beings" in the sense of risk taking, bounding of outcomes, and their deviations for the "norm," what ever that "norm" is. This is the basis of "models," calibrated models.

This takes us back to core issue - How do we construct a credible Performance Measurement Baseline? If you're not using estimating models, built and validated by the subject matter experts, and calibrated against past performance, then the credibility question is in question.

I’m still on vacation but am at least online after two offline weeks in Crete. The photo is by my husband, Hakan, b t w.

For me, reading is essential to the ultimate vacation, and this is no exception. I’ve tried to stay away from software development literature, but I was not totally successful. When you’re working with something, I guess every book is somehow related to the subject. So, my head is packed with new interesting topics for this coming fall. And I’m starting off with the subject of risk. Is a risky project/task worth it?

A basic principle of agile is to try to get as much business value for the least resources. This would mean that you would pick story A of the following user stories:

Story Cost Business value A 10 10 B 20 10 C 10 5

But what happens if you take risk into consideration?

Story Cost Business value Risk A 10 10 10 B 10 20 20 C 10 5 30

What would you pick? A safer story with less business value or a riskier one with higher potential?

My basic instinct tells me to pick story A, and I’ve also felt somehow that the agile principles are backing me up on this. But would happen if we always pick the less risky stuff?

In Blue Ocean Strategy, W C Kim and R Maubourgne describes the differences between a red ocean and and a blue ocean when it comes to competitive situations and businesses. The principle is described very well in the current (July 2010) Wikipedia article, so feel free to take a detour if you prefer before going back here. To summarize; the red ocean is a business where the competition is dense and you need to fight for your position. If you have found a blue ocean, you’re competitors cannot truly compete with you. Of course, most blue oceans are temporary, but the successful organizations find new blue oceans when their current ones turn red. But what has this to do with my product backlog?

The answer is the question, in some ways, but you could also call it Innovation. How can you find a blue ocean? Well, it’s probably not by deselecting all the high risk items, is it?

David Andersson also points to this in his brilliant Agile Management for Software Engineering. If you’re shy of innovation, you will never be able to implement lean and agile values in a long term successful way. Implementing The Theory Of Constraints require a successful selection of both risky and non risky items. He also points at the importance of measuring the right stuff when it comes to high risk things: if you only measure the factual outcome, people will become shy of innovation and only pick the sure wins. This is probably the best way to stay in a red ocean. (And if you have not yet gotten this book, be sure to read it.)

But what are risky projects, items, tasks? It is very important to know why they are risky? Is it a risky technology? Are we unsure of the potential income? Are there special circumstances in the current project? David Andersson points at the importance of knowing WHY something is risky, but not deselecting for example all tasks with high technological risks. I’ve for example worked with a client who only worked with old versions of tools in order to lower the risk. Yes, the risks were lower but he lost out on innovation. But I’ve also worked with clients who ran for all new technology and had to be one of the first on all new versions. This is of course not good either. Now we’re not talking about risky user stories: we’re talking about misplaced focus. We won’t find the blue oceans because of new technology, but it can help us get there.

So, when my vacation is over in about two weeks, I will look differently at risk. Is there a potential blue ocean hiding there or can I get over another constraint while handling it? But now I’m heading back to my vacation.

“Human knowledge has been changing from the word go and people in certain respects behave more rationally than they did when they didn’t have it. They spend less time doing rain dances and more time seeding clouds.” Herbert Simon Cloud computing – that server in the sky – has gotten a lot of press recently. [...]

I have some major changes ahead of me, which will probably have an impact on this blog.

Medium time scale
In just two days, on August 1, I will send the final manuscript of my book to my publisher. That doesn’t mean I’m done with the book. It only means the writing will be replaced by marketing. But I look forward to that. It’s another creative project, and a new opportunity to learn while doing something useful. And you know me, I love self-promotion.

Long time scale
Yesterday, I told my boss that I want to leave ISM eCompany by the end of the year. I had a great time these past seven years. But with the launch of my book I intend to initiate a new career of full-time writing, speaking, and training. And self-promotion. Again, it will be a road full of learning opportunities. It’s probably a painful road, but definitely worth traveling.

Short time scale
But first, I will need to prepare for the Agile 2010 conference in Orlando, where I will be presenting/hosting one session. Self-promotion for that will commence in about three days. Immediately after the conference Raoul and I intend to enjoy a brief but well-deserved vacation in Norway, with all channels switched off. Except for the verbal ones.

These are the three big changes I’m facing.
It’s so exciting, it’s almost scary.

Don’t expect too much from me on this blog in August. My self-promotion will be back, with a vengeance, in September!

(image by Capture Queen)

 Twitter -  Subscribe - Newsletter - LinkedIn - SlideShare

tweetmeme_url = 'http://www.noop.nl'; tweetmeme_source = 'jurgenappelo'; Latest, greatest and favoritest posts:
Make a “Social Contract” with Your Team
The Open Space Policy for Managers
Agile (Wrongfully) Assumes Craftsmanship

Anatomy of an Effective Project Manager By Gary Hamilton, Gareth Byatt, and Jeff Hodgkinson It’s first thing in the morning, and you are preparing to interview prospective project managers for an open position on your team. Whether it is your first candidate interview or you have conducted many before in your career, you are likely to be [...]

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my tenth and final reason for using Agile is that Agile addresses the realities of software development and the needs of the business. This post will expand upon this topic.

One reality of software development is that there is still too much unsustainable development in the industry as a whole. Squeezed projects with significant overtime and stress prevent us from getting smarter, let alone working smarter. Technology and business are continuing to change at ever-increasing rates; keeping pace with the business while keeping our skills current and relevant is a major challenge, yet essential if we are going to improve productivity. Through sustainable development and continuous improvement, Agile development helps teams and individuals do both.

A few tangible benefits associated with Agile development that meet the realities of both software development and the business – contrasted against traditional, plan-driven development projects – can be categorized broadly as: The early bird gets the worm.

Software project teams need feedback about features from the users. No matter what documentation was prepared in advance of development, there is nothing like having the users seeing the working software to obtain final, definitive agreement on whether the software meets their needs. Signing off on a specification and “building to the spec” might meet a project measurement, but it won’t necessarily satisfy the customer.

Why? In my experience, I’ve found that users aren’t particularly good at dealing in the abstract, or at least they aren’t as good at dealing in the abstract as software developers. They need to see things, not conceptualize them. That, coupled with the fact that communication between individuals is always subject to misinterpretation at some level, leads to the ever-present “requirements problems.”

I’ve heard the phrase, “Now that I see it…” quite often over the years, followed by a change request. Agile development’s frequent delivery of features enables users to routinely inspect the (working) software to provide immediate feedback. And because the highest-priority features are delivered first, software project teams get early feedback on the most valuable business features, allowing for course-corrections to occur early, instead of late, in the project.

Because the highest-priority features are being worked to completion early, the business can benefit from an early return on investment. With traditional, plan-driven projects, software is delivered as a complete package, but the business must wait until the end of the project to begin realizing a benefit.

Another, related benefit to using Agile development is that valuable time is not spent on the details of features that may or may not be a priority later. With traditional, plan-driven projects, you run the risk of investing time and energy into defining and designing features that may not be desirable or even needed. Time is money, and I prefer not to waste either.

Early delivery, early feedback, and an early return on investment; the early bird (the business) does get the “worm” with Agile development – or is it really a caterpillar that morphs into a butterfly in the cocoon of the Agile team?

Another reality is that business is rapidly changing. As Faisal Hoque points out in an article, The Speed of Business Today, “Constant change is the new dynamic of the global economy, and makes agility even more necessary than at any point in business history.”

Software development should align itself to the needs of the business by being able to respond change, and Agile development does just that. In the 2009 State of Agile Development Survey sponsored by VersionOne, the number one benefit obtained from implementing Agile development was the ability to manage changing priorities. 90 percent of the respondents said implementing Agile either improved or significantly improved their ability to manage changing priorities. (The survey data included 2,570 participants in 88 countries.) An older survey, the Shine Technologies 2003 Survey, was another global survey of actual experiences using Agile development. The highest ranked positive feature reported in the survey was the ability to “Respond to change over plan.”

In terms of general alignment between the business and IT, the 3rd-highest benefit obtained from using Agile development reported in the 2009 State of Agile Development Survey was an Improved Alignment Between IT and Business Objectives. This is supported by the Shine Technologies 2003 Survey, which reported that 83 percent of those surveyed stated that business satisfaction was better or significantly better.

Satisfaction can be a function of a variety of factors, but one reality is that software is being designed and built for the customer. People like to feel involved, that their needs are paramount in your mind. Traditional, plan-driven projects “respond” to the customer, but usually with change control and a “here’s how the project will be impacted” assessment. A rigid, process-oriented approach isn’t bad, but it can develop into an almost adversarial relationship where the customer feels that they are the one that must press for changes to their software.

This is because traditional, plan-driven projects have a focus on “planning the work and working the plan” where delivering to the specification is the key measurement. While there are project managers who run their projects as a series of negotiations (which helps maintain the relationship between the team and the customer), the very nature of the process causes people to lock in on what is defined – and change is becomes disruptive.

Agile development is designed to handle change; people don’t become wedded to a plan because they haven’t invested a lot of time and effort defining, estimating, and designing everything prior to actual construction. Agile teams wait until the last responsible moment to begin diving into the details of any one feature. And features that haven’t been started yet are considered negotiable at all times. This allows the business to change its mind with ease. And because Agile teams work with the customer throughout the project, the customer feels involved and valued throughout the entire process.

Agile development also stresses the need for the use of sound technical practices as a means of improving productivity. While you don’t have to be on an Agile team to make use of these practices, Agile development promotes them. Automated testing, Test-Driven Development, continuous integration are all geared towards improving throughput without sacrificing quality or sleep.

My final point, and final reality, is that software professionals are all adults, and they should be treated as such. Agile development, with its autonomous, self-directed teams allows people to define and manage their own work, which provides a motivational – and productive – boost.

Dan Pink, in his book Drive: The Surprising Truth About What Motivates Us, cited a study conducted by researchers at Cornell University that examined 320 small businesses, half of which granted the workers autonomy, the other half relying on top-down direction. The businesses that offered autonomy grew at four times the rate of the control-oriented firms and had one-third the turnover.

Top-down, control-oriented management brings its share of problems. In his book, The Way We’re Working Isn’t Working, Tony Schwartz puts this into solid perspective. Schwartz states, “The all-too common dynamic is today’s workplace is parent-child. Most employers tell employees when to come to work, when to leave, and how they’re expected to work when they’re at the office. Treated like children, many employees unconsciously adopt the role to which they’ve been consigned. Feeling disempowered and vulnerable, they lose the will and confidence to take real initiative or to think independently. Doing what they’re expected to do often becomes more important than doing what makes most sense.”

Command-and-control management creates the need to manage people more closely! Is that what we want? I know that I don't. Agile development's final contribution to the realities of software development is that it provides the opportunity for everyone to be treated as valued, professional, adults. That works for me!

The above headline seems to be the message in an article in the latest edition of Computer Weekly.: “The article, called “How will suppliers be able to cut IT costs?” starts with the words: “The government has told IT suppliers it wants them to reduce the cost of contracts with government departments.” The paragraph ends with: “But it could be the government that has to change the most.”

Nineteen IT suppliers met the Cabinet Office Minister, Francis Maude to start the process of reducing the cost of contracts.

The article goes on to suggest that ‘red tape’ needs to be addressed and the government must overcome a lack of trust if suppliers are to meet targets without just cutting costs to the bone or stripping service levels.

“The government wants immediate reductions in costs and ongoing cuts” says the article.

Francis Maude said he was challenging major government suppliers to take costs out of contracts. “Some of this will come from margins, but we will invite ideas on how we can structure things differently to reduce complexity and cost.”

There will no doubt be more articles like this as time progresses and for sure, this will impact on many people - anyone involved in IT, project management or procurement plus of course shareholders.

If you are based in the US you also have some ‘issues.’ Shortly before completing this article I came across the headline: “White House to Review High Risk Projects”. This can be found here.

An extract of the Computer Weekly article can be found here.

I dislike giving out immediate answers when someone comes with me to help them solve a difficult problem. Sure, I can give you a quick answer which may or may not be the best possible answer, and possibly may not even be a good answer, but difficult problems deserve plenty of time and numerous thoughts before they should be solved.var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E")); try{var pageTracker = _gat._getTracker("UA-xxxxxx-x");pageTracker._trackPageview();} catch(err) {} var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));try {var pageTracker = _gat._getTracker("UA-4191072-4");pageTracker._trackPageview();} catch(err) {}

Albert Einstein's general relativity was developed from 1907 to 1915, not in the blink of an eye. Yes, it is unlikely that any of us will ever encounter any problem which would require thinking of that deep and sustained nature, but that does not mean we will not combat and conquer only easily solved problems.
When I need to do my heavy thinking, I go to sleep. More precisely, I lay down in bed and let my mind take a meandering path through the problem space. I uncouple the 'obvious' linkages within the problem and see what happens when I couple items which would normally make an unimaginable combination. Its during these times of free association that I regularly hit upon a solution to my most difficult issues.
When I read a blog post by Paul Graham about The Top Idea in Your Mind, I instantly understood where he was going in reference to his morning shower. His best thinking is done there, just as mine is done in the minutes (and sometimes hours) between laying down and falling asleep.
As a child, I was raised in a Baptist church. One of my youth ministers claimed that reading the Bible as the first act of the day was vitally important for everyone and is something he wanted all the youth to do every day. As a means of 'encouragement', he once set up a phone tree so that the youth such as myself who were NOT morning people would be called every 2 minutes by the youth who were morning people to remind us to get out of bed.
I bring up that story as a way to show that sometimes even the best of intentions cannot produce quality results. Just like I could never do quality thinking in the morning like Paul Graham does, it is unlikely that he could ever do his deep thinking as he fell asleep in bed.
To give a fair assessment to problems my stakeholders bring, I generally ask them to let me sleep on it first so I can give them the best answer possible. My stakeholders know this about me and I believe they respect me for it, mostly because they keep coming to me for answers.
What about you? Where and when do you do your best thinking?

"volatility per se, be it related to weather, timing of the morning newspaper, is simply a benign statistical probability factor that tells us nothing about risk until it is coupled with a consequence," - Robert H. Jeffrey, "A New Paradigm for Risk," Journal of Portfolio Management, Volume 11, Number 1 (Fall), pp. 33-40.

When we speak about capturing ranges of duration or cost, or speak about compliance with technical performance measures and do not speak about the consequences, then we're pretty much wasting our time with Risk Management.

Here's The Core Problem(s)

• When capturing the estimates using what every method you choose - I'd recommend NOT asking the estimator the Hi, Most Likely, and Lo for lots of reasons discussed else where - more information is needed than just the numbers.
• After eliciting the ranges of values, the probability distributions can then be used to drive a Monte Carlo simulator. This approach produces a Credible forecast of the probability of completing On or Before a Date or having the project cost A Value or Less.
• When static 3 point estimates are used, there can be up to a 27% unfavorable (under estimating) of the completion date. So don't use 3 points and don't use PERT.
• There is a whole cottage industry on why the PERT formulas are bogus and the problems with them. Here's a start "Why PERT Has Problems."

For places to look for PERT background start with:

[1] "Quantitative Risk Analysis for Project Management,: A Critical Review," Lionel Galway, Rand Working Paper, WR-112-RC, February 2004.
[2] "The Polaris System Development: Bureaucratic and Programmatic Success in Government," Harvey M. Sapolsky, Harvard University Press, 1972.
[3] "PERT Completion Times Revisited," Ted Williams, University of Michigan-Flint, School of Management, Working Paper 2005-02, September 2005.
[4] "Hidden Assumptions in Project Management Tools," Dr. Eva Reginier, DRMI Newsletter, January 10, 2006, Naval Postgraduate School, Monterey CA.
[5] "Activity Completion Times in PERT and Scheduling Network Simulation," Dr. Eva Reginier, DRMI Newsletter, April 8, 2005, Naval Postgraduate School, Monterrey CA.

“Project management?  That’s just a lot of useless overhead!”   I remember hearing variations of this a lot years ago.  I don’t hear this much anymore (due at least in some part to the good work at PMI), but I have been seeing something of a variation.

Working with some engineering teams at some big companies, I’ve had conversations with engineers go something like this: “Why do our projects go so much slower here?  When we were a startup things seemed to go much faster — and we didn’t even use project management!”

Upon digging a bit deeper, it turns out that there are a few things to consider.

Some of the slowness was a perception due to having more formal procedures (and the documentation surrounding those procedures).   Some of the slowness was due to pretty bad meeting management — long periods in meetings where nothing pertinent to most of the participants was going on.  Some of these meeting were “needed” because so many different groups of people were involved.

Some of the “slowness” is really due to the fact that the big company project is often creating a product that is more “polished.”  The engineers would admit that in the “old” days they would celebrate success with a product not quite ready for commercial release.

So, some of this “slowness” is about some of the things that come with being bigger.  We can talk about ways some innovative organizations overcome these issues some other time.

Looking at this big company “friction” or overhead still did not seem to account for this slowdown.  Could it be that project management really is an insidious overhead?   Well, all of us can probably remember a few cases of project management gone wrong — too many meetings, analysis paralysis, inadequate or too elaborate change controls, etc.  So, we deal with these items by doing a project management well.

On the other hand, there is something that seems to slow projects down by design — schedules.

How can that be?   Let’s start with those unrealistic schedule first.  You know, the ones that aren’t really set through a rational process of estimation, but rather a target meant to “push” people.  I don’t think we need to dwell on this too much — we end up spending a great deal of effort changing and blaming rather than doing the value adding tasks.

How about those realistic schedules?  They are based on rational estimates and good statistics based reasoning.  It’s not uncommon to have a policy of 90% — that is, give me an estimate with a 90% probability of being met.  This sounds pretty good, yes?

schedule scenario 1

Let’s use this as an example.  A = 12 days, B = 14, C = 1, where each estimate is at  90%.
So, the project = 39 days.  Can we say that we have a reliable schedule?  Maybe.
.
.

schedule scenario 2

One way to look at this is that the 90% is like adding a “buffer” — for example, if each task was a 10 day task at  50%, we add buffer until we get to 90%.

Unfortunately, the way a lot of us build a schedule from this estimation, it is only as good as the last task.  Our schedule would show that task C starts at the end of day 26 and ends at the end of day 39.  If we coordinate based on the schedule, task C will not be able to take advantage of any earlier finishes — this is especially true if there are different teams used in the different tasks. So, in many projects, task might be late, but never early (we don’t even have to get into the psychological impact of “having more time”).

schedule scenario 3

What about that 50% version?  If we use the 10 days each for the tasks, we would get a 30 day project…  a 23% improvement in duration.  Of course, we can’t expect that we will make this schedule.  On the other hand, what is more important — reliability or speed?  If we schedule a 39 day project, what are the odds we’ll finish in 30 days?  In my experience, it’s so close to zero that it is zero…  If we schedule for 30, we may make it very few times, but it’s not zero. We at least have a shot!

Next time, we’ll look at this a bit more — including some additions AND obstacles with this idea.

Technorati Tags: agile, best-practices, Planning

Adapting Pareto Principle For Project Management By Kathlika Thomas There are not many observable instances in our natural surroundings where we find an equal distribution of anything of value. Population, wealth, workplace productivity, even the harvest of a farmer’s plot - there is a concentration of produce generated by the minority that accounts for the majority of [...]

There appears to have been a data center outage early morning, affecting a number of applications including Pivotal Tracker. This has caused connectivity problems for users in some locations, and it appears to still be persisting for some.

We're working with our hosting provider to get this resolved as soon as possible, this is our top priority.

This is the second data center outage this week. At the moment, we do not have enough information to know whether the outages are related.

Also, we have received reports that Tracker has been unreachable from certain parts of the world (including China) since the migration to a new data center last week. We've filed a request with Engine Yard to investigate, and hope to have this resolved soon.

Our apologies for the inconvenience these outages have caused. We'll post more information here as we receive it. You can also follow @pivotaltracker on Twitter for updates.

The Essence of Driving - A Crash Course in Project Management By Ivo Manolov About a year ago, I got feedback from my team that I needed to clarify what I meant by “drive this effort” or “lead that effort”. So I decided to create a quick document explaining what I meant. Below is that document. There are [...]

In episode 16 of Project Potion Dave Prior and Bas de Baar talk about coworking and using video for status reports.

As defined by Wikipedia:

“Coworking is a style of work which involves a shared working environment, sometimes an office yet independent activity. Unlike in a typical office environment, those coworking are usually not employed by the same organization. Typically it is attractive to work-at-home professionals, independent contractors, or people who travel frequently who end up working in relative isolation.”

Click here if you want to subscribe to this podcast with iTunes.

Other people who liked this article liked these too

• Project Managers. On Video? Awesome!This blog and the domain ProjectShrink.com will go their separate ways. No worries. It's a good thin...
• Project Potion: The RecipeDifferent project circumstances require different approaches to ensure optimum effectiveness. As men...
• Potion 12 – iPads and AshcloudsIn this episode of Project Potion Bas talks about his visit to the PMI EMEA Congress in Milan and Da...

Bas de Baar is an independent consultant based in the Netherlands. He uniquely combines over a decade of project and team leadership with nearly a decade online presence in the area of Project Leadership in a global and virtual world.

Potion 16 – The Yeti

There are unlimited opinions floating around about the role of management. Many come from voices that would have management be removed from the process - the extreme agile point of view.

For those looking to understand how management has and is evolving in the "post industrial" age, here's a nice article A NEW ROLE FOR MANAGEMENT IN TODAY’S POST-INDUSTRIAL ORGANIZATION

The Ivey Business Journal is free Business Journal from the Ivey School of Business, University of Western Ontario.

How to Pick a Good Project Sponsor By Peter Taylor Critical to any projects success is having a good project sponsor, but, like the saying goes ‘you can pick your friends but you can’t pick your relatives’ and the same is true of project sponsors. So what makes a good project sponsor and how do you deal with [...]

Introduction

Uncertainty is a fact of organizational life –   managers often have to make decisions based on uncertain or incomplete information. Typically such decisions are based on a mix of intuition, experience and blind guesswork or “gut feel”.  In recent years, probabilistic (or statistical) techniques have entered mainstream organizational practice. These have enabled managers to base their decisions and consequent actions on something more than mere subjective judgement – or so the theory goes.

Much of the statistical analysis in organisational theory and research is based  on the assumption that the variables of interest have a Normal (aka Gaussian) distribution. That is, the probability of a variable taking on a particular value can be reckoned from the familiar  bell-shaped curve.  In a paper entitled Beyond Gaussian averages: redirecting organizational science towards extreme events and power laws, Bill McKelvey and Pierpaolo Andriani, suggest that many (if not most) organizational variables aren’t normally distributed, but are better described by power law or   fat-tailed (aka long-tailed or heavy-tailed) distributions. If correct, this has major consequences for quantitative analysis in many areas of organizational theory and practice. To quote from their paper:

Quantitative management researchers tend to presume Gaussian (normal) distributions with matching statistics – for evidence, study any random sample of their current research. Suppose this premise is mostly wrong. It follows that (1) publication decisions based on Gaussian statistics could be mistaken, and (2) advice to managers could be misguided.

Managers generally assume that their actions will not have extreme outcomes. However, if organisational phenomena exhibit power law behaviour, it is possible that seemingly minor actions could have disproportionate results. It is therefore important to understand how such  extreme outcomes  can come about. This post, based on the aforementioned paper and some of the references therein discusses a couple of general mechanisms via which power laws can  arise in organizational phenomena.

I’ll begin by outlining the main differences between normal and power law distributions, and then present a few social phenomena that display power law behaviour. Following that, I get to my main point – a discussion of general mechanisms that underlie power-law type behaviour in organisational phenomena. I conclude by outlining the implication of power-law phenomena for managerial actions and their (intended) outcomes.

Power laws vs. the Normal distribution

Probabilistic variables that are described by the normal distributions tend to take on values that cluster around the average, with the probability dropping off to zero rapidly on either side of the average. In contrast, for long –tailed distributions, there is a small but significant probability that the variable will take on a value that is very far from the average (what is sometimes called a black swan event).  Long-tailed distributions are often  described by power laws.  In such cases, the probability of variable taking a value is described by a function like   where is called the power law exponent .  A well-known power law distribution in business and marketing theory is the Pareto distribution.  An important characteristic of power law distributions  is that they have infinite variances and unstable means, implying that outliers cannot be ignored and that averages are meaningless.

Power laws in social phenomena

In their paper Mckelvey and Andriani mention a number  of examples of power laws in natural and social phenomena.  Examples of the latter include:

1. The sizes of US firms : the probability that a firm is greater than size N (where N is the number of employees), is inversely proportional to N .
2. The number of criminal acts committed by individuals: the frequency of conviction is a power law function of the ranked number of convictions.
3. Information access on the Web: The access rate of new content on the web decays with time according to a power law.
4. Frequency of family names: Frequency of family names has a power law dependence on family size (number of people with the same family name).

Given the ubiquity of power laws in social phenomena, Mckelvey and Adriani suggest that they may be common in organizational phenomena as well.  If this is so, managerial decisions based on the assumption of normality could be wildly incorrect. In effect, such an assumption treats extreme events as aberrations and ignores them. But extreme events have extreme business implications and hence must be factored in to any sensible analysis.

If power laws are indeed as common as claimed, there must be some common underlying mechanism(s) that give rise to them.  We look at a couple of these in the following sections.

Positive feedback

In a classic paper entitled, The Second Cybernetics: Deviation-Amplifying Mutual Causal Processes, published in 1963, Magoroh Maruyama pointed out that small causes can have disproportionate effects if they are amplified through positive feedback.   Audio feedback is a well known example of this process.  What is, perhaps, less well appreciated is that mutually dependent deviation-amplifying processes can cause qualitative changes in the phenomenon of interest. A classic example is the phenomenon of a run on a bank : as people withdraw money in bulk, the likelihood of bank insolvency increases thus causing more people to make withdrawals. The qualitative change at the end of this positive feedback cycle is, of course, the bank going bust.

Maruyama also draws attention to the fact that the law of causality – that similar causes lead to similar effects – needs to be revised in light of positive feedback effects. To quote from his paper:

A sacred law of causality in the classical philosophy stated that similar conditions produce similar effects. Consequently, dissimilar results were attributed to dissimilar conditions. Many scientific researches were dictated by this philosophy. For example, when a scientist tried to find out why two persons under study were different, he looked for a difference in their environment or in their heredity. It did not occur to him that neither environment nor heredity may be responsible for the difference – He overlooked the possibility that some deviation-amplifying interactional process in their personality and in their environment may have produced the difference.

In the light of the deviation-amplifying mutual causal process, the law of causality is now revised to state that similar conditions may result in dissimilar products. It is important to note that this revision is made without the introduction of indeterminism and probabilism. Deviation-amplifying mutual causal processes are possible even within the deterministic universe, and make the revision of the law of causality even within the determinism. Furthermore, when the deviation-amplifying mutual causal process is combined with indeterminism, here again a revision of a basic law becomes necessary. The revision states:

A small initial deviation, which is within the range of high probability, may develop into a deviation of very low probability or more precisely, into a deviation which is very improbable within the framework of probabilistic unidirectional causality.

The effect of positive feedback can be further amplified if the variable of interest is made up of several interdependent (rather than independent) effects. We’ll look at what this means next.

Interdependence, not independence

Typically we invoke probabilities when we are uncertain about outcomes. As an example from project management, the uncertainty in the duration of a project task can be modeled using a probability distribution.  In this case the probability distribution is a characterization of our uncertainty regarding how long it is going to take to complete the task. Now, the accuracy of one’s predictions depends on whether the probability distribution is a good representation of (the yet to materialize) reality.  Where does the distribution come from? Generally one fits the data to an assumed distribution.  This is an important point: the fit is an assumption – one can fit historical data to any reasonable distribution, but one can never be sure that it is the right one. To get the form of the distribution from first principles one has to understand the mechanism behind the quantity  of interest. To do that one has to first figure out what the quantity depends on .  It is hard to do this for  organisational phenomena  because they depend on several factors.

I’ll explain using an example: what does a  project task duration depend on?  There are several possibilities – developer productivity, technology used, working environment or even the quality of the coffee!  Quite possibly it depends on  all of the above and many more factors. Further still, the variables that affect task duration can depend on each other – i.e. they can be correlated.  An example of correlation is the link between productivity and working environment. Such dependencies are  a key difference between Normal and power law distributions. To quote from the paper:

The difference lies in assumptions about the correlations among events. In a Gaussian distribution the data points are assumed to be independent and additive. Independent events generate normal distributions, which sit at the heart of modern statistics. When causal elements are independent-multiplicative they produce a lognormal distribution (see this paper for several examples drawn from science), which turns into a Pareto distribution as the causal complexity increases. When events are interdependent, normality in distributions is not the norm. Instead Paretian distributions dominate because positive feedback processes leading to extreme events occur more frequently than ‘normal’, bell-shaped Gaussian-based statistics lead us to expect. Further, as tension imposed on the data points increases to the limit, they can shift from independent to interdependent.

So, variables that are made up of many independent causes will be normally distributed whereas those that are made up of many interdependent (or correlated) variables will have a power law distribution, particularly if the variables display a positive feedback effect.  See my posts entitled,  Monte Carlo simulation of multiple project tasks and the effect of task duration correlations on project schedules for illustrations of the effects of interdependence and correlations on variables.

Wrapping up

We’ve looked at a couple of general mechanisms which can give rise to power laws in organisations.  In particular, we’ve seen that power laws may lurk in phenomena that are subject to positive feedback and correlation effects. It is important to note that these effects are quite general, so they can apply to diverse organizational phenomena.  For such phenomena, any analysis based on the assumption of Normal statistics will be flawed.

Most management theories assume  simple cause-effect relationships between managerial actions and macro-level outcomes.  This assumption is flawed because  positive feedback effects can cause  qualitative changes in the phenomena studied. Moreover,  it is often difficult to know with certainty all the factors that affect a macro-level quantity becasues  such quantities are typically composed of  several interdependent factors.  In view of this it’s no surprise that managerial actions sometimes lead to unexpected  extreme consequences.

Interdependence, not independence

Typically we invoke probabilities when we are uncertain about outcomes. As an example from project management, the uncertainty in the duration of a project task can be modeled using a probability distribution.  In this case the probability distribution is a characterization of our uncertainty regarding how long it is going to take to complete the task. Now, the accuracy of one’s predictions depends on whether the probability distribution is a good representation of (the yet to materialize) reality.  But where does the distribution itself come from? Generally one fits the data to an assumed distribution.  This is an important point: the fit is an assumption – one can fit historical data to any reasonable distribution, but one can never be sure that it is the right one. To get the form of the distribution from first principles one has to understand the mechanism behind the quantity  of interest. To do that one has to first figure out what the quantity depends on .  It is hard to do this for  organisational phenomena,  which generally cannot be studied in controlled conditions.

To take a concrete example: what does a  project task duration depend on?  Developer competence? Technology used? Autonomy? Quality of the coffee??  Quite possibly it depends on all of the above. But even further, the variables that make up the quantity of interest can depend on each other – i.e. the can be correlated. This is a key difference between Normal and power law distributions. To quote from the paper:

The difference lies in assumptions about the correlations among events. In a Gaussian distribution the data points are assumed to be independent and additive. Independent events generate normal distributions, which sit at the heart of modern statistics. When causal elements are independent-multiplicative they produce a lognormal distribution (see this paper for examples drawn from science), which turns into a Pareto distribution as the causal complexity increases. When events are interdependent, normality in distributions is not the norm. Instead Paretian distributions dominate because positive feedback processes leading to extreme events occur more frequently than ‘normal’, bell-shaped Gaussian-based statistics lead us to expect. Further, as tension imposed on the data points increases to the limit, they can shift from independent to interdependent.

So, variables that are made up of many independent causes will be normally distributed whereas those that are made up of many interdependent (or correlated) variables will have a power law distribution, particularly if the variables display a positive feedback effect.  See my posts entitled,  Monte Carlo simulation of multiple project tasks and the effect of task duration correlations on project schedules for illustrations of the effects of interdependence and correlations on variables.

Scientific management theories assume a simple cause-effect relationship between managerial actions and macro-level outcomes.  In reality however , it is difficult to know with certainty all the factors that affect a macro-level quantity; it is typically influenced by several interdependent factors.  In view of this it’s no surprise that simplistic prescriptions hawked by management gurus and bestsellers seldom help in fixing organisational problems.

Filed under: Causality, Management, Organizations, Power Laws, Probability, Risk analysis, Statistics

Critical Project Resources - The Software Architect By Dave Nielsen In the early days of software development little thought was given to how the software applications and systems we built were architected. There were several reasons for this: firstly, software development being new, the concept hadn’t been thought of, and secondly we didn’t realize how important architecture [...]

Ah. Adventure Maps.

I was happily surprised that Mondays post about Project Adventure Maps took so much interest. As it should. Away with spreadsheets, checklists and grid prisons.

Bring on your crayons!

So. Adventure Mapping.

A highly playful, interactive and intuitive way of communicating complex elements in your project. Or just an excuse to go overboard on the lame jokes and awkward analogies.

I find it to do wonders for stakeholder analysis.

Stakeholder analysis is a technique to identify and analyze the stakeholders surrounding a project. It provides information on stakeholders and their relationships and expectations. A proper analysis of the stakeholders will help you to construct a project approach suited to the situation and will allow you to negotiate better with the stakeholders.

It works like this (you can click on the image to enlarge).

Invite some relevant people to brainstorm the stakeholder map. Get a whiteboard. This can be an old fashioned one, you know the kind you put on a wall. Or this can be an online version for remote organizations.

Draw an image of the goal. A treasure. A princess in a tower. A shovel.

Draw a line that flows towards the goal. Not a straight line. Create the suggestion that the Big Adventure is one that includes obstacles and challenges. The openness and flow stimulate creativity. It suggests you have room to think.

Now instead of a Gantt chart, you will be using a little brass gong to synchronize the teams flow.

Just kidding.

The next step is concerned with the question “Who are the stakeholders?”

For this, you basically draw people or smileys along your project road map. What is the first time they pop up? That’s the place where you draw them on the flow. If you draw them closer towards your path, they have more influence, are more important. Often you start with the obvious stakeholders, and the longer you talk about it, the more crowded the whiteboard gets.

The attitude of the stakeholders towards the project determines their behavior. Happy people are more likely to cooperate than an angry mob. With the use of smileys or + and – sign, try to assess the indicate of the stakeholder towards the project.

Not all stakeholders are created equal. Not everyone has to be involved. And you don’t want to have everyone messing around with your scope. So. Draw walls between your path and the stakeholders that don’t have or need an involvement. And when you draw, yell: “Block Them!” For those that need involvement draw a nice and inviting corridor (or arrow) between the flow and the stakeholder. I am still looking for a good phrase to say when you draw the arrow.

Now you have a nice Stakeholder Adventure Map.

Of course, it’s the process that is important. Not only the map.

The stakeholder analysis will help you create your communication strategy and project organization.

Read this post if you want to know more about Stakeholder Analysis.

Other people who liked this article liked these too

• The Project Adventure Map. Go Left At Scope Creep Mountain.So. We should have fun and creativity in projects. So. The team should have a shared understandin...
• Future Project Management Software Tells You Bob Will Be Unhappy Predicting the future is always more an art than a science, but I will give it a shot today. What...
• Using Stakeholder Analysis in Software Project ManagementWhile preparing a workshop stakeholder management, I came across an article I wrote four years ago. ...

Bas de Baar is an independent consultant based in the Netherlands. He uniquely combines over a decade of project and team leadership with nearly a decade online presence in the area of Project Leadership in a global and virtual world.

Stakeholder Adventure Maps. Drawing Smileys And Walls.

Knowing where you are most likely to fail can help set the stage for reducing the chances of failure.

- Bilal M. Ayyub, Peter G. Prassinos, and John Etherton