In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my tenth and final reason for using Agile is that Agile addresses the realities of software development and the needs of the business. This post will expand upon this topic.

One reality of software development is that there is still too much unsustainable development in the industry as a whole. Squeezed projects with significant overtime and stress prevent us from getting smarter, let alone working smarter. Technology and business are continuing to change at ever-increasing rates; keeping pace with the business while keeping our skills current and relevant is a major challenge, yet essential if we are going to improve productivity. Through sustainable development and continuous improvement, Agile development helps teams and individuals do both.

A few tangible benefits associated with Agile development that meet the realities of both software development and the business – contrasted against traditional, plan-driven development projects – can be categorized broadly as: The early bird gets the worm.

Software project teams need feedback about features from the users. No matter what documentation was prepared in advance of development, there is nothing like having the users seeing the working software to obtain final, definitive agreement on whether the software meets their needs. Signing off on a specification and “building to the spec” might meet a project measurement, but it won’t necessarily satisfy the customer.

Why? In my experience, I’ve found that users aren’t particularly good at dealing in the abstract, or at least they aren’t as good at dealing in the abstract as software developers. They need to see things, not conceptualize them. That, coupled with the fact that communication between individuals is always subject to misinterpretation at some level, leads to the ever-present “requirements problems.”

I’ve heard the phrase, “Now that I see it…” quite often over the years, followed by a change request. Agile development’s frequent delivery of features enables users to routinely inspect the (working) software to provide immediate feedback. And because the highest-priority features are delivered first, software project teams get early feedback on the most valuable business features, allowing for course-corrections to occur early, instead of late, in the project.

Because the highest-priority features are being worked to completion early, the business can benefit from an early return on investment. With traditional, plan-driven projects, software is delivered as a complete package, but the business must wait until the end of the project to begin realizing a benefit.

Another, related benefit to using Agile development is that valuable time is not spent on the details of features that may or may not be a priority later. With traditional, plan-driven projects, you run the risk of investing time and energy into defining and designing features that may not be desirable or even needed. Time is money, and I prefer not to waste either.

Early delivery, early feedback, and an early return on investment; the early bird (the business) does get the “worm” with Agile development – or is it really a caterpillar that morphs into a butterfly in the cocoon of the Agile team?

Another reality is that business is rapidly changing. As Faisal Hoque points out in an article, The Speed of Business Today, “Constant change is the new dynamic of the global economy, and makes agility even more necessary than at any point in business history.”

Software development should align itself to the needs of the business by being able to respond change, and Agile development does just that. In the 2009 State of Agile Development Survey sponsored by VersionOne, the number one benefit obtained from implementing Agile development was the ability to manage changing priorities. 90 percent of the respondents said implementing Agile either improved or significantly improved their ability to manage changing priorities. (The survey data included 2,570 participants in 88 countries.) An older survey, the Shine Technologies 2003 Survey, was another global survey of actual experiences using Agile development. The highest ranked positive feature reported in the survey was the ability to “Respond to change over plan.”

In terms of general alignment between the business and IT, the 3rd-highest benefit obtained from using Agile development reported in the 2009 State of Agile Development Survey was an Improved Alignment Between IT and Business Objectives. This is supported by the Shine Technologies 2003 Survey, which reported that 83 percent of those surveyed stated that business satisfaction was better or significantly better.

Satisfaction can be a function of a variety of factors, but one reality is that software is being designed and built for the customer. People like to feel involved, that their needs are paramount in your mind. Traditional, plan-driven projects “respond” to the customer, but usually with change control and a “here’s how the project will be impacted” assessment. A rigid, process-oriented approach isn’t bad, but it can develop into an almost adversarial relationship where the customer feels that they are the one that must press for changes to their software.

This is because traditional, plan-driven projects have a focus on “planning the work and working the plan” where delivering to the specification is the key measurement. While there are project managers who run their projects as a series of negotiations (which helps maintain the relationship between the team and the customer), the very nature of the process causes people to lock in on what is defined – and change is becomes disruptive.

Agile development is designed to handle change; people don’t become wedded to a plan because they haven’t invested a lot of time and effort defining, estimating, and designing everything prior to actual construction. Agile teams wait until the last responsible moment to begin diving into the details of any one feature. And features that haven’t been started yet are considered negotiable at all times. This allows the business to change its mind with ease. And because Agile teams work with the customer throughout the project, the customer feels involved and valued throughout the entire process.

Agile development also stresses the need for the use of sound technical practices as a means of improving productivity. While you don’t have to be on an Agile team to make use of these practices, Agile development promotes them. Automated testing, Test-Driven Development, continuous integration are all geared towards improving throughput without sacrificing quality or sleep.

My final point, and final reality, is that software professionals are all adults, and they should be treated as such. Agile development, with its autonomous, self-directed teams allows people to define and manage their own work, which provides a motivational – and productive – boost.

Dan Pink, in his book Drive: The Surprising Truth About What Motivates Us, cited a study conducted by researchers at Cornell University that examined 320 small businesses, half of which granted the workers autonomy, the other half relying on top-down direction. The businesses that offered autonomy grew at four times the rate of the control-oriented firms and had one-third the turnover.

Top-down, control-oriented management brings its share of problems. In his book, The Way We’re Working Isn’t Working, Tony Schwartz puts this into solid perspective. Schwartz states, “The all-too common dynamic is today’s workplace is parent-child. Most employers tell employees when to come to work, when to leave, and how they’re expected to work when they’re at the office. Treated like children, many employees unconsciously adopt the role to which they’ve been consigned. Feeling disempowered and vulnerable, they lose the will and confidence to take real initiative or to think independently. Doing what they’re expected to do often becomes more important than doing what makes most sense.”

Command-and-control management creates the need to manage people more closely! Is that what we want? I know that I don't. Agile development's final contribution to the realities of software development is that it provides the opportunity for everyone to be treated as valued, professional, adults. That works for me!

There appears to have been a data center outage early morning, affecting a number of applications including Pivotal Tracker. This has caused connectivity problems for users in some locations, and it appears to still be persisting for some.

We're working with our hosting provider to get this resolved as soon as possible, this is our top priority.

This is the second data center outage this week. At the moment, we do not have enough information to know whether the outages are related.

Also, we have received reports that Tracker has been unreachable from certain parts of the world (including China) since the migration to a new data center last week. We've filed a request with Engine Yard to investigate, and hope to have this resolved soon.

Our apologies for the inconvenience these outages have caused. We'll post more information here as we receive it. You can also follow @pivotaltracker on Twitter for updates.

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my ninth reason for using Agile is that Agile development is motivating and engaging. This post will expand upon this topic.

My position is that developers are knowledge workers, and as I pointed out in my article, “knowledge workers have the greatest understanding about their own work, and that they are the ones best qualified to plan and organize how they will accomplish that work.”

I’m not alone in this observation. In their book, High-Performing Self-Managed Work Teams by Dale E. Yeatts and Cloyd Hyten, Yeatts and Hyten stated, “Case studies have shown that the decisions made by SMWTs (Self-Managed Work Teams) are extremely effective because those making the decisions – the team members – are the most knowledgeable persons about the work.” (Buchholz, Roth & Hess, 1987; Ray & Bronstein, 1995).

Not only are those decisions effective, but allowing teams to operate autonomously – self-directed – is a great motivator. Why do so many people seek to start their own business? Money, yes, but another very common reason is autonomy. People want to live their lives the way they want to; they don’t want a boss telling them what to do, when to do it, and how to do it. The control factor present in most workplaces is a major factor in the decision to go it alone.

Tony Schwartz discussed this control factor in his book; The Way We’re Working Isn’t Working. Schwartz states, “The all-too common dynamic is today’s workplace is parent-child. Most employers tell employees when to come to work, when to leave, and how they’re expected to work when they’re at the office. Treated like children, many employees unconsciously adopt the role to which they’ve been consigned. Feeling disempowered and vulnerable, they lose the will and confidence to take real initiative or to think independently. Doing what they’re expected to do often becomes more important than doing what makes most sense.”

As a manager, I certainly want people thinking independently and taking initiative. I want people to do what makes sense, and by all means avoid following a procedure because “that’s the way we’re supposed to work” when that procedure doesn’t apply! I want people to understand our business goals and to think and act in accordance with those goals.

A great treatment on the subject of engagement and motivation is given by Dan Pink in his book Drive: The Surprising Truth about What Motivates Us. He relates a theory put forth by Edward Deci and Richard Ryan called “self-determination theory” which argues that we have three innate psychological needs – competence, autonomy, and relatedness – ingredients that Agile development endorses. Deci and Ryan feel that when these needs are met, we are happy, productive, and motivated. (No argument from me!)

In support of productivity gains through autonomy, Dan Pink cited a study conducted by researchers at Cornell University that examined 320 small businesses, half of which granted the workers autonomy, the other half relying on top-down direction. The results were fantastic! The businesses that offered autonomy grew at four times the rate of the control-oriented firms and had one-third the turnover.

While I'm on the subject of engagement and motivation, good morale is vital to positive motivation and engagement. Studies support that adopting Agile development definitely improves morale. The 2009 State of Agile Survey by VersionOne lists Improved Team Morale as the fourth-highest benefit obtained from adopting Agile development. Mike Cohn reports in his book, Succeeding with Agile: Software Development Using Scrum, that fifteen months after adopting Scrum, Salesforce.com surveyed its employees and found the 86 percent were having a “good time” or the “best time” working at the company. Prior to adopting Scrum, only 40 percent said the same thing – a solid improvement to say the least!

A couple of quotes that that capture the concept of autonomy, motivation and engagement:

“Mediocrity is expensive – and autonomy can be the antidote.” – Tom Kelley, General Manager of IDEO

“Traditional management is about compliance. If you want engagement, self-direction works better.” – Dan Pink

Finally, Agile development provides a sense of accomplishment that is very motivating. Through the delivery of working software early and often, there is a sense of meaningful accomplishment on a regular basis. There is also a feeling of belonging and relatedness due to the highly collaborative nature of Agile development.

I’ll summarize by quoting from my article. “This control over their workday, plus operating in a sustainable mode and the feeling of accomplishment that is a by-product of continuous delivery of working software, all combine to provide a solid motivational boost to each and every person on an Agile team.”

What is a manager to do? As Josh Leibner, Gershon Mader, Alan Weiss observed in their book The Power of Strategic Commitment: Achieving Extraordinary Results Through Total Alignment and Engagement, “Outstanding managers can create environments conducive to people motivating themselves.” Agile development creates this very environment – provided that you (as a manager) and your organization endorse and support it.

Twitter provides a great rule of thumb for what kind of information to share with other people on a project.

Before inviting someone to a meeting or sending them an email ask yourself:

Is this information something they’d choose to “follow” on Twitter.

If yes, send it to them.

If not, or if its something you just think they “should” be following, don’t.

This will help keep meetings in check and emails in check and hopefully more aligned with the specific information each person needs to do their job (and not get bogged down in useless information).

As discussed in my recent article in ComputerWorld, this works because Context Trumps Content (as the growth of Twitter makes abundantly clear).  There is no end to the amount information people can find.  What is valuable is not the quantity of information, but the quality of information. To be high quality and valuable, the information needs to be relevant to the recipient.  (And Twitter makes it easy for people to receive only the information they find relevant to themselves.)

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my eighth reason for using Agile is that, Agile development expects continuous improvement. This post will expand upon this topic.

As I noted in my article, “Agile development expects its practitioners to be continually learning and adapting…” This concept is articulated in the final principle of the Agile Manifesto, which states:

“At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

As a manager, I want to see everyone to improve and grow, and i feel that continual learning is vital in the software business. If you aren’t learning and growing (you’re “maintaining”), you are effectively losing ground. The world will move past you.

It is equally important that my development organization is utilizing the best practices possible to make individuals and teams as effective and productive as possible. Agile development actually makes a manager’s job easier with respect to learning and improving people and the software development process. You don’t have to explicitly create a “learning opportunity” for your staff (carving out time in busy schedules); the opportunity is created through the Agile process itself.

As I stated in my article, continuous improvement is enabled in part through sustainable development. Improving productivity requires that people grow and change, and in order to accomplish this, people need time to learn, think and reflect. Another component of improving productivity is getting people to accept change. The best way that I know of to accomplish this is to have them initiate the change on their own.

The other enabler is the individuals themselves. People must have the desire to actively learn and grow; they must value learning and be willing and able to act on what they learn, improving their personal and/or team productivity. By applying what they learn and then thinking and reflecting on how well it worked (and tweaking where necessary), new knowledge and expertise is developed. This is a true learning organization at work.

To facilitate this, Agile development incorporates the concept of slack. As James Shore and Shane Warden described it in their book, The Art of Agile Development:

“To keep up with their constantly expanding field, programmers must continually improve their skills. In doing so, they will often learn things that enhance their work on the project.

“Dedicated research time is an excellent way to encourage learning and add additional slack into your iterations. To introduce it, set aside half a day for each programmer to conduct self-directed research on a topic of his choice. Be completely hands-off. I recommend only two rules: don't spend this time on project stories or tasks, and don't modify any project code.”

I’ll make one point about the first sentence, which in part states, “…they will often learn things that enhance their work on the project.” Some people are better learners than others, and often times it is better to learn about specifics that are directly related to the work that is being performed or is about to be performed then something that you might need to know about in the future – there is direct applicability and reinforcement of the learning. It’s an application of the age-old, “use it or lose it” advice.

Team retrospectives are another way that Agile development fosters improvement. A retrospective is a meeting where a team looks back on their past iteration so that they can learn from their experience and apply this learning to future projects. In contrast, traditional, plan-driven projects normally hold something like a “post-mortem” at the conclusion of the project, when it is too late to improve that project.

I like this continuous improvement cycle of Agile development, not only because small improvements are being routinely made, but because the changes are automatically bought into by team members because they are the ones initiating the change. I also like regular retrospectives because they are also a great time to reflect on what went well, and what should be celebrated. Celebrating success helps keep people energized and engaged.

When it comes to improvement, some key points about retrospectives that I feel are important:

No blame. Retrospectives shouldn’t look to lay blame, but to examine – professionally and depersonalized – what needs improving.

Focus on what is in the team’s control. A team retrospective that focuses on exclusively that is outside of their control isn’t particularly useful to the team. Sure, maybe the team could have used an extra resource or two. The question is, did the team maximize the use of everyone? What or how could the current team have done to work more productively?

Drill down to actionable tasks. Don’t settle for high-level statements of a problem and/or vague notions of what could be done. Decompose the issue into actionable tasks that could be taken on in the next iteration.

Prioritize and select a task. A team doesn’t have to take on a suite of improvements all at once. In our organization, we use two-week iterations, so every two weeks there is an opportunity for improvement. Teams need to ask themselves, what would have the greatest impact on productivity for the team?

There are some managers who get uncomfortable with the concept of slack during an iteration and time dedicated to routine retrospectives. They don’t feel that they are maximizing productivity because team members aren’t engaged in “producing” software on a constant basis. This is counter-intuitive, in my opinion.

If you want productivity gains out of your knowledge workers – increasing the capacity of your existing workforce – then those people need to put knowledge into their knowledge banks. Knowledge workers cannot improve their capacity without investing time and effort to do so. This is a two-way street, however. The organization must recognize and provide the opportunity for this to happen, and the employees must take responsibility and take the initiative to follow through.

Agile development creates this opportunity. And it is up to the teams and the individuals on the teams to exploit the opportunity that is presented to them.

A new case study was released describing how Creativity, Inc. of Van Nuys, California doubled the productivity of their internal art department using Vertabase project management software. The case study looks at performance over the last four years and concludes:

“I would strongly recommend Vertabase to coordinate project management activities for internal creative teams. There is absolutely no downside.”

Here are links to the case study, a pdf of the case study and the press release about it.

Recently the Director of the Office of Management & Budget (OMB), Peter Orszag issued a directive that was posted on the OMB blog that outlined three specific actions for IT reform. The actions include a freeze on all new IT modernization task orders for financial systems, reviews of current high risk IT projects and require agencies to submit improvement plans to the CIO; thirdly, the OMB Deputy Director will develop recommendations within 120 days to improve the federal government’s overall IT procurement and management practices. Orszag states:

“While a productivity boom has transformed private sector performance over the past two decades, the federal government has almost entirely missed this transformation and now lags far behind on efficiency and service quality.  We are wasting billions of dollars a year, and more importantly are missing out on the huge productively improvements other sectors have benefited from.

Quite simply, we can’t significantly improve the efficiency and effectiveness of the federal government without fixing IT.”

My experience is that government IT project managers are very competent people and they want to deliver on time, on budget results.  However, as confirmed by this survey, 78% of government IT project managers feel that they are not equipped with the people, processes and tools to determine accurate project estimates and to conduct effective program affordability management.  The root cause of late, over budget IT projects are inaccurate, over-optimistic project estimates.  I and others at my company have worked hard to change this, and to ensure that government IT project management are educated about project estimating software but little has changed over the past ten years.  Fundamental change will only result from greater leadership focus on estimating accuracy throughout OMB and agency program management. 

 

I often hear questions from Pivotal Tracker users about how to organize teams and projects. We also see many requests for features that would make it easier to see stories from across multiple projects.

Tracker is designed for full immersion in one project at a given time. This stems from how we work at Pivotal Labs.

We organize teams such that a single team (and the people on that team) have a single backlog (and Tracker project). This means that within a team, there are no conflicts in terms of priorities, there is less context switching, and the team is completely focused. It leads to more consistency from iteration to iteration and therefore a steadier velocity, which allows you to have a more accurate insight into how long the rest of the backlog (or a release) might take to complete.

We also make it so that anyone on a given team can grab the next available story from the top of the backlog (or the current iteration). This implies few or no specialists (there is no back-end guy), and is generally referred to as collective ownership. It increases overall efficiency by allowing the team to dynamically re-balance, and minimizes reliance on any individual person (which among other things, leads to more relaxing vacations for developers).

The project's customer (or product manager) focuses on prioritizing stories in the backlog, and the development team is collectively responsible for delivering software based on the backlog.

We use labels to tie related stories together within a project. These can represent a major feature, specific end customer, etc. Labels can help answer questions like, how much work is left in this large feature?

A single backlog for the entire team does put more work on the plate of the owner of the backlog (customer / product manager), as he or she has to constantly make potentially difficult prioritization decisions, but, thinking hard about priorities is a good thing, and it allows the development team to focus on getting more work done. That ultimately makes everyone happier.

Also, there are people in certain roles (for example executives and designers), that given their nature, tend to be involved with multiple projects at once. Tracker could certainly use some features to help these roles, and we're thinking about these, but overall, it's more oriented towards enabling the immersed team.

A single team/project can get large enough to the point where it becomes challenging to manage a single backlog. For us, this point is generally reached with 5 to 6 pairs of developers (or 10 - 12 people). Assuming that more developers can actually add value to the overall project (this is not always the case), it's probably worth considering splitting the team into multiple smaller teams, each with their own single backlog.

To avoid knowledge and cultural silos with multiple teams, we find it helpful to rotate a few people around teams every iteration. It's important to maintain consistency (and therefore a steady velocity), so you don't want to shift too many people around too often - usually rotating just 1 person (on each team) each iteration is enough, assuming you're pairing and switching pairs within each team often.

In a multi-team (and Tracker project) environment, the product/project manager acts as a load balancer, and allocates work across the multiple teams/backlogs by considering velocity, dependencies, etc. This is typically a full time job. Tracker doesn't have much out of the box to help with this, but we're thinking about this as well, although it may be that some of this kind of work is better done in a spreadsheet, or other, more traditional project management tools. (As a side note, we did recently add the ability to move stories between Tracker projects, making things a tiny bit easier for people who manage multiple teams/projects).

I'd love to hear your thoughts on any of this, including suggestions for how to organize large projects and multiple teams (and how Tracker can help with that).

Pivotal Tracker is moving to a new private cloud hosting environment at Engine Yard this Thursday, July 22, starting at 8pm PDT.

Planned downtime is approximately one hour, but because we're changing IP addresses of the Tracker servers, it may take longer for DNS changes to full propagate.

If you've opened your firewall to a specific IP address for Tracker integrations, you'll need to make changes. We'll post the new address of the integrations server after the move, you can also 'ping api.pivotaltracker.com' to resolve it.

Apologies for the inconvenience, we're hoping for noticeable performance improvements in the new environment.

This list was adapted from ITBusinessEdge

Lack of governance: Project criteria, roles, processes & outcomes not used or accepted by management.  Not understanding project risk.

Internal politics: Territorial fights.  Its not my job, or “they” messed up.

Communication issues between the business and IT: IT talking with the business stakeholders about bandwidth and blobs rather than end user oriented benefits.

Unclear expectations: Bad estimates and ambiguous expectations.

Lack of fact based analysis: Plans not based on facts but on opinions.  Studies have shown, for example, that projects of any magnitude can’t produce a viable estimate without a model like SEER.

Lack of input from users: IT may know how to do it but users probably know what they need better.

Changes in project without re-planning

Unplanned changes in key personnel

Unrealistic schedules: Projects on death marches.

Unanticipated operations costs: These must be estimated well up-front

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.

Related posts:

1. More Software Project Failure / Challenge Information From CAI
2. Poor Estimation: Major Root Cause of Project Failure
3. Software Project Failure Costs Billions.. Better Estimation & Planning Can Help

Related Posts Computer Generated

• Step One: Establish Estimate Scope and Purpose
• SEER TO Remediate Troubled Corporations
• Eight Causes of Project Failures: UK Government IT Projects
• COSMIC Function Points In SEER

Project management is currently undergoing its own Cambrian era, the geological period some 500 million years ago in which an epic explosion in diversity and transformation took place.  Whereas just a decade ago there were only a handful of robust desktop project management applications available in the market, there are now dozens (if not hundreds) of products available on the desktop, online, and on mobile devices.

Likewise, the Web has enabled more voices to contribute to the evolution of the project management discipline. Here’s a list that the team at LP likes to follow. It’s reasonably thorough, definitely subjective, and full of people with interesting things to say.

Steve McConnell, founder of Construx and author of several groundbreaking books on software development and task estimation. Steve’s estimation book was hugely influential to the design of LiquidPlanner in terms of our approach of using ranged estimates.  Steve is one of the foremost thinkers today on the topic of software engineering and how the latest methodologies are bringing new efficiencies to light, enabling development teams to iterate faster and improve overall quality.

(Full disclosure: Steve serves on LiquidPlanner's board of advisors.) 

Bas de Baar. Based in the Netherlands, Bas is the man behind the popular Project Shrink blog which looks at project management through a humanistic lens. Bas understands that individual team members trump process and his blog digs deep into issues such as team dynamics, project leadership, and management techniques. His article, 25 Sure Fire Ways to Motivate Your Team Members should be required reading for anyone who manages a team.

 

Michael Krigsman. Michael is the author of the IT Project Failures blog, which as the name implies catalogs how and investigates why so many IT projects fall flat on their face. IT projects are unique from other projects in that they tend to have their own unique complexities (distributed teams, firm deadlines, etc.). His blog reminds me of that demotivation poster that depicts a ship sinking behind a setting sun with the caption: “Mistakes. It could be that the purpose of your life is only to serve as a warning to others.” By chronicling these IT failures, we too can (hopefully) learn from the mistakes of others.

Elizabeth Harrin. I’ve had the pleasure of speaking to Elizabeth a few times since starting LiquidPlanner.  Her blog, A Girls Guide to Project Management, is a well rounded composition on virtually every facet related to project management. She also recently authored a well-reviewed book entitled Project Management in the Real World which includes more than 50 case studies drawn from a variety of industries. Her perspective as a woman in the project management discipline also informs much of her writing and provides some practical advice on how women in the field can better advance their careers.

Scott Berkun. While Scott doesn’t write about project management exclusively, the core themes that he regularly touches on are wholly relevant.  Having spent nine years as a program manager at Microsoft, he knows a thing or two about innovation and deadlines.  Scott is author of several best-selling books (including Making Things Happen), and his highly-opinionated style shows that he isn’t afraid to ruffle a few feathers.  If you haven’t had a chance to see him speak, you should. Check out his classic lecture on The Myths of Innovation delivered at Carnegie Mellon.

Rick Freedman. Rick is the founder and principal consultant at Consulting Strategies. If that weren’t enough, Rick is a prolific writer who contributes regular, highly insightful columns to TechRepublic and CIO Update and has also authored several books on IT consulting best practices. Rick has made a name for himself these past few years writing about agile methodologies. His post from the beginning of this year provides some great, practical tips for transitioning to an agile methodology.

Jim Highsmith. One of the coauthors of the Agile manifesto, Jim is another leading thinker on the topic of agile project management (in fact, that’s precisely the title of his 2005 book). If anyone understands the concept of uncertainty in IT planning, it’s Jim. He recognizes that software development is a constantly moving target and that in order for traditional project management practices to succeed, they have to be highly adaptive.

Glen Alleman. In addition to serving as a Vice President for consulting firm Lewis & Fowler, Glen also authors the Herding Cats blog, which looks at project management from a variety of perspectives. As the name of his blog implies, large and complex projects often have multiple points of failure. From the mechanics of decision making and evaluating risk to estimation and scheduling, Glen brings the perspective of an experienced practitioner to bear on the quantitative aspects of project management.

Pawel Brodzinski. Pawel is another practitioner who shares his insights from the front line of software project management on the unambiguously named Software Project Management blog. Like Bas de Baar, Pawel writes mostly about the dynamics of successful teams, the qualities found in effective managers, and practical tips for software teams to build better software. Best of all, Pawel has a very candid and often funny style which makes a sometimes dry subject very entertaining.

Johanna Rothman. Johanna is the founder of Rothman Consulting Group and author of both the Managing Product Development and the Hiring Technical People blogs (as well as several books on these and other related subjects). As a consultant, Johanna also has a great deal of experience with implementing Agile methodologies and holds a number of different workshops aimed towards helping teams realize their agile ambitions.  

By no means is this a comprehensive list, but if you’re interested in staying atop the latest industry trends, be sure to keep tabs on the sites listed above. And if you’re on Twitter, here’s a list of handles:

Bas de Baar: twitter.com/projectshrink

Michael Krigsman: twitter.com/mkrigsman

Scott Berkun: twitter.com/berkun

Jim Highsmith: twitter.com/jimhighsmith

Pawel Brodzinski: twitter.com/pawelbrodzinski

Elizabeth Harrin: twitter.com/pm4girls

Johanna Rothman: twitter.com/johannarothman

About LiquidPlanner

LiquidPlanner is an online project management solution for scheduling, collaboration, and time-tracking in one easy package. We are focused on new and innovative features to meet the needs of this generation's project teams.

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my seventh reason for using Agile is that Agile development builds trust and relationships. This post will expand upon this topic.

My article highlighted two key areas where trust and relationships are built using Agile development:

• Between the team and the business (customers and/or stakeholders). 
• Between members of the team.

Building Trust with the Business
James Shore and Shane Warden explained this issue in their book, The Art of Agile Development:

“Many organizations I've worked with have struggled with an ‘us versus them’ attitude between customers and programmers. Customers often feel that programmers don't care enough about their needs and deadlines, some of which, if missed, could cost them their jobs. Programmers often feel forced into commitments that they can't meet, hurting their health and relationships.”

The business side has its drivers; sometimes they are the ones dealing with an imposed date – like meeting regulatory requirements. Other times, the business is responding to a competitive threat where delay is costly in very real terms, such as revenue and market share. And in other cases, there is an opportunity that the business wants to capitalize on to drive growth – a “first to market” scenario. Whatever the situation, the DATE that customers have in mind is very real.

On the software project side of the fence, other dynamics can come into play. In response to the business pressure, for example, a software project team can be driven into “optimizing” a tradition, plan-driven project before the project has even started. Why? Because the schedule – and indirectly, the team – isn’t trusted. Typically, it is management that is examining project schedules, and they are usually responding with one or more of the following motivations:

• A strong desire to complete the project as quickly as possible so that the team can move on to other, pressing projects.
• Pressure to meet the business needs within certain timeframes to win the business and drive revenue (or prove the worth of an in-house organization).
• A strong desire to ensure that the team is motivated by “just enough” schedule pressure to work up to their potential.

The reality is that everyone is concerned about whether the project team can deliver working software even close to the expected timeframe and anticipated budget. Documents and plans are fine, but it is the long gap between project inception and the first delivery of working software creates distrust with traditional, plan-driven projects.

Let’s face it, after months of dialog, reviews, and eventual sign-off to “get the requirements right,” customers are still going to feel uneasy, particularly in the early software construction phases when there is nothing to see. And despite everyone’s best efforts to fully comprehend the needs of the business, there will be questions during the development of specific features, and some of those questions will reveal changes – or even new features – that should have been a part of the requirements, but weren’t.

What can happen next is “the case of the imposed date” from a development point of view. The customer thought that they had expressed what they needed to development, and development provided an estimate. The problem is, as the team gets into the project, a deeper understanding occurs and the original estimates are in need of revision.

It is at this point when someone (usually a manager) reminds the team that they created the estimates – which are now considered cast in stone and not the approximation that they really were – and the team is held to its “commitment.” Doubt starts creeping in, and the question that is on everybody’s mind is: Will this project make it?

These days, it is no secret that a great many projects suffer schedule overruns. And when projects become challenged, conversations invariably begin about trimming features from the planned delivery or extending the delivery date, even before the customers have seen much – if anything – of the software that is under construction. At this point the customer’s fears are realized, and trust takes a hit.

If customers can see it, they can believe it – and ultimately trust those producing it. This is precisely the point that my article made, the benefit of trust that Agile development provides: “Through early and often delivery of working software to the business – even if this is nothing more than demonstrating the features – the business will grow to trust the development team. In addition, the continual dialog and the ability for the business to adjust and adapt the software gradually changes the ‘us versus them’ dynamic into a ‘we.’”

Referring back to The Art of Agile Development, James Shore and Shane Warden summed this up nicely: “Stakeholders don't know how to evaluate your process, but they can evaluate results.”

Customers don’t understand all of the ins and outs of software development, but they do measure us. Their ultimate measure is the end result – working software that either meets or fails to meet their needs. Their satisfaction can be – in part – a direct result of how much participation and input they experienced during the software project.

This is where Agile development lends itself to establishing trust and relationships. Agile development expects and embraces continual customer involvement with the development team. Instead of a large amount of time spent up front defining and refining all of the requirements, Agile development spreads this involvement out over time, accomplishing the same goal, but in different ways. Agile development also includes regular iteration reviews that demonstrate new features to the customer – a very visible display of progress that is much more convincing than reaching a milestone marker on a project plan.

This increased trust and improved relationship provides a couple of other benefits. If the customer wants add new features that affect the schedule, the trust and relationships that are built allow for candid conversations about what the customer will receive and when, including discussing trade-offs because new features are being added.

Even if the development team is struggling with a feature that is proving to be more complex than anticipated, the visible progress and constant contact with the team makes the “we’re struggling to deliver this feature” conversation is much easier to have. Because of the connected nature of the relationship, the customer has a greater appreciation of the effort involved.

The other benefit of trust and the frequent delivery of working software is that a significant amount of documentation is simply not required. In sequential projects, mounds of documentation exist to inform and guide development efforts because it is assumed that conversations will not occur. The requirements document is intended to capture all that there is to know prior to it being handed off to development. Continual conversations cut down on the need for comprehensive documentation; and contrary to what some contend, Agile development does NOT mean no documentation.

Building Trust between Team Members
The traditional, plan-driven approach to development organizes and optimizes its work functionally – with formal hand-offs between functions. People can start to identify with their area of specialty and not with the project or worse, not with the customer. The notion of specialists and optimizing work isn’t bad, and this model does work. Given that work centers around human beings, problems can and do arise.

When I first became a development manager, one problem that our organization had was that there was a wall between Development and QA. I worked hard with the QA manager and everyone involved, breaking those barriers down.

This involved coaching – casting QA as a benefit by focusing attention on the fact that our job was to prevent defects from getting out the door and into our customers’ hands. I began working on setting higher goals for our developers, asking more in terms of design, code reviews, unit testing, and the like. The QA manager worked at raising the QA game and pointing out that developers weren’t being sloppy just because bugs were showing up – at least not if they were using sound practices like conducting design and code reviews.

We also held project meetings that fostered and encouraged more open dialog. Eventually, everyone began to gain an appreciation for the skill and work required on both sides of the fence, and despite that fact that at the time we were still a plan-driven, functionally-oriented shop, the teamwork and rapport significantly improved between the Development and QA organizations.

Agile development takes building teamwork and rapport to a higher level. Project work matters more than a functional role, and as I noted in my article, “Agile teams make the most effective use of the team members as dictated by the needs of the team to meet its commitments. The shared goal becomes more important than each individual working strictly within his or her area of expertise, with defined policies and procedures for handing off work between functional roles. This breaks down barriers between functional disciplines, enabling the team to reach higher levels of productivity.”

This means that a developer can and should help test (particularly if that testing involves creating automated tests) to help the team finish its work. Meeting an iteration commitment shouldn’t be an individualized, “I’ve completed my tasks,” but a, “How is the team doing in meeting our commitments?” With Agile development, there is a strong expectation that the work is all about the team getting across the finish line together.

Because everyone is working closely together – preferably co-located – and focused on completing the team’s highest-priority items first, trust is established through continual interaction and delivery. The daily stand-up meeting reinforces individual commitments to accomplishing meaningful work each and every day, and the expectation that Agile development places on continuous team and individual improvement builds trust as team members and teams improve their game.

Here’s a summary of some of the posts that made me stop and think this week. I believe this new format is more useful than a dump of all the posts I tweeted during the last week. Thanks to @coryfoy for helping me with the Google Reader logistics on this. Enjoy!

Connectivism in the Enterprise

“When information develops rapidly, the content is no longer the object of learning activities. Instead, the development of the learner’s capacity for ongoing growth (adaptation) becomes the key focus.”

15 Awesome Quotes on Creative Collaboration

"It is the long history of humankind (and animal kind, too) those who learned to collaborate and improvise most effectively have prevailed." – Charles Darwin

Taming the Agile Enterprise: Value Stream Mapping for Knowledge Work

Value Stream Mapping is a proven and effective way to address challenges within the manufacturing environment, an environment with static processes. This article expands on that, showing ways to
modify it to be compatible with the world of knowledge work, where the challenges are more dynamic, complex and fast paced.

Favorite Tweet of the Week

@rkelly976: My 3yr old got upset, my 5yr old "Dad, explain it to her, don't just tell her". A communication lesson there #pmot #leadership

Kanban, Mental Models, and Double Loop Learning

That means we need to create an environment where:

• valid information is apparent
• it is safe to explore underlying assumptions
• participants are actively involved in controlling their tasks and collaborative problem solving
• the participants are focused on a bigger goal
• we can compare the result of our change actions with the actual outcomes

Capability Development

“The one statistic that for me sums up where China is going is the 60 million qualified university graduates that enter the workforce each year.”

Strategy Is For Amateurs, Logistics Are For Professionals

“…implementation, not strategy, is what usually separates winners from losers in most industries, and generally explains the difference between success and failure in most organizational change efforts, sales campaigns and so on.

Cross Functional Teams Don’t Come Free

“Good communication is both hard and crucial for any organization. It is therefore imperative that we let communication be one of our guiding principles when choosing between the two variants.”

Convincing People

“Trust is a fluctuating resource. Everyone starts with a level of trust. Sometimes this can be negative (for example, criminals). Your interactions with others either increases or decreases the trust. The unfortunate thing about trust is that while it takes a huge effort to build complete trust, a single wrong move can totally deplete it. People have different trusting personalities, ranging from gullible to suspicious. And they may also (rightly) trust you differently on different subjects. For example, while they may accept everything you say on electronics, your advice on car maintenance falls on deaf ears.”

Does a Kanban System Eschew Estimation?

“So kanban teams do not eschew estimation simply because it is hard. Some teams choose not to estimate because they can realise the same benefits that estimation gives with a lower cost.”

Alistair Cockburn, in the Oath of Non-Allegiance, has issued a call to action for the software development community to stop bickering and calling out contending methodologies. He has called for “discussion about whether an idea (agile plan-driven impure whatever) works well in the conditions of the moment.” As someone who has earned his PMP, CSM , received certificates from the Lean Enterprise Institute, and who completed David Anderson’s Kanban Coaching workshop – I have had the opportunity to see this bickering up close. I have occasionally even as the target of it.  Here is Alistair’s call to action:

I promise not to exclude from consideration any idea based on its source, but to consider ideas across schools and heritages in order to find the ones that best suit the current situation.

This is becoming an increasing familiar theme. We see it in Laura Brandeburg’s Business Analyst Manifesto. It is expressed by by Liz Keogh, Jean Tabaka and Eric Willeke in “A Community of Thinkers”. I am involved in a PMI Agile team where we are trying to make sense out of the benefits associated with bringing together the PMI Body’s of Knowledge (Portfolio, Program, and Project) in the context of Agile. I am working with the IIBA’s Agile Business Analyst core team to express the Business Analysis Body of Knowledge in an Agile fashion. I also participate in various Yahoo and Google groups where we are working at having these kinds of conversations involving Kanban and Lean.

These teams and discussion groups bring together practioners and thought leaders from various communities working to understand and to share. Sometimes the discussions get heated. You have a lot of successful, intelligent people sharing their experiences and trying to make sense out of their successes while simultaneously trying to expand their world view. The awesome thing is that there is a lot of learning and connecting going on in these communities.

You may want to protect your turf -  but this is the future. The tools and resources that support software development have improved orders of magnitudes in the last two decades. It’s crazy to believe we don’t have a long way to go to figure out the best ways to deliver software – especially at the enterprise.  Tom Peters quotes General Eric Shinseki, Chief of Staff, U.S. Army – “If you don’t like change, you’re going to like irrelevance even less.” I choose to join the community who is looking for ways to honor the fundamental premise of the Agile Manifesto – We are uncovering better ways of developing software by doing it and helping others do it.

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my sixth reason for using Agile is that Agile development enables emergent innovation. This post will expand upon this topic.

It is no secret that companies desire greater innovation to differentiate themselves in today's marketplace. Emergent innovation is all about deriving innovation organically from within the organization as a natural course of performing day-to-day work.

I’ll admit that this particular reason for using Agile development was of my own choosing, and I cannot point to specific studies that support this as a benefit. Since I was responsible for writing the article, I wanted to give my opinion on a benefit that I feel should be a part of Agile development.

Step One in enabling emergent innovation is to provide a sustainable development environment. As I noted in my article, “Teams that are working constant overtime to meet schedules simply lack the time or inclination to think about anything else other than the difficult schedule in front of them. In sustainable development environments, people have the time to think more about the business and explore – creating the potential for innovation that did not exist previously.”

Step Two is to have conversations as a team. Requirements (User Stories) should not be considered cast in concrete, but should serve as a basis for a conversation about what the business wants and the anticipated benefit(s) the business expects as a result. As I noted in the article, “…the dialog between business experts and the technical experts can yield unexpected results, like turning complex, difficult features into elegant, differentiating features.”

We’ve experienced this. We’ve had teams review User Stories and discuss how certain features could be implemented – a back-and-forth conversation about the business workflow and what was realistic for software to handle automatically along with how the user needed to interact with the software to accomplish his or her goal. We’ve been able to eliminate unnecessary steps and simplify the software while still providing a great user experience that met the needs of the business.

Sometimes, users are asking for features that are based on other software that they have experience with. The trick is they don’t always know how to separate how they perform tasks today with what is possible tomorrow. This is why understanding the anticipated benefit is an important component of the User Story, and why conversations between those who understand the business and those who understand software construction are highly beneficial. You can end up i a much better place than either side could have imagined by working in a sequential, isolated hand-off development approach.

Emergent innovation can also involve an entire product. We initiated one project that was little more than a vague notion of the actual problem that we wanted to tackle. This was a classic R&D effort where we gradually – and experimentally – tackled the issue of exposing complex, back-end insurance processing to web portals.

We initially felt that we needed to build a portal itself, but as the team immersed itself into the effort, a shift occurred. The result was a middleware set of tools and technologies that enable us to interrogate a back-end system and generate schema and meta-data that describe the workings of the back-end system. (And an even better, recognition for the developers in the form of a patent application.)

The result is that we don’t create web portals, we allow the definition and management of templates, rules, and workflows that govern the semantic display and behavior of information that can be contained within web portals. We make it easy to access back-end systems and propagate changes out to the web, with our middleware facilitating the dialog between the web portal and the back-end system(s). Our customers have full control over the look and feel of their web portals, leveraging our technology to make their lives a lot easier.

Naturally, this type effort required some of our best people. People who have a proven track record of creative thinking, the ability to deal with ambiguity, and that inquisitive, exploratory mindset that enabled them to define the goal and then work towards achieving it.

What are your thoughts? Does emergent innovation belong among the Top 10 Reasons for Using Agile?

Back in September of last year I wrote about The Fifth Discipline and the Agile Enterprise. In that article I connected mental models and double loop learning with Agile. Mental Models are a way to describe a person’s intuitive perception of the world around them. How we act on that world, our decision rules, are based on  our mental models. In single loop learning, we may change our decisions, but we leave our underlying mental models and decision rule unchanged. In double-loop learning, we change our underlying mental models and decision rules to better serve us in the real world. I talked about how Agile, when done well, inspires us to explore our mental models and improve our decision rules. When our mental models go unexplored, we won’t change our decisions and so we won’t get new results.

Theory in Use and Espoused Theory

Recently, I have seen several references to Chris Argyris in the Agile community. Argyris is an American business theorist who developed a way of explaining organizational behavior called Action Science. In Action Science he describes two simultaneous mental models that make it difficult to create change in an organization. The first is our Espoused Theory. Our Espoused Theory describes the model we say we use to describe how we act (or how we would like others to think we act). Our Theory in Use is one we actually use to make decisions. From a personal standpoint, the Theory in Use is complicated for a number of reasons. It is shaped by how we are participating in the situation. I might respond differently to my brother (who I work with on clients) then I would respond to a client. It is shaped by the threat we feel in the situation. I might respond differently when the situation is under control then I do when I feel threatened. This becomes even more interesting when we are dealing with organization’s and changing the mental models that shape organizational behavior.  Making changes in the stories we tell about why we behave the way we do won’t change our decision. A key to making change is to intervene at the Theory in Use.

Model I-Inhibiting Double Loop Learning

Argyris tells us that when human beings deal with issues that are embarrassing or threatening, their reasoning and actions conform to a model called Model I. Trying to make change in a Model I organization is difficult because you are dealing with their Espoused Theory. It is neither rewarding nor safe for them to explore or actually change their mental models and decision rules – so there is a wide gap between their Espoused Theory and their Theory in Use. The defensive behavior in Model I organization’s create a vicious make this divide even greater. Model I organizations have the following values and supporting behaviors.

1. Define goals and try to achieve them. Participants rarely develop mutual definition of purposes – nor are they open to altering their perception of the task. Participants plan actions secretly and  and manipulate others to  agree with a their definition of the situation.
2. Maximize winning and minimize losing. Participants feel that once they have decided on their individual goals it is sign of weakness to change them.
3. Minimize generating or expressing negative feelings. Expressing or permitting others to express feelings is a bad strategy. Participants unilaterally protect themselves.
4. Be rational. Interactions are objective discussions of the issues.Participants withhold the truth, suppress feelings, and offer false sympathy to others.

Model I behavior results in organizational defensive behaviors that block exploring underlying mental models and the resulting maturity that arises. Most organizations exhibit Model I values and behavior most of the time.

Model II-Encouraging Double Loop Learning

Argyris describes a much more productive type of organization that he calls Model II. In a Model II organization, it is safe and rewarding to the participants to explore underlying mental models and decision rules. Model II organizations have the following values and supporting behaviors.

1. Valid information. Participants design environments where accurate information is shared and underlying assumptions can be openly explored.
2. Free and informed choice. The participants jointly control tasks and focus on collaborative problem solving.
3. Internal commitment. The participants jointly protect each other in learning and risk taking. Mental models and decision rules are jointly explored.

Model II behavior results in organizational behavior that enhances underlying learning. High maturity organizations exhibit Model II values and behavior.

Kanban and Action Science

So, if we want double loop learning (and we do), we want to promote Model II values and behavior. That means we need to create an environment where:

• valid information is apparent
• it is safe to explore underlying assumptions
• participants are actively involved in controlling their tasks and collaborative problem solving
• the participants are focused on a bigger goal
• we can compare the result of our change actions with the actual outcomes

Kanban explicitly creates this environment. The board, the tasks on the board, and the metrics make valid information readily available. The visualization of the work helps make it safe to explore underlying assumptions. You are no longer looking to cast blame, you are looking for an understanding of the system – it is depersonalized. The participants are involved in defining the board, the policies, and participate in problem solving on the board. The board and the focus on reducing lead time, reducing defects, and the policies changes the focus of the team to the bigger goal. Finally, the data available to us helps us make sure the improvements we attempt are achieved and sustained.

Kanban and Maturity

Chris Argyris’s intervention method for developing Model II behavior is simple. Map the system, internalize the map, test the model, invent solutions, intervene and study the impact. Kanban puts a simple, actionable model of this in place. Even better than the single cycle intervention model, the Kanban cycle supports continuous learning that the team internalizes. Argyris’s model gives us some insight into why Kanban teams are consistently achieving double-loop learning and rapid maturity.

Further Reading

Argyris, C. and Schön, D. (1996) Organizational learning II: Theory, method and practice, Reading, Mass: Addison Wesley.

Argyris, C. (1993) Knowledge for Action: A guide for overcoming barriers to organizational change, San Francisco, CA: John Wiley & Sons, Inc.

To move your organization forward you have to manage change.

Change can come from outside the organization, like a change in the laws or the economy. It can come from inside the organization, like a quality improvement initiative.  Or, it can come from the market.  Often, you need to proactively make changes to succeed.

These can include changing the way you do things, changing the products or services you provide or changing how you market those products.

Here is a six step guide to managing change successfully. It connects the dots from the previous two posts (with a philosophical p.s. on the importance of change and good project management).

First, a definition. At its core, change is about going from where you are to where you have to be.

1. Create a work-flow. A good work-flow gives you a picture of your current state.  Analyzing the work-flow can help you decide where you want to go. Use data on the effectiveness of the current work-flow as a baseline against which to compare your future work-flow.

2. Create what you want your future work-flow to look like.  This is your target work-flow. It should be driven by your goals. That is, figure out what it is you want to improve (customer satisfaction, profitability, job satisfaction, costs, etc.) and build a target work-flow that should deliver those improvements.

If change is being imposed from outside your organization (e.g. because of the economy) your target work-flow is about figuring out how to do what you currently do in a different, but equally effective, way.

3. Build a project plan that maps out how you are going to get from here to there. It should tell you the road you’re going to take, who’s getting you there and  how to keep moving forward when you hit road-blocks.

4. Once you get there, once the target work-flow is implemented, measure its effectiveness to see if it meets your goals. Did it improve what you wanted it to?

5. If it didn’t meet your goals, figure out what else has to change and go through the process of creating that change.

6. If it did meet your goals, find another area to improve and do it again.

It has been said that change is inevitable. All things change. Having a process in place to manage change makes it more comfortable and increases your chances of have a successful transition.

On a Philosophical Level:

Economic growth depends on change. If that’s the case, a good project manager, one that can manage change and make positive changes a reality, is an engine for economic growth.

Here is the presentation I gave the Agile Atlanta and to the PMI Professional Growth Event this month.

Kanban 101 View more presentations from Dennis Stevens.

In my post, What is Better About Agile Development? (a reprint of the article Top 10 Reasons to Use Agile Development that I wrote for DevX.com), my fifth reason for using Agile development is that Agile development creates a sustainable development environment. This post will expand upon this topic.

As a software development manager, my responsibility is to make sure that our development organization produces working software day in and day out, year after year. And like many other managers out there, it seems that every year I’m being asked to do more with less. I can accomplish this in one of two ways:

1. Squeeze my staff for all they are worth (and replace them out when they burn out).
2. Work smarter.

I personally like creating a collaborative, fun, sustainable work environment that is focused on retaining good employees and not working them into the ground to meet imposed, aggressive (ridiculous) deadlines that are based on little more than hopes and dreams. Call me crazy, but I like treating professional knowledge workers as professionals. And I enjoy seeing people develop and grow – it’s truly rewarding to see people reach new heights!

Sustainable software development is all about meeting the needs of the business on an ongoing basis, which is typically translated into meeting project schedules. There are other ways of meeting the needs of the business, which I discussed in Agile Development Embraces Business Agility. The issue of software project schedules and the usual overtime response in order to meet those schedules is worth examining in term of productivity and results.

In my article I stated that, “Software projects rarely meet their initial schedule. Overtime with the mandate to ‘work harder’ is frequently used as it becomes apparent that a project will be running longer than anticipated.”

This has been an all too-common experience for many software professionals, and a great deal of anecdotal evidence and various studies that support that schedule overruns are frequent:

In a Dynamic Markets Limited Independent Market Research Report commissioned by Tara Consultancy Services in August 2007, 62 percent of the IT projects failed to meet their anticipated project schedule.

The latest Standish Group report in 2009 report shows that only 32 percent of IT projects were considered successful – meeting the schedule, budget, and delivered with the expected features.

In his book, Software Runaways: Lessons Learned from Massive Software Project Failures, Robert Glass found that schedule overruns were very common, pegging schedule overruns at 89 percent. (The book profiled 16 of the largest and most publicized software failures of the 1990s.)

Using overtime as a mechanism to deal with software project schedule issues creates far more problems than it solves, and generally inhibits the creation of a viable, productive, long-term organization. The longer-term risks of burnout, turnover, and overall sustainability are commonly acknowledged (and frequently ignored), but the short-term effects are rarely mentioned.

In the first place, the productivity of knowledge workers (like programmers) plunges with heavy doses of overtime. Research from The Journal of American Epidemiology – studying people who work greater the 55 hours – found that individuals had problems with memory, problems with functioning appropriately and problems with reading and comprehension.

From a software development standpoint, a major short-term issue associated directly with overtime is quality. In a paper, Impact of Overtime and Stress on Software Quality by Balaji Akula & James Cusick, Baleji and James focused on the impact of project teams working on an aggressive schedule, studying four projects over a two-year period (three of which had an aggressive schedule).

The chart below demonstrates a dramatic difference in defect rates when no overtime is involved:


Project #1 Project #2 Project #3 Project #4 Estimated person hours 784 416 416 528 Overtime person hours 175 167 0 198 Defects 2038 1545 142 2000
I’m quite sure that I know why defect rates are low when no overtime is present; I’ve experienced this many times as both a programmer and as a manager. Unrealistic schedule pressure and overtime not only introduces fatigue that produces more defects, programmers stop thinking as deeply and carefully about their work in response to the pressure to meet the deadline. Corner-cutting begins to creep in at every turn as output becomes a major focus – at the expense of practices that help to produce more reliable code.

Overtime has other problems as well. Since programmers are knowledge workers, they need to putting knowledge into their “knowledge banks,” something that will pay dividends later. Routine overtime on routine project tasks eliminates the time for the professional growth that will be of future benefit. (And in other cases, people are doing themselves a great disservice by failing to continually acquire new knowledge, as Pawel Brodzinski pointed out in a post, People Don’t Want to Learn.)

Then there is the whole work/life balance issue. My wife, Lauri-Ann, has referred to herself as a “software widow” at times when I’ve been very involved – and I mean working some very long hours – on software projects. You can tell when you are tipping the scales in favor of work too much when your wife pleads with you to put the computer aside to watch your daughter play soccer on a Saturday morning. (True story, Lauri-Ann almost had to hit me over the head.) In the long run, this isn’t healthy…

Sustainable Development
I support Agile development because it promotes sustainable development as one of its 12 Principles, as found in the Principles behind the Agile Manifesto. The principle states, “Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.”

Agile development doesn't try to predict the future by estimates alone, let alone try to make that future happen via overtime. With Agile development, the future can be predicted based on actual team velocity. Agile teams size User Stories in points and tracks team velocity – the User Story points that are completed in each 2-4 week iteration.

Stories are considered done when they are delivered as working software with high quality. Over a period of time, it becomes crystal clear how much work can be realistically accomplished by a team on a sustainable basis. Future delivery can be predicted by contrasting the team velocity against the active product backlog and the size of the User Stories contained in that backlog.

Once you have a clear picture of what is realistic and sustainable, how should you seek to improve team performance? It shouldn’t be about working harder, it should be about working smarter. How is the collaboration? Are there personnel issues or friction? Are practices such as Test-Driven Development being utilized? I’ve witnesses Agile teams working very productively, accomplishing more than they were when they were working overtime through the proper implementation of Agile development. The output was higher, it was sustainable, and morale was better.

When it comes to having developers maintain their skills and working smarter, one contribution that Agile development brings to the table (if you choose to use it) is the concept of slack as a technique to provide for research time each iteration. I first encountered this concept in the book, The Art of Agile Development by James Shore and Shane Warden. They state:

"To keep up with their constantly expanding field, programmers must continually improve their skills. In doing so, they will often learn things that enhance their work on the project.

"Dedicated research time is an excellent way to encourage learning and add additional slack into your iterations. To introduce it, set aside half a day for each programmer to conduct self-directed research on a topic of his choice. Be completely hands-off. I recommend only two rules: don't spend this time on project stories or tasks, and don't modify any project code."

What about teams meeting their commitments? Shouldn't they work overtime to meet what they committed to? If you have a team that struggles to meet its commitments every now and then, you actually have a great situation! It means that they are taking on work that tests and stretches them. It should be a judgement call whether to roll the work into the next iteration or deal with it now. It really depends upon the urgency and circumstances.

If there is a critical customer deliverable with a tight timeframe, a good team will recognize that and get the job done, and they will set aside their research time to make things happen. And yes, there are times that as a manager I've had to remind teams that a customer was waiting on us, and that yes, working over the weekend was necessary. There is nothing wrong with a spike of activity from time to time.

There are other circumstances when it is OK to let the work slide. Software development isn't highly predictable, and what appears simple when it is estimated can turn out to be more difficult in the actual implementation. In this case, why punish people by mandating overtime? Since it is a difficult problem, taking the extra time to deal with it – keeping people fresh at the same time – will improve your odds that it will be dealt with correctly the first time.

I've seen comments on LinkedIn and on other blogs that lead me to believe that there are some very wrong implementations of Agile development out there. Instead of sustainable development, the term "Agile" is being misused and abused to drive a series of short-term, "mini death marches" where the goal is to code quickly (without thinking) and to drive the output of the team in unrealistic and unhealthy ways. Unfortunately, this is the very issue that Agile development seeks to overcome.

Don’t be misled, though. Agile development takes discipline. It takes knowledgeable, capable, engaged people. It takes a supportive management team. It takes training and time. It is work and it is change, but in the end I believe that it is worth it. Regular use of overtime should be viewed as a serious problem.

• @jeffpatton You can find a list of their authors at http://oreil.ly/9UDudf. Maybe someone you know. in reply to jeffpatton #
• @jeffpatton Or I can introduce you to JRoss and Addison-Wesley if interested. in reply to jeffpatton #
• @jeffpatton Looks like http://bit.ly/c9uTol and http://bit.ly/bifYp2 may live nearby. Hope that helps. in reply to jeffpatton #
• @jchyip When lost in the wilderness and the map doesn't match the terrain, in all cases believe the terrain. in reply to jchyip #
• I am presenting How to Get Started with #Kanban tomorrow night at Agile Atlanta at http://bit.ly/aHEwel at 6:45 PM. #
• Vote for Billy Wagner for the NL All-Star http://bit.ly/9Zif93 #braves #mlb #allstar #
• @jurgenappelo Does squarespace.com support a wiki? in reply to jurgenappelo #
• @derekhuether That's pretty funny; in reply to derekhuether #
• Check out: Organizational Complexity http://bit.ly/9hNrip #pmot #kanban #baot #
• Watching World Cup Soccer with @leanopinions (@ Taco Mac) http://4sq.com/6TStWL #
• RT @davidjbland #scrum terminology post up on http://bit.ly/btASgT & http://bit.ly/a8Wfot #duck amp;cover – hear hear #
• RT @mlevison @donaldegray @mcottmeyer @dennisstevens… quite an esteemed crew at agile Atlanta this evening – it was a great time #
• @OlafLewitz We have your Dad in our prayers. in reply to OlafLewitz #
• with @mcottmeyer (@ Caribou Coffee w/ @mcottmeyer) http://4sq.com/7EdsKw #
• @mlevison Did you get my Evils of Silos game email yesterday. Want to give it a run? in reply to mlevison #
• Check out: Okay… Just What are we Transforming? http://bit.ly/cDjdty #pmot #kanban #baot #
• @mhsutton Mike, maybe the transparency of Kanban is used to expose dysfunction so it can be acted on at a system level. in reply to mhsutton #
• @ThirstyBear Kanban will expose a broken process – highlighting the problems – making it easier to address in the organization. in reply to thirstybear #
• @rachelcdavies What part did Kanban play in exposing to the team and management the benefit of focusing on TDD and refactoring? in reply to rachelcdavies #
• @rachelcdavies Because TDD and re-factoring are great practices – why were they being done before kanban? in reply to rachelcdavies #
• @JoshuaKerievsky Facade and Decorator in reply to JoshuaKerievsky #
• @mlevison Moving to Atlanta? in reply to mlevison #
• @JoshuaKerievsky Factory, Facade, Decorator, Observer, and Command in reply to JoshuaKerievsky #
• Nice: RT @hemppah: s/push/pull/g #kanban #
• #kanban #baot Our new paper, "Taming the Agile Enterprise: Value Stream Mapping for Knowledge Work" will be available next week. #
• @rachelcdavies Very cool. That is consistent with my experience. Kanban provided insight – triggered right improvement focus. in reply to rachelcdavies #
• @ThirstyBear Observable, Data Driven, Defensible – good places to start the conversation – you are right, this still requires leadership. in reply to thirstybear #
• With Brian Huelskamp – my brother in law – back from Afghanistan. (@ Tavern at Medlock) http://4sq.com/bXqWr1 #
• @Bullet_Boy they still can't beat the Lakers. And I don't like the Lakers. in reply to Bullet_Boy #
• @Coty fjjshuennzkz #
• @Coty You are awesome. #luckykid in reply to Coty #
• @scrumnl Except kanban doesn't have anything to do with a deathmarch – a project that is destined to fail. in reply to scrumnl #
• @scrumnl Kanban supports realistic scheduling, flexibility in scope management, and focusing of learning opportunities. in reply to scrumnl #
• @scrumnl #kanban #039;s WIP limits also create slack – its just not timeboxed. in reply to scrumnl #
• @pascalpinck @mcottmeyer and I apply Argyris' Double Loop Learning in our Cutter paper Rethinking the Agile Enterprise http://bit.ly/JsNLJ in reply to pascalpinck #
• @@pascalpinck Here I integrate Argyris model with Senge's (Sytems Thinking from a Structural and Organizational level) http://bit.ly/b5orB0 #
• @pascalpinck This is one of the reasons Kanban , A3, and VSM-KM work well. They encourage double-loop learning for the participants. in reply to pascalpinck #
• @hroark From your neighbors garden? in reply to hroark #
• Check out: PMBOK 5 – Rejected http://bit.ly/bzEMGn #pmot #kanban #baot #
• Check out: New “Agility Now!” Newsletter http://bit.ly/cgTWBy #pmot #kanban #baot #

Powered by Twitter Tools