The Scrum approach to agile software development marks a dramatic departure from waterfall management. Scrum and other agile methods were inspired by its shortcomings. Scrum emphasizes collaboration, functioning software, team self management, and the flexibility to adapt to emerging business realities.
All the objections that have been brought up seem to me to be trying to work around the rather obvious impediment of someone acting badly.
No, that’s not it at all. I agree if the Product Owner (or anyone) is being a jerk, this is an obvious thing for the Scrum Master (and others) to fix. My concern is something much less obvious, and (in my experience) more powerful. As I’ve written it out below, I realize it’s also more complicated to explain than I first thought.
- It is normal human nature to overestimate what we are able to perceive about a situation. We never grasp the full extent of this because we lie to ourselves about the fact we lie to ourselves.
- As Keith Johnstone (the father of theater improv) also observes, there are no status-free transactions among humans. None. (By the way, learning theater improv is an exciting way to learn how true collaboration really works.)
- 99.9% of organizations designate material power to some individuals over others. I’ll call them “bosses” and “subordinates” instead of the various euphemisms. While most bosses don’t enjoy this, they are burdened with deciding which subordinates will be retained in the event of a layoff, which subordinates get promoted, who gets to work on the new sexy projects vs. maintaining the turkeys, etc. (Some bosses tell me they really hate doing performance appraisals.) The 0.1% exceptions (maybe Valve?) are probably past the point they need Scrum rules anyway.
- The people with the vision, authority, business acumen, and maturity to be good Product Owners are also often burdened by the organization with some boss responsibilities.
- It is human nature to assume others are feeling the way we are. For example, once before I ran a retrospective, a team member told me it was not necessary to waste time on a safety check because they were all very comfortable with each other. I did one anyway (anonymously, using a hat), which revealed some team members felt quite unsafe. Confident people overestimate the comfort of others.
- Combine all the above and you get the “invisible gun effect.” THE INVISIBLE GUN EFFECT HAPPENS EVEN WITH THE NICEST BOSS IN THE WORLD. It’s not caused by the boss’s actual actions, it’s caused by the way subordinate behavior is naturally distorted around bosses. It’s normal human nature, not something Scrum Masters can “fix” any more than they can fix gravity. It’s most visible to the lower-status people (which might include newhires, people with less impressive job titles, etc.), less visible to higher-status people (perhaps including senior developers, socially confident people, people with better technical skills than social perception skills, and Scrum consultants), and practically invisible to bosses who haven’t done an A/B test yet. Nearly everyone in the discussion had reduced awareness of the invisible gun effect due to spending the past few years in relatively high-status occupations: trainers, expert consultants, senior developers, Product Owners, business owners, authors, etc. The fact that all 104 messages in the thread were written by men might also suggest some blind spots.
- There is also what my wife calls the “chirping bird effect.” I once read that when a male bird is around, a female bird will feign incompetence at something she’d normally be able to do. My wife brought this up when I asked her how she was able to solve a problem with her computer — something she’d been bugging me to fix — while I was on a business trip. This effect is gender neutral: when she’s gone I have no problem with things I seem less able to do unassisted when she’s home, such as finding my car keys or preparing food for our kids. In class we sometimes get groups that keep asking the trainer for more detailed instructions, or help manage another team member. That’s often a good time for the trainer to leave the room! We can talk about self organization until we’re blue in the face, and they won’t really get it until we leave them alone — temporarily — to let them do it. I heard a story of one Scrum coach taking a nap under the table to get a team to step up more, and of Ken Schwaber standing outside the door of the team room during a Daily Scrum. I’ve seen teams have breakthroughs in self management when the person traditionally responsible for managing them left the room. I am inspired by the breakthroughs Scrum led to, especially among the lower status team members who aren’t well represented in this discussion. Team expecting micromanagement? Try management by leaving the room — temporarily. Of course we always hold the team accountable at the Sprint Review Meeting.
- Facilitators need to learn that when safety is low, smaller breakout groups often work better. For example I’ve noticed some people in Finland aren’t inclined to talk in front of a larger group (especially in English), but liven up in small groups. Once they’ve discussed some things in their small groups they’re more comfortable sharing with the larger group. The paradox here is that appropriate and temporary boundaries can actually increase transparency.
It’s New Years Eve and personal and professional transformation seem to be top of mind for most everyone around this time a la (a) goal setting exercises and (b) the obligatory New Years resolutions.
I wanted to offer two different takes on transformation, one is a bottom up and one is top down. In the bottom up interview the #1 blogger in the world talks about how to make incremental changes through experimentation when you don’t have the authority to change the entire system. In the top-down interview we hear from a professional CIO how to transform a failing business to be more data driven and innovative.
The first take on transformation is from Seth Godin, the #1 blogger in the world. Here’s Seth Godin’s interview on transformation. Have a watch. Do you agree or is this characterization oversimplified?
Next – have a watch of Uber-CIO Randy Mott. Randy Mott was the CIO of Wal-Mart, Dell as well as HP and is now CIO of GM). In this interview with Information Week where he talks through his transformation playbook:
*Data Driven Decisions
*The Speed Merchant – bringing cycle times down
*Efficiency vs. Effectiveness
*How to move money around to be more around innovation and less around keeping the lights on
He will be in-housing over 8,000 IT jobs to innovation centers to hot tech markets like Austin, Texas or Silicon Valley.
Do you agree with Randy or Seth? What are your guiding goals in 2013? Looking forward to reading your comments.
Tags: agile, interview, transformation
Here’s a link to a number of blog posts I’ve been reading that have culture and transformation at the core. I’ve aggregated my favorites. Feel free to add to the list below in the comments section. I’ll respond with what I think.
- Here’s an article I stumbled upon while reading LinkedIn. It’s written by Jeff DeGraff a Professor from the US. In it, he talks about culture change vs. culture growth and digs back into his Hungarian roots (yay! He’s Hungarian just like me). Have a read here http://linkd.in/U6rPjj
- If you are new to systems thinking, here’s a great blog post I read by John Wenger: bit.ly/10HzGec
- What is DevOps anyways? With all the hype around Developer Operations or DevOps – this blogger reminds us to consider that it’s culture at the middle http://bit.ly/UfH0sX
We recently set a team of consultants from my company to conduct a formal assessment of a medium sized financial firm’s an Agile capabilities. I’d like to share thew approach here. The team went on site to conduct interviews and observations in 5 areas –
• Value delivery
• Agile engineering
• Project Management
• Product management
• Environment and Organizational Culture
Also, the investigation took input on the demographics of the individual project being examined, the stakeholders involved and the competitive/regulatory environment in which the organization as a whole operates. Understanding the context in which an organization operates is crucial to understanding the optimal level of Agility, and thus, the plan of action.
Understanding the goals of the organization is particularly important. Not every axis needs to be top-ranked to achieve the company’s goals. In fact, on this particular assessment we found that only one needed urgent attention – Project Management. I’ll provide details in another post.Tags: agile, Agile assessment, Agile Conference 2008, Scrum Basics, Scrum Methodology
My consultants and indeed my own software development teams often grapple with technical debt. often Products carry technical debt when they are difficult or risky to change. Technical debt isn’t listed on your balance sheet, yet it can destroy your business. It’s important to understand where Tech Debt comes from in order to effectively address it:
- A common reason for bringing technical debt into a code base comes from the business stakeholders. Assuming they have a reasonable understanding of the consequences, the business might consider getting something released sooner is of more value than avoiding technical debt. They should understand the “interest” payment that will be incurred if they insist on this path! In many cases, businesses stakeholders simply don’t understand the ramifications of what they are asking for, nor do they fully grasp the concept of. They make decisions solely on immediate business pressures rather than taking a more long-term view.
- Technical debt also comes in the form of poorly constructed, inflexible software. This may come about when functions or interfaces are hard-coded, and as such, are difficult to change.
- Lack of documentation is another reason for technical debt, both in the code itself and in the external documentation. When documentation is poor, new team members who want to modify the code in the future have a hard time coming up to speed on the code which that slows development.
Enlightened management can have a real impact on mitigating the addition of technical debt and in paying it down as you go, by constant refactoring. There is an interesting webinar on this topic available here.Tags: agile, agile scrum methodology, Scrum Basics, technical debt
When organizations adopt an agile approach to development like Scrum there is so much focus on the iterative nature of agile development that long range vision and strategic product design can get lost. Jimi Fosdick is doing a webinar on November 28 to discuss the need to include long term product vision, coherent user experience and User Centered design and architecture along with specific best practices for achieving a coherent product that delights users.
Topics will include:
• Discussion of Product Vision and approaches to crafting a compelling overall vision for products
• Discussion of User-Centered/Value-Driven design and approaches to incorporating user experience (UX) and software architecture early in the development process
• Explanation of the pitfalls of a lack of vision and so-called “hybrid” models for incorporating UX and architecture into Scrum Projects
You can register for the webinar hereTags: agile, Agile Conference 2008, agile scrum methodology, retrospective meetings, scrum backlog grooming, Scrum Basics, scrum daily standup, Scrum Methodology, scrum sprints, sprint review, The ScrumMaster Role, user stories, user stories and scrum
The webinar, “A Marriage Made in Heaven: Agile and Project Portfolio Management,” took place on October 27. I (David Parker) hosted, along with Russ King, Vice President, Product Development, Results Positive, Inc. and Caleb Brown, Systems Engineer at CollabNet. During this session, we explored the benefits of marrying Agile Project Management and PPM and we did a live demo showing this using HP’s PPM solution and CollabNet’s ScrumWorks Pro to demonstrate the powerful capabilities of managing a resource constrained project portfolio.
If you’re interested, you can watch the recording and download my slides. Here, I post some of the questions and our answers:
Q: How feasible is Agile on Projects & Programs?
A: Agile is typically thought of in the context of individual projects. Companies sometimes fail to scale that paradigm to a program level, where the program is a superset of multiple projects, each running its own lifecycle and release plan. The trick is to weave those separate lines of development (projects) into a coherent and seamless deliverable (program). The complexity comes in gathering meaningful metrics and planning releases that thread the elements together. This is exceedingly difficult to do manually. CollabNet’s ScrumWorks Pro is a tool that can make this manageable. It supports the planning of complex releases that weave in multiple development threads.
Q: Will this process will be feasible for maintenance related projects (Incident handling, less than 8 hours development works, etc.,)?
A: From the PPM perspective, an individual defect is not in and of itself a project and as such, would not be tracked. What might be tracked is a larger group of maintenance items in the form of an Epic. From an Agile perspective, a bug report or defect is just another piece of deliverable business value, like a User Story or any other Product Backlog Item. From a bug report, the product owner and team would create a Product Backlog Item (PBI), along with success criteria (definition of done). It is prioritized against all of the other Product Backlog Item by the product owner. Again, multiple bugs/defects are often grouped in an Epic.
Q: It seems the PPM is geared toward a waterfall process. It appears there is only visibility into the Development phase, but with agile, you could potentially address all phases within a single sprint. Is that just because of the way this implementation was set up or is it there isn’t a true marriage of the agile within PPM?
A: PPM in this scenario is focused on evaluating the ROI of different projects and deciding where to make investments. Agile is focused on execution of the projects that are chosen. That said, the scenario we propose makes the entire organization more Agile, in that the feedback loop is instantaneous. This allows those that are making the investment decisions to adapt and make course corrections that are indicated by that feedback loop. The integration gives all team members the ability to work in a more Agile fashion, and gives Stakeholders and Project managers the ability to benefit from the faster feedback and data generated by the team working this way.
Q: Can the tasks in Scrum WorksPro be connected to tasks, timelines in Source forge?
A: Not with Sourceforge. However this is possible with Collabnet Teamforge, the current commercial version of Sourceforge.
Q: Can you clarify what part of Agile PPM can be done in scrum works pro without need for HP PPM?
A: ScrumWorks Pro is focused on project execution and project management. As such ScrumWorks does a number of things not accomplished in HP PPM. These include PBI tracking and prioritization, Task management sprint planning, release planning, team velocity, forecasting, and many other functions related to the management of an Agile project.
Q: So are you proposing (in the demo) to combine a phase/waterfall planning and design phase, but then execute in an agile framework?
A: Combining HP PPM and ScrumWorks Pro adds to the agility of the entire organization. Feedback loops between the development team and the PMO are enhanced allowing the PMO to make course corrections required. I would not say that as a result the entire enterprise has become agile – only that they’ve become more agile. Generally, we do not see many organizations that practice a pure version of ANY methodology –be it Agile or otherwise. The reality is that organizations have a mix of methodologies, like Scrum, Kanban, Waterfall, hybrids, etc. Different teams in large organizations will often build software differently, so the challenge is to roll up the data from those disparate teams. Despite their differences, there are a number of common metrics you can track regardless of project type. These include actual cost versus budgeted cost, scope change, personnel/resource change, delivery dates, and others. Tools like ScrumWorks and HP PPM do a good job in tracking these kinds of numbers.
Q: Continuing from the first question, from a portfolio perspective, having “”open-ended”” project budgets within the Agile/SDLC process is not in the best interest of my customer. How does budget planning and Agile development work together while still having some control over costs?
A: Project prioritization and the associated budgeting/funding are is under the purview of the PPM tools. The agile project management tool tracks the amount of time individuals spend on the project. The integration between the HP PPM tool and the Agile Project Management tool, allows you to easily compare budgets against actuals.
Q: For the Forecast report in SWP, if new backlog items are added during the sprint, does that add to the top or bottom of the bar? Also, how does the Project Portfolio Management tool fit into the larger Enterprise Architecture discipline?
A: It depends on what report you are looking at. In the forecast report, added backlog item appears on the bottom of the report and impacts the forecasted delivery date.
The “Burn-Up” Report shows the relationship between work completed per iteration (sprint velocity) and project scope change. The forecast feature extrapolates the rate work gets done against the rate of scope change to provide an empirical release completion forecast for more accurate release planning.
Q: In agile, what are the differences between being adaptive to late changes in requirements within a sprint and scope change?
A: Scope change refers to any added or subtracted scope, typically measured in some form of relative effort unit like Story Points. As such, scope may be added as a team discovers more about an existing requirement. In other words, if the team finds out that a requirement is more complex than was originally envisioned, they may re-estimate the number of story points and this might add scope to a sprint. The opposite could also be true. Whether this occurs because of a discovery inside a sprint or outside of it doesn’t change the nature of how it is tracked or reported upon.
Q: When a committed backlog item could not be completed in a sprint, naturally it holds the top most priority in the following sprint. How does ScrumWorks helps in tracking this item from the beginning to end?
A: An unfinished PBI may or may not be a high enough priority in a future sprint. The determination is made by the product owners. In any event, any activity against that PBI is tracked. Tasks completed that relate to that PBI are tracked, as are those that were uncompleted.
Q: What certification do CollabNet-trained scrum masters receive?
A: Those who attend one of our Certified ScrumMaster or Certified Product Owner training are eligible to take the exam deliver by the Scrum Alliance. It should be noted that CollabNet is one of the leaders in ScrumMaster product Owner training. We have more Certified Scrum Trainers on staff than any other vendor, and we’ve trained more than 12,000 ScrumMasters.
Q: If an organization wants to be able to report a metric of time to resolution for individual PBIs, what settings are available in this integration to include/exclude a PBI from the current active lists so that a countdown starts appropriately?
A: Forecast reports in ScrumWorks can be filtered on any number of aspects, allowing a user to deliver estimates on individual tasks, Stories, Epics or Themes. By the way, you can try out ScrumWorks Pro either in a hosted environment or as a free download.
A pattern I’ve noticed is that Scrum projects are typically managed informally, with the only measures used being various velocity metrics and burndown charts. This may be an issue. Many project managers and executives resist scrum because these only measure the speed of delivery, not the project’s cost or the business value it generates. One of the major differences between traditional and agile projects is that traditional projects focus on delivering software that satisfies requirements, while agile projects focus on maximizing ROI through continuous feedback and re-planning.
This is where Earned Business Value calculations come in. It fits well with Agile projects, since the focus of agile projects is on business value rather than conformance to requirements (outcomes over outputs). In many cases, EVM metrics are easier to calculate and understand in agile environments than in traditional ones. There are three key management measures – Cost Performance Index (CPI), Schedule Performance Index (SPI), and Earned Business Value (EBV) – that provide information to help manage an agile project from and ROI perspective.
There is a solid white paper on this topic at .
I’d also be very interested in your comments to this post.Tags: agile, Agile Conference 2008, agile scrum methodology, CSM Exam, Danube, Earned Business Value, EBC, product owner scrum, retrospective meetings, scrum backlog, Scrum Basics, scrum daily standup, scrum effort estimation, Scrum Methodology, scrum story points, sprint review, sprint review meetings, The ScrumMaster Role
There is an interesting article on the Scrum Alliance Website entitled “Negotiating Scrum Through a Waterfall”. Phil Southward, CSM, CSP details the considerations development teams must face in a transition. Is it all or nothing, or can these two development methods coexist? Does it depend upon the project or how willing the organization is willing to embrace new methodologies? http://www.scrumalliance.org/articles/189-negotiating-scrum-through-a-waterfall
Phil’s intent in writing this article is not to pit the two methodologies against each other. Although his preference leans toward agile, he realizes that going full agile is sometimes not possible, particularly in the beginning of the transition. Rather his intent is to explore how Scrum can fit into a waterfall environment when the organization is unable, or unwilling to implement Scrum completely.
If you are interested in this discussion and want to explore the differences of Scrum and Waterfall in more detail – check out this great whitepaper “ Introduction to Agile Software Development” written by Victor Szalvay, CollabNet CTO and CST. It’s an excellent primer for those new to Scrum and agile. Szalvay discusses agile’s origins, its basic concepts, and how it has revolutionized the way software is developed.
In February 2001 seventeen software developers met at a ski resort in Snowbird, Utah and wound up declaring the values and principles that came to be known as “Agile.” Here is what they wrote.
Manifesto for Agile Software Development
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, Dave Thomas
© 2001, the above authors
this declaration may be freely copied in any form, but only in its entirety through this notice.
Supplementing the Manifesto, the Twelve Principles further describe what it means to be Agile.
Principles behind the Agile Manifesto
We follow these principles:
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity–the art of maximizing the amount of work not done–is essential.
- The best architectures, requirements, and designs emerge from self-organizing teams.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Scrum Training Series
- Scrum based funding model – 20 percent May 9, 2013
- The Next Big Idea March 5, 2013
- On Being Available February 17, 2013
- Should Scrum Always Require the Product Owner to Attend the Sprint Retrospective Meeting? February 5, 2013
- Happiness Metrics January 23, 2013