This time, however, feels different and it has given me some (rather obvious) insights into some of the issues I have encountered with Agile transitions over the past few years.
Intrinsic Motivation
First, and foremost, you as an individual have to want to make the change. Yes, that falls squarely unto the "Duh!" category, but a significant part of wanting to make the change is understanding clearly why the change in necessary. People often have sudden health issues or a similar crisis that triggers their desire to lose weight, exercise, or make some other behavioural change. Indeed, starting the change is relatively easy - my father used to say,
Quitting smoking is easy... I've done it hundreds of times!The tough part is making the change last. That requires motivation that is more than superficial - the type of intrinsic motivation that Dan Pink speaks of in The Surprising Science of Motivation. Put simply no change, whether it be losing weight or moving from a serial product development lifecycle, is going to work unless you really, really want it to work.
This means that the leaders in an organization must not only talk the Agile talk, but they must walk the walk as well. A senior manager cannot just say, “Thou shalt be Agile”, she must get to the gemba and attend Daily Standup Meetings, go to Sprint Review / Product Demo’s. She must let go of matrix management practices and allow long-lived teams to be created. She must eliminate silos in her organization in order to allow groups to come together determine the best way to deliver their product or system. She must be willing to eliminate a “fail-safe” environment in favour of a “safe-fail” one.
Leaders don’t have to be managers. Respected software developers, testers, infrastructure people and project managers, for example, also have to embrace this new way of working. For a transition to succeed (or even just get off the ground) these people must be engaged and willing to give new practices a try. They must be given the time to learn not just the practices, but the underlying principles and values. The people who look to these leaders for guidance need to be given time & support to learn the values, principles and practices.
At the grassroots level, though, a certain amount of cynicism can be expected. After all, Agile is just the latest fad that management is foisting upon them and, like all the other fads, it will be gone in 6 months to a year. The key to success is to expect that there will be a short term hit in productivity as people learn a new way of working. Providing support in the form of on-going training and coaching will help minimize that productivity hit, and ideally maximize the improvements.
Support
In order to obtain the required intrinsic motivation, one of the first key steps is to ensure that you have the support of others around you. Yes, you can lose weight by yourself while everyone else in your household is continuing to use the same eating patterns, but your chances of long-term success are reduced.
This applies equally to an Agile transition. A single development team tries an Agile approach and sees quite a bit of improvement. When people try to take that improvement and get other teams to adopt Agile, there isn't as much success. Why? Often there isn't the support required in other teams or management to make the change stick. Sometimes that comes from within the team itself, sometimes from their management, and sometimes from both. For example, how motivating is it for the pilot team members who worked really well together to be split up and used as "seeds" for making other teams Agile?
There are many stories of attempts to introduce Agile methods as a grassroots movement from development teams. In almost all of those cases, the attempts didn't lead to substantial change beyond the original team. There just wasn't the support outside the original team to allow Agile to grow, and there were likely many non-Agile principles and practices that continued to be used that stifled any attempts to change. That's similar to someone putting a huge bowl full of pasta in front of me for lunch and dinner, with some rich chocolate dessert after each meal. If others don't support the change, the change likely won't stick.
So, this does require some education of those who must support the change. The truly successful Agile transitions I've seen have been a combination of executive and grassroots movements. At the executive level, managers understand that their previous ways of working are no longer sustainable and are willing to incur the shorter-term costs of change in order to achieve medium and long-term gains. At the grassroots level, the people "in the trenches" understand why the executives want the change to take place, and know that they have the support to learn how to make it work.
Both levels, and everyone in between, needs to understand that the pace at which each individual will change is going to be different. Indeed, there will be some people who will refuse to change regardless of how much support there is, and how much sense it makes to put the change into effect. If you don't believe this, simply look at how many people smoke almost 3 generations after the landmark 1964 Surgeon General's report.
Discipline
OK, here's the tough part. Making the change, whether you're a CIO or a programmer, requires discipline. Yep, the 'D' word. That often turns people away immediately.
Sorry, I don't have time for that.While psychologists have long said that it takes 21 to 28 days to make a new practice a habit, new research from the UK suggests that value is actually 66 days. If you assume those are working days, that's over 3 months.
From an exercise and weight-loss perspective, I can say that it took about 2 months to change my routine and diet from a point of "this is different - I have to work at it" to "this is the new normal and I don't want to change". For the longest time, I didn't want to exercise first thing in the morning because I have always found I'm more stiff. However, trying to exercise in the evenings was running face-first into the wall of excuses - I'm tired, I need to get 'x' done, I ate too much, etc. So, when I too a realistic look at my schedule, I had to admit it just wasn't working and changed my workouts to the morning. Yes, I was stiff, but I loosened up quickly enough once I got going.
Some Agile practices can be relatively painless and easy to pick up, such as iterative development and showing working software at the end of each iteration. This is especially so if your initial foray into Agile is with a greenfield project. However, some practices are going to be more difficult and will indeed feel like you're writing with your wrong hand when you first try them.
Retrospectives are a good example of this - many teams have held "post-mortems" after projects, but that assumes that something is dead. Traditionally, not many teams have taken time during development to review how their are performing. For many people I've encountered, the Retrospective seems very "touchy-feely" and uncomfortable. Over time, though, people learn to decouple a critique of the process from team implementing the process. In that respect, the Retrospective becomes a very valuable tool and even a social event!
I can also speak from personal experience with Test-Driven Development that it took a couple of months of daily use to move from "this feels like I'm writing with my wrong hand" to "I don't want to write code any other way". I was lucky in that I had a partner who was also willing to give TDD a solid try before passing judgement on it. We paired almost 100% of the time as well, to the point that we both prefer to pair with others now, over 10 years later!
A common issue I've seen with Agile adoptions that falls into this category is "cherry-picking" of practices. In many cases, any practice that is seen to be difficult or requiring discipline is discarded from the start. That would be OK if the people doing so understood the Agile principle or principles behind the practice and they substituted practices that supported the principle. However that usually isn't the case. One of the reasons why Scrum is so popular is that it didn't include any technical practices. Teams that have been truly successful with Scrum have been the ones that learned and understood the Agile principles and ensured that either their existing technical practices supported the principles, or they chose new practices that did.
The key is to ensure that the type of support mentioned above is going to be there for the time required for the behavioural changes to truly become new habits. You must be willing to invest 3-4 months in a team to make those changes stick - even longer if you're dealing with multiple teams.
Tools
As I mentioned before, this isn't the first time I've tried to lose weight and exercise. I played a lot of sports when I was in high school, and still played competitive basketball into my late 30's. I was also a long-distance runner on the track team back in the day, and running has always been something I liked. Since running is something I can still do, it has become my primary tool for cardiovascular exercise.I also invested in a treadmill some 15 years ago. I'm an expert at finding excuses to not go outside and run, so the treadmill was bought to prevent me from saying that there's snow in the Rocky Mountains (3000km away), so I'd better not run today! Even when I was a competitive runner, I hated running in the winter and in the rain. So, I bought a good quality treadmill to remove that excuse.
Over the years I accumulated some dumbbells, a weight bench and even fashioned my own device to exercise my forearms. A couple of years ago I bought some surgical tubing for exercising after seeing it during a stint in physiotherapy. Most recently, I bought a large exercise ball and a 10 lb. medicine ball to augment my "tools".
Finally, I went to Home Depot and picked up a bunch of packages of the soft foam floor mats and created a decent workout area in our unfinished basement. The cost of all the items wasn't insignificant but it was literally spread out over 15 years, with most coming in the last 5 or 6 years.
All of these things came together to create a critical mass of having a good area to work out and enough exercise equipment to allow me to do many different exercises and to run inside away from winter in the Great White North.
From a diet perspective, I had learned about making smoothies from frozen berries, milk and protein powder a few years ago (more about that in a moment), but it required the use of a blender. Our traditional blender did the job, but it was loud and a pain to clean afterwards. During an unrelated trip to a local Costco store, I noticed that they had the Magic Bullet on sale for something like $40. I took a small leap of faith and bought this seemingly redundant tool. I now have a smoothie almost every day for breakfast using the Magic Bullet. It's quieter than our blender and much, much easier to clean. It enabled me to start and continue my diet because it lowered the bar for using it.
Shortly after I started running again, I bought a new pair of shoes. I went looking for ones that would provide some more cushioning to replace shoes that were about 5 years old and left my knees aching after an outside run. Was there anything 'wrong' with my older shoes? Not really, and they certainly weren't worn out. However, with them I was experience an adverse reaction in the form of sore knees that was providing an excuse to not run! I was fortunate enough to be able to afford to buy those new shoes, and my knees haven't ached since.
So what tools does a group entering into an Agile transition need to support them? I advise taking the same approach - what do you have now, and what simple additions can be made? What can be done to provide the proper environment, like I did by getting the foam pads for the concrete floor in our basement? What subtle changes can be made to existing tools to remove demotivating factors, like I did by buying new shoes?
Remember, the goal is to provide an environment in which the proper behaviours become easy to do. Yes, the treadmill was a significant expense in 1997, but it has enabled me to run regardless of the weather. What can you do to remove your team's excuses for sticking to unproductive behaviours?
- Eliminate cubicles in favour of an open workspace for your teams
- If that change is too radical, rearrange tiny Dilbert-esque cubes into 'pods' of 6 to 8 desks where the central space is open, and people can simply turn around if they need to have a conversation with a colleague
- Invest in as many whiteboards as you can possibly fit in any given work area
- If you absolutely, positively can't have all of your team members in the same physical space, invest in tools that allow them to be there 'virtually' - place a web cam so that anyone can view the card wall, for example
- Examine your existing tools and practices with those tools - something that made sense when you were working on a 6 month cycle may no longer make sense on a 2 week iterative cycle
- Modernize your development and testing tools - open source software is your friend in this case... you can get almost everything you need for free, and it's often better than something homegrown or purchased for many thousands of dollars!
- Provide easy access to snacks and refreshments - don't underestimate how important that is
When I started this latest journey, I didn't simply discard everything I had learned before and start from scratch. As I said before, I like running - I believe I first started when I was 13. I'm not big on cycling, and I pretty much hate cross-country skiing outright. So, since I already had the equipment to support it, the decision to use running for my primary exercise was simple.
However, that alone wasn't enough. I had also learned about 10 years ago about how the human body reacts to the consumption of carbohydrates. That was back in the days of "No Carb" diets, and I wasn't interested (probably my Irish heritage and thus love of potatoes!). One variation of those diets I did see that was interesting focused more on controlling your insulin output by avoiding carbs except for one meal per day. While I didn't apply the diet, that idea stuck with me.
A few years back I started going to Greco Lean and Fit, a local physical fitness franchise that advocates a holistic approach by advising you on diet and providing high-energy circuit workouts. It was there that I learned about the protein smoothies and supplements, and how to manage food intake better. Their workouts also showed me how variations can keep you interested, i.e. don't always do the same old thing! For a number of reasons, I stopped going to those classes, but again it was always in the back of my mind.
When I started working out again, I originally thought that I would literally train for a while so that I could go back to Greco without injuring myself when I was there! However, I found that with a few tweaks and a couple more tools I was able to create my own exercise space in our basement and do my own 10 exercise/3 repetition circuits, followed by a run. It has worked wonderfully, and I've removed the excuse of having to drive all the way to the gym, or that their schedule didn't work for me!
What knowledge, what parts of your existing environment can you leverage to make an Agile transition work? Are there tools that people are using that are working really well for them? Are there existing practices that everyone says, "Yeah, that works really well!"
If your group has an existing monstrous build, don't simply get rid of it but rather look for ways to make it faster. What are the bottlenecks? Can newer, faster hardware be thrown at it? Are there ways to build portions in parallel? Are there ways for developers to run local builds that may not represent the production grade product, but those builds are very fast and enable the developers to use Test-Driven Development?
Similarly, if your existing test suites take hours or days to run, what can you do to reduce that cycle time? If you do decide to change testing environments, don't throw out the old one immediately. Instead, use the new environment for any new development and for validating bug fixes. Gradually replace the old test environment, rather than a big-bang cutover.
From a people perspective, ensure that before starting a transition and certainly during that everyone involved is consulted and asked for their input on "what works". Leverage the collective knowledge of your people, which is a much more productive approach than simply tossing out everything that was done before the transition. You don't have to spend months doing this, but certainly investing a few days finding out what the people who have to live in the testing and development environments really like and don't like is well worth it.
You have to be somewhat judicious, though. One key factor in my diet that seems to be helping with my motivation is that I eat whatever I want for dinner. I have my smoothie for breakfast, some nuts and a piece of fruit for lunch, and water or coffee to drink. At dinner, however, I can eat anything. When we go to my Mom's place, I'll eat whatever is for dinner as well as dessert. So, I don't feel like I'm suffering by dieting the way I am.
So, although I can eat whatever I want for dinner, I no longer have grab a soft drink in the afternoon like I once did. I don't get a muffin or a doughnut with my morning coffee. I made changes, but just didn't completely change everything.
How can you and your teams take that approach? How can people such as Architects and Build Managers incorporate themselves into the teams, leveraging their deep knowledge of specialized areas? Can the teams use their existing tools to move to Acceptance Test-Driven Development, or even true TDD at the microtest level? Can they continue to their existing electronic tool for holding requirements and live with duplicating them on index card or stickies in order to have a real, tactile, card wall? Can their current workspace be rejigged slightly to make it more open?
When working with groups that are in the early stages of a transition, I often hear the phrase,
We can't just throw the baby out with the bathwater!I couldn't agree more! Look for what has worked and worked well, and look for ways to incorporate it into the new way of working. However, be aware that when using an Agile approach what was 'efficient' before may be a local optimization now.
Short, Medium and Long-Term Goals
I didn't become overweight and out of shape overnight. To get that way takes years of work and effort. :) Similarly, it takes a lot of time and effort to get back into shape. If you're dealing with a legacy system or product that is becoming more and more difficult to change, you aren't simply going to spend a few days writing some tests and making it all better. Similarly, if you have had a serial approach to product delivery for decades, you aren't going to change in the blink of an eye.
Whether you're trying to get back into shape or change how you deliver products, it's helpful to have short, medium and long-term goals to provide focus for your efforts.
For example, I gave myself a short-term goal which had nothing to do with my weight, but was to simply run 5 kilometres (without stopping to walk) and live to tell the tale. To do that, I had to dial back the running pace that my brain thought it was capable of sustaining for that distance. I achieved the goal on my 10th workout. Cool! I could do it, and now I just wanted to improve my conditioning level so that I could do it faster.
I set a medium-term goal of reaching 10 lbs. in weight loss and running 5K in 30 minutes. I achieved the running goal about a week before the weight goal, but I was now into a mindset where I was positive that, as long as they were realistic, I could set the goals and achieve them!
So, I set a long-term goal of 30 lbs. weight loss and running 10K in 50 minutes (a pace of 5 minutes per kilometre). To test my ability to run that pace, I did a short 15-minute run at that pace last week. I was gasping for air by the end - I obviously still have a lot of work to do!
What this means, though, is that you must also accept that there is going to be some pain involved with this fundamental level of change. You are going to be gasping for air. There are going to be sore muscles. There are going to be blisters. It will hurt. Whether you are wanting to exercise and lose weight or change from a silo-ed, plan-driven development culture, you have to accept that you didn't get to the point of wanting and needing to change overnight, and thus you won't be able to make that change overnight.
If you have a pile of legacy code with minimal or ineffective automated testing, you must understand that part of the pain is going to be reduced capacity to deliver new features while you get the level of automation in place to allow you to move faster. You can't simply go from minimal or no exercise for years to running a 50-minute 10K - you have to build up the capacity to do so. If you simply try to make the change in the blink of an eye, you will injure yourself. Similarly, trying to move from low levels of test automation to 100% coverage will likely not work, and end up with detrimental effects as people experience the pain of applying tests to existing legacy code.
So, everyone involved in the transition must understand that this isn't a quick fix. It will take time, there will be some pain, and you will need to stick with it long enough for the new mindset and practices to become habits. The larger the transition, the longer it will take. This is on the scale of many months to multiple years!
No Finish Line
While a 10K race has a finite distance, my efforts to improve my weight and fitness levels do not. Running for a fixed distance or fixed time is a means to an end, not an end in itself. If I simply said, "When I hit 30 lbs., the diet is over!", then I've missed the point. The real goal is to establish a lifestyle that is going to improve and sustain my health for decades, not some arbitrary weight. In other words, for what I have started, there is no end! If I assume there is, then I'll regress.
This is the same approach you need to take with an Agile transition. The goal isn't to get all your teams using Scrum. The goal isn't to reduce defects to 1 per KLOC. The goal isn't to ship on time, on budget. The goal is to create a sustainable environment in which your people can do their best work, be innovative, and enjoy themselves!
Over time, the benefits you see will diminish in size. I'm certainly not going to lose weight at my current pace indefinitely... I would reach a weight of 0 sometime in 2013. Similarly, teams that transition to Agile may see great improvements in productivity and reduced defect rates initially. Eventually, though, the rate of those improvements will decline, even on the best teams. That's OK, though, and expected. As long as you understand that you should continue to learn, continue to look for ways to improve, then you're doing just fine. If you once had 50 field-reported defects every 6 months and now you have 2, you should continue to look for ways to get to 1 or even 0, but don't do so to distraction! Don't knock yourself out, or berate the teams. The chances are that your customers are now delighted at the new level of quality!
Conclusion
Dan Pink boiled motivation down to three key points: Autonomy, Mastery and Purpose. My current weight loss and exercise program is working because I'm not just another student in someone else's class (Autonomy), I have the equipment, environment and the capability to know how to change my diet and how to do the exercise (Mastery), and... what was my Purpose? My 11-year-old daughter had joined the cross-country running team at her school and wanted me to practice with her. I couldn't say no, but I was in such poor shape I had to do something to be able to run with her.
If you don't know why you are embarking on an Agile transition - you don't know that Purpose - then don't start until you do. Don't start until each and every person involved knows the Purpose. Don't start until you can look yourself in the mirror and say, "This is absolutely worth the effort it's going to take."
Comments
I also like to meet 1 to 1 with people from all levels of the org chart, and ask each one of them, "If you were made Boss For A Day and allowed to change one thing that would be implemented without question, what would it be?"
I've had some interesting responses to that one. :)