Monthly Archives: June 2006

agile, post-agilism

That “Agile” Word

I just came across an interesting post on The Hijacking of the Word Agile. Here is a quote from that post that got my attention:

The word ‘Agile’ seems to have been hijacked once again. Now, it is commonly used to justify just about any bizarre ‘practice’ and strange behavior in a software project. In some cases, it is even misused to justify complete chaos.

Inspired by the list of weird behaviors in that blog post, I thought I’d create my own list. Here are some similar so-called “Agile” actions I’ve seen:

  • Quote: “We do releases every 3 months, so we’re Agile.”
  • Quote: “We don’t have a process or do much documentation, so I guess we’re Agile too!”
  • Being in an organization for months, and seeing no outward clues of anything remotely Agile going on. Later, we were told the organization was following an Agile method that normally is very visible (as in “hard to miss”); most people there had no idea they were doing anything Agile at all.
  • “Quality Police” moving from enforcing traditional processes to enforcing Agile processes. This included political manipulation and attempts to humiliate or blackmail developers. All under the banner of “Agile Testing”.
  • Copying a variety of RUP, iterative development and Agile writing quotations. Mixing them up into a cornucopia, that, when used out of context, sounded an awful lot like what the team had been doing all along.
  • Consultants taking CMM or V Model, (or whatever they have been selling for years,) and sticking the term “Agile” on it, (presumably for the marketing buzz), without actually investigating Agile processes and adapting.

It’s not surprising that this kind of behavior by those ignorant of Agile methods, turns some people off of the term “Agile”. Couple that with some of the obnoxious Agile elitism and zealotry that is displayed by a few who are well-versed in and experienced with Agile methods, and even more people are finding themselves suspicious of this marketing term “Agile”. Tim Beck decided to do something about it, and came up with his own term for software development: “Pliant Software Development.”

We like and use many agile techniques and methodologies. We believe that in theory, agile development and pliant development are pretty much the same. In practice however, we feel that agile development has become a little too un-agile. There are whole sets of “process” which fall under the banner of agile development and unfortunately some people have gotten to the point where they are unable or unwilling to veer from the prescribed course of action. This leads to problems, because no two software development projects are the same. There are far too many technical, social and political issues to be able to come up with a 12-step program that fits all software situations.

–From the Pliant Alliance FAQ

Here are some different ideas on “Agile” from various people:

Unfortunately the industry has latched on to the word “Agile” and has begun to use it as a prefix that means “good”. This is very unfortunate, and discerning software professionals should be very wary of any new concept that bears the “agile” prefix. The concept has been taken meta, but there is no experimental evidence that demonstrates that “agile”, by itself, is good.

The danger is clear. The word “agile” will become meaningless. It will be hijacked by loads of marketers and consultants to mean whatever they want it to mean. It will be used in company names, and product names, and project names, and any other name in order to lend credibility to that name. In the end it will mean as much as Structured, Modular, or Object.

The Agile Alliance worked very hard to create a manifesto that would have meaning. If you want to know their definition of the word “agile” then read the manifesto at www.agilemanifesto.org. Read about the Agile Alliance at www.agilealliance.org. And use a very skeptical eye when you see the word “agile” used.

—Robert C. Martin

It really gripes me when people argue that their particular approach is “agile” because it matches the dictionary definition of the word, that being “characterized by quickness, lightness, and ease of movement; nimble.” While I like the word “agile” as a token naming what we do, I was there when it was coined. It was not meant to be an essential definition. It was explicitly conceived of as a marketing term: to be evocative, to be less dismissable than “lightweight” (the previous common term).

—Brian Marick

…That’s most of what patterns are about: codified collective experience, made personal through careful feeling and thought. Unfortunately, it turned other people into sheep or, maybe worse, lemmings.

—Jim Coplien

As with any mythodology, principles that begin with reasonable grounding can be misapplied and corrupted. This reminded me of James’ observation that the Traditionalists and the Agilists are both instances of the Routine or Factory School of Software Testing (as Bret Pettichord describes in his wonderful paper Four Schools of Software Testing). I may disagree with James to the degree that I don’t think the early or smarter Agilists intended to behave in a Routine-School manner, but I think he’s right to point out that a large and growing number of their disciples appear to have slipped into Routine behaviour.

The common element between Bad Traditionalist processes and Bad Agile Processes is that both want to remove that irritating thinking part.

—Michael Bolton (emphasis mine)

The lesson to me is this: our words are important. When a term is used in many contexts meaning something different in each, it’s a vague word. When a word is vague, it is important to get a definition from the people using the word. Otherwise, it may only seem like we are using a shared language, when we really aren’t. We should also be careful with our own language, and the terminology we associate ourselves with, because, in the case of vague words, it can have unintended consequences.

quality theory, test automation, TQM

Employee Empowerment and How I Became a Contextualist

Back when I was in college, I had an instructor who was a big fan of Total Quality Management (TQM). TQM was quite popular, but in our business school setting, it wasn’t taught that much. We learned about Juran, Feigenbaum and Total Quality Control, Deming and a lot of Quality theory. We learned about kaizen, quality circles, flat organizations and employee empowerment. When our instructor had us memorize W. Edwards Deming’s 14 Points, I was hooked. Most of the Quality theory just made sense to me. Why do business any other way?

I particularly liked the idea of flat organizations with empowered employees who were in charge of their own destiny. I had worked part-time with various organizations for several years, and already saw how much waste can occur while executing business tasks. Often, the people doing the work had a much better idea on how to be more efficient, but layers of management seemed to get in the way. I often thought about Quality theory, and in these cases, about using employee empowerment. It seemed to be the answer to inefficiency issues I was seeing, and might help employee morale.

Later on, I learned an effective heuristic for business school case studies:

If I use Quality theory with a good dose of Deming as a solution for any problem, I’ll get a good grade.

So I often used Quality theory for case study assignments. It worked quite well. It seemed that few of my classmates used these kinds of ideas, so I probably got marks for being unique. Then I had Dr. John Rutland as a professor, and things changed. I found out that the heuristic didn’t always work, and that it was a poor one at that. Dr. Rutland figured out quite quickly that Quality theory was my silver-bullet, and deftly pricked the balloon of my pomposity.

After working on a business case study, he asked me for my ideas. I began reciting my typical answer:

  1. empower the employees.
  2. flatten the hierarchy.
  3. form Quality Circles to work on solutions to problems.
  4. etc.

I had barely uttered something about “empowering employees” when he cut me off.

“Jonathan. Have you talked to the individual employees yet?”

“No.” I answered.

“Do you know what makes them tick? Do you know what their motivations are? What their likes and dislikes are? Do you know not only their frustrations but their sources of pleasure?”

“No.” I stammered. “Why did you suggest it then?” I choked out something about having better job satisfaction if employees feel they have a say.

“You mean to tell me that you think an employee who works in the auto industry whose job is to turn a bolt on an engine block a quarter turn all day, five days a week doesn’t have job satisfaction? You may not enjoy that job, but other people might. They may be happy with the way things are, and miserable if they are ’empowered’. How can you be so arrogant to think that without finding out what the problems are, or what the motivations of your employees are, you can just go ahead and impose a system without taking them into account?”

Wow. I was shocked. He was right. I wasn’t empowering anyone, I was doing what Jerry Weinberg describes as “inflicting help.” I felt ashamed at my ignorance. Dr. Rutland went on: “Jonathan, people are cognitive beings. They are all unique. It’s your job as a manager to find out what makes them tick. It’s your job to help them reach their goals and remove obstacles from their way. The right solution is the one what works in their environment, taking their needs and talents into account. You can’t just impose the way you think is the best way for all environments.” I was stunned, but after I swallowed my pride, I decided I should learn everything I could from him. The lesson that day (thanks to me), turned out to be on context, and to not settle on one solution, and then always use that one solution for every problem. That was at best naive, at worst lazy, and negligently incompetent. “Use the solutions that work right now, given the team, environment and culture you have.” Dr. Rutland said. “You can learn a lot of different techniques, but in the real world, things have a habit of not going according to plan. A good manager will learn as much about these tools as possible, but will adapt and adjust accordingly, using the tools that work for each unique situation.

Furthermore, there are cases where Quality theory has failed spectacularly, in real, live organizations. The real world has a habit of yanking the rug out from under us.”

I have never forgotten that lesson on how the context for what we do is extremely important. Years later, some people echoed similar wisdom, calling themselves the Context-Driven Testing School. Their ideas resonated with me, thanks in no small part to what I learned from Dr. Rutland, who had first taught me to be a contextualist.

Furthermore, I’ve been in situations where a flat, employee-empowered organizational structure was a complete disaster. Sadly, despite being prepared for it, I was shocked. I worked for an organization that had several business school case studies that holding it up as a model for an employee-driven organization. Once I was inside, I found it was a terrible place to work. I’d never before seen such bitter political infighting, deliberate project sabotage, posturing projects, coup attempts, inefficiency, waste and poor morale. When I moved on, I was so relieved to get out of that organization with its confusing reporting structure, paralyzing indecision (from so many competing employee committees), and general toxicity. It was a breath of fresh air to go into an organization that many would decry as being “Taylorist” or “Command and Control”. At least most of the decision making was in fact decisive, and I knew who had the decision making power based on the organizational chart.

I told Brian Lawrence a bit about this experience, and he simply said: “The hierarchy went underground.” I was pleasantly surprised by this bit of wisdom. In a top-down organization, at least you know the hierarchy more or less. The unpleasant “Taylorist” and “command and control” behavior did not really go away, even in a company that was founded with a flat hierarchy and text-book employee empowerment programs. It just went underground, and it got very ugly, having the opposite effect that the founders intended. I was reminded of Weinberg’s “Rule of Three”, which goes something like this: if you can’t think of three ways a potential solution can fail, you don’t completely understand the problem space.

I’ve learned from wise teachers, and I’ve been fortunate to learn from people like Dr. Rutland and Brian Lawrence. My learning has shown me that any kind of process, structure or tool has the potential to fail. The context is important, and it is foolhardy to charge ahead with a potential solution without fully exploring the problem space. Life is full of tradeoffs: any potential solution we suggest in an organization has the potential to fail. It’s okay to still go forward with a potential solution even if we can imagine it might fail. (I don’t even care if it is the “right” one, as long as we are prepared to adapt when needed.) To believe a practice, tool, process or organizational structure is unquestioningly right, in any context, will eventually lead to disappointment. When I read back over some of my TQM notes, I find that there are some great ideas there, but I don’t treat them as my hammer and every problem I see as a nail anymore. I hope I manage to continue doing that with other processes and tools.

So what do I do now that I’ve learned something? I:

  • resist jumping to a solution too quickly;
  • learn about the context;
  • talk to the people doing the work, and do a lot of listening;
  • do careful analysis before suggesting solutions; and
  • use Rule of Three to do contingency planning and raise awareness of the potential for problems to come up in the future.

In my line of work, it is common for people to immediately ask me if I’m going to do test automation once they learn I’ll be on a project. My answer tends to be something like this:

If, after careful analysis, we find that test automation is a good fit to solve the problem we need to solve, we’ll explore test automation.

That answer seems to surprise people, but I’ve learned my “test automation as problem solution” lesson several times over. I have seen so many test automation efforts fail because “test automation” was thought of as a solution to all testing woes. I’ve seen enough failures to know that test automation is a tool that requires careful thought and planning. As Tim Beck would say, “Are we solving the real problem?” My job is to work with a team to find out what the problem is, and work with them on a solution to that problem. Test automation, like employee empowerment may or may not help us solve that problem. In some cases it may just compound our existing problems with new and exotic ones.

post-agilism, tools

Pliant Software Development

Recently, I met Tim Beck, who is behind Pliant Software Development. I like his ideas, and I support what he’s doing.

When I came across this website, I liked what I saw, and when we got together in person, I really liked what I heard. Tim is thoughtful, principled, positive and forward-looking. He wants to see software development move ahead and continuously improve. Like me, he doesn’t think we have the final word on software development yet. I was also pleased to see in his work that he is also a contextualist. You won’t get any “one-true-way” marketing from him.

Here’s an excerpt from the Pliant About page:

Again, there is no right answer. Therefore, there is no point in trying to find one. What we must do is strive to adapt our processes so that we get better at developing software. As the definition at the top of this page implies, in order to be pliant, we must bend or fold our current process to fit the situation we are facing. Since the situation is constantly changing, we have to be constantly changing our process.

One comment Tim makes frequently is this:

Are we solving the real problem?

Inspired by his description of the company where he worked that coined this phrase, and hung it up on a banner, I had an idea. This question should be printed out on a large banner in software development companies, visible to all (executives included), particularly where process or tool zealotry is reaching a fever pitch. All too often we reach for a process/tool without looking at the real problem. In many cases, we then end up with our problems being compounded. Not only does the real problem remain unaddressed, but we end up with even more systems to maintain, distracting and robbing us of productive problem-solving time. We might get so distracted while working on the “solutions”, that we have even more fog preventing us from seeing the real root of the problem. Sure, it might be more fun to adopt an ambitious test automation project, or buy that shiny new tool we’ve been looking at, but if we don’t understand the real problem, we may create an even bigger one.

Check out Tim’s work. I find his posts thoughtful as well as thought-provoking. In an industry that seems prone to silver-bullet-itis, we need more thoughtful process skeptics like Tim, who genuinely want to see the world become a better place.

agile, extreme programming, popular, post-agilism, TQM

Post-Agilism: Process Skepticism

I’m a fan of Agile practices, and I draw on values and ideas from Scrum and XP quite often. However, I have become a process skeptic. I’m tired of what a colleague complained to me about several years ago as: “Agile hype”. I’m tired of hearing “Agile this” and “Agile that”, especially when some of what is now branded as “Agile” was branded as “Total Quality Management”, or <insert buzzword phrase here> in the past. It seems that some are flogging the same old solutions, and merely tacking “Agile” onto them to help with marketing. It looks like Robert Martin was right when he predicted this would happen.

More and more I am meeting people who, like me, are quiet Agile skeptics. I’m not talking about people who have never been on Agile projects before that take shots from the outside. I’m talking about smart, capable, experienced Agilists. Some were early adopters who taught me Agile basics. While initially overcome with Agile as a concept, their experience has shown them that the “hype” doesn’t always work. Instead of slavishly sticking to Agile process doctrines, they use what works, in the context they are currently working with. Many cite the Agile Manifesto and values of XP as their guide for work that they do, but they find they don’t identify with the hype surrounding Agile methods.

That said, they don’t want to throw out the practices that work well for them. They aren’t interested in turning back the clock, or implementing heavyweight processes in reaction to the hype. While they like Agile practices, some early adopters I’ve talked to don’t really consider themselves “Agile” anymore. They sometimes muse aloud, wondering what the future might hold. “We’ve done that, enjoyed it, had challenges, learned from it, and now what’s next?” Maybe that’s the curse of the early adopter.

They may use XP practices, and wrap up their projects with Scrum as a project management tool, but they aren’t preachy about it. They just stick with what works. If an Agile practice isn’t working on a project, they aren’t afraid to throw it out and try something else. They are pragmatic. They are also zealous about satisfying and impressing the customer, not zealous about selling “Agile”. In fact, “Agile” zealotry deters them.

They also have stories of Agile failures, and usually can describe a watershed moment where they set aside their Agile process zeal, and just worked on whatever it took to get a project complete in order to have happy customers. To them, this is what agility is really about. Being “agile” in the dictionary sense, instead of being “Agile” in the marketing sense.

Personally, I like Agile methods, but I have also seen plenty of failures. I’ve witnessed that merely following a process, no matter how good it is, does not guarantee success. I’ve also learned that no matter what process you follow, if you don’t have skilled people on the team, you are going to find it hard to be successful. A friend of mine told a me about a former manager who was in a state of perpetual amazement. The manager felt that the process they had adopted was the key to their success, and enforced adherence to it. However, when anything went wrong (which happened frequently), he didn’t know what to do. Their project was in a constant state of disarray. I’ve seen this same behavior on Agile projects as well. (In fact, due to the blind religiosity of some Agile proponents, some Agile projects are more prone to this than we might realize.) Too many times we look for the process to somehow provide us the perfect answer instead of just using our heads and doing what needs to be done. “Oh great oracle, the white book! Please tell us what do do!” may be the reaction to uncertainty, instead of using the values and principles in that book as a general guideline to help solve problems and improve processes.

I have also seen people on Agile teams get marginalized because they aren’t Agile enough. While I appreciate keeping a process pure, (such as really doing XP rather than just calling whatever you are doing “Agile” because it’s cool right now), sometimes you have to break the rules and get a solution out the door. I have seen people be pushed out of projects because they “didn’t get Agile”. I have seen good solutions turned away because they weren’t in the white book (Extreme Programming Explained). I’ve also seen reckless, unthinking behavior justified because it was “Agile”. In one case, I saw a manager pull out the white book, and use a paragraph from it in an attempt to justify not fixing bugs. I was reminded of a religious nut pulling quotes out of context from a sacred book to justify some weird doctrine that was contrary to what the original author intended.

Here’s a spin on something I wrote earlier this spring:

In an industry that seems to look for silver-bullet solutions, it’s important for us to be skeptics. We should be skeptical of what we’re developing, but also of the methodologies, processes, and tools on which we rely. We should continuously strive for improvement.

I call this brand of process skepticism exhibited by experienced Agilists “Post-Agilism”. I admit that I have been an Agile process zealot in the past, and I don’t want to fall into that trap again. I’ve learned that skill and effective communication are much more powerful, so much so that they transcend methodologies. There are also pre-Agile ideas that are just fine to pick from. Why throw out something that works just because it isn’t part of the process that’s currently the most popular?

Test your process, and strive for improvement. Be skeptical, and don’t be afraid to follow good Post-Agilist thinking. Some good Post-Agilist thinkers are former process zealots who know and enjoy Agile development, but also don’t throw out anything that isn’t generally accepted by Agilists. Others are just plain pragmatic sorts who saw the Agile hype early on, and decided to do things on their terms. They took what they found effective from Agile methods, and didn’t worry about selling it to anyone.

Post-Agilist thinkers just don’t see today’s “Agile” as the final word on software development. They want to see the craft to move forward and generate new ideas. They feel that anything that stifles new ideas (whether “Agile-approved” or not) should be viewed with suspicion. They may benefit from tools provided by Agile methods, but are more careful now when it comes to evaluating the results of their process. In the past, many of us measured our degree of process adoption as success, sometimes forgetting to look to the product we were developing.

What does the post-Agile future hold? Will individuals adapt and change Agile method implementations as unique needs arise? Will “Agile” go too far and become a dirty word like Business Process Re-engineering? Or will it indeed be the way software is widely developed? Or will we see a future of people picking what process works for them, while constantly evaluating and improving upon it, without worrying about marketing a particular development methodology? I certainly hope it will be the latter. What do you think?

Edit: April 28, 2007. I’ve added a Post-Agilism Frequently Asked Questions post to help explain more. Also see Jason Gorman’s Post-Agilism – Beyond the Shock of the New.