Exploratory Testing

Exploratory Testing is an effective way of thinking about software testing that skilled testers can use to think of techniques to find important bugs quickly, provide rapid feedback to the rest of the team, and add diversity to their testing activities.

To learn more about Exploratory Testing, check out James Bach’s site for articles. Cem Kaner has also written about it. Either of those sources can explain it better than I can.

I sometimes get confused looks from some practitioners when I tell them that I’ve found Exploratory Testing to be effective for my own testing. Some of the confusion may come from not knowing exactly what Exploratory Testing is.

I’ve worked as a testing lead on projects and have directed others to do Exploratory Testing to complement automated tests, and have sometimes met with resistance. Those who resisted later told me that they weren’t used to finding so many defects when they tested, but still were uncomfortable doing unscripted testing even though it seemed to be more effective. When I repeat what James Bach says, that “…testing is an interactive cognitive activity” and I value them for their brains and expertise, they are smart testers who can add a lot of value and I am pleased with the results, it’s rewarding to watch their confidence grow.

Some of the confusion may come from working in an unscripted environment when one is used to following a script. When confidence begins to displace confusion, testers often get more creative, they seem to improvise more when testing, and they seem to want to collaborate and communicate more with the rest of the team. When they are finding important issues quickly, sharing them with the developers and getting rapid positive feedback on their own work, it seems to help build team cohesion as well as individual confidence.
Here’s where I think another source of some of the confusion is. Some people seem to think that you can only do Exploratory Testing on software you haven’t used before. At least that’s the impression I get. If I say I will do Exploratory Testing on each build after the automated test suite runs, some people act puzzled that I would be doing Exploratory Testing on a program I am already familiar with.

As a software tester, I can still explore, inquire and discover testing software I’m familiar with. If we think about it, that’s often how scientific research works. Scientists deal with the familiar and look for patterns or occurrences that don’t seem to fit known models. Reviewing the familiar to verify that the models still hold true is common. Discovery can readily come from exploring something we are familiar with. A known behavior might change under certain conditions that we haven’t seen yet. We may try a new experiment (test) in a different way that yields results that we haven’t seen before.

Exploratory Testing isn’t always working as a discoverer working through a program for the first time. Maybe we are thinking of an explorer analogy like Lewis and Clark when we should be thinking of a scientific method exploration analogy. Sometimes Exploratory Testing can be like a voyage of discovery charting the unknown, but it is very often like a scientific experiment where we are exploring changing variables based on observed behavior.

The pursuit of knowledge and creative problem solving are facilitated with Exploratory Testing. Scripted testing or Directed Observation is a common “best practice” in software testing. How often do we miss problems because we direct what we want to see in a program and miss the obvious? Exploratory Testing is one way to help test with diversity and look at the familiar in new ways.