Learning Testing Heuristics

When I was in junior high school, one of my tasks was to learn metric conversions. My science teacher at the time took this task seriously. To help us remember the kilo, hecto, deca, deci, centi, milli prefixes to meter, liter and gram, he had us memorize a mnemonic. The mnemonic looked like this: KHDMDCM, and there was a sort of song that went through it: “King Henry Danced Merrily…something something something”. Trouble was, I barely had the mnemonic memorized in time for a pop quiz, and I couldn’t remember what each letter stood for. I ended up only being able to write the King Henry song out on my test paper and handing it in. I couldn’t do the required conversions such as decimeters to meters, or kilometers to centimeters. My teacher gave me a cold look when he handed the exams back, and admonished me to stop fooling around and actually start trying. Trouble was, I was trying. I just couldn’t get that stupid mnemonic to make sense in my brain.

At home that night, I sang the King Henry song over and over, repeating what each letter stood for. It was hopeless. I had to constantly look at my cue cards.

My older sister happened along and asked: “What on EARTH are you doing?”

I explained.

“Don’t do it that way,” she said, and quickly sketched out a diagram.

She wrote out all the measurement prefixes from largest to smallest, with “meter, liter, gram” in the center, and drew an arrow going up to the left on one side, and down to the right on the other. I stared at the diagram for a few seconds, and she had me reproduce it cold. Within minutes, I had the image memorized. She then taught me how to use it for conversion. It was a snap. On my metric conversion exam later that week, I ploughed through the questions with ease. “Forget King Henry,” was my thought, “this works way better.” And, forget him I did.

However, over twenty years later, I can still scratch out that conversion diagram and convert deciliters to decaliters, or kilograms to grams on the back of a napkin with relative ease.

“Different people learn differently” my father once told me. His first career was a school teacher, and over twenty five years he taught hundreds of different students. He was surprised by little, and was quite pragmatic. If colors helped you learn and remember, use colors. If mnemonics and little songs worked, try them. If images worked, go with the images. Try to involve most of your senses if you want to remember something, he said. Furthermore, if the system you are trying to use doesn’t work for you, make sure you clearly understand the concept, and then create your own. (These didactic sessions usually came from the depths of a newspaper, my Dad gazing at me over the rim of his glasses with CBC Radio blasting in the background.)

When I started working with James Bach, he talked about using mnemonics for testing. I was familiar with, and had used most of his published mnemonics, but I had never memorized them. I used them more for analysis prior to testing. They seemed suited for analysis and test planning, and I was able to use them to generate extremely effective test strategies and plans.

Then one day I saw James at work, testing software. He was thinking out loud, using test heuristics, rattling off testing mnemonics, and simultaneously designing and executing tests under each letter of a mnemonic in real time. He would pause periodically to point out his findings or jot down notes. He would generate and execute many test ideas in different categories in a short period of time. This resulted in surprisingly thorough testing, with test ideas no on else had thought to try. James’s testing provided a veritable treasure trove of bugs within minutes. Whoa.

James was using SFDPOT (San Francisco Depot), and he would say the first letter of the mnemonic out loud like this:”S – SYSTEM!” Then he would mutter and brainstorm and figure out ways to test the system part of the application. He might alter the files in an installed app, or remove dependencies. He would use system tools to monitor memory use and CPU usage for example, while testing. If he saw that memory use jumped up when doing a particular action, he would repeat it rapidly to see if memory use kept increasing, and if it fell back down to where it was before. He would uncover installation issues, improper dependencies, memory leaks, and all sorts of things in a very short period of time.

Once he was satisfied with those tests he would sound out “F- FUNCTION!” Then he would generate a few ideas of functional tests, and he would examine areas that seemed out of the ordinary and push them. If he found an awkward workflow, he would note it down. If he also noticed memory leaks or CPU usage spikes, he would note what he was doing that caused them. He would continue on through Data, Platforms, Operations and Time, then he would stop testing, consult his notes and start logging bugs.

I’d never seen anything like it, and I knew right then I had to master his approach. But memorizing mnemonics was a bit of a blind spot for me. I needed to figure out how to memorize those mnemonics, so I too could quickly slice through a testing problem (any testing problem) with thorough ease. However, the spectre of King Henry loomed above me, reminding me of my early mnemonic memorization failure.

James peered down at me over the rim of his glasses: “You need to memorize testing mnemonics.”

His tone was casual, but firm. To him it was like telling me I needed to learn to cross the street. Obvious.

I complained about how I often found it hard to memorize mnemonics. Then, much like my sister did back when I was in junior high, James told me not to do it that way anymore.

“That’s because you are memorizing someone else’s. You need to create your own. If they are yours, you’ll remember them.”

He then spent time with me helping me practice memorizing a mnemonic, using the mnemonic to test in real time, and then encouraged me to personalize it and make it my own.

It turned out I could learn new tricks. After all, I had used mnemonics as a student that were quite useful and effective. Who can forget SOHCAHTOA, an effective way to remember how to calculate the sine, cosine and tangent of an angle? I remembered memorizing that mnemonic and using it to great effect. “Soak-a-toe-ah” I repeated in a unique cadence, and imagined a big toe in a wash basin. The imagery and the rhythmic cadence of “soak-a-toe-ah” helped me remember the mnemonic, and practice helped me remember the application. I could memorize testing mnemonics just as easily, and banish King Henry from my memorization nightmares. It turns out that motivation and seeking results can go a long way to memorizing a mnemonic.

I was faithful to James’s advice, and mnemonics have served me well. While I use other people’s mnemonics in my own work, I find I return to my own more often. I’ve added mnemonic memorization and creation to my testing thinking repertoire. I use ones from others that are particularly effective, and I create my own as I need to as well.

When I teach testing techniques, I find, as my father did many years ago, that some thinking models appeal to different people in different ways than others. Mnemonics and other thinking aids don’t need to be lists, they can be imagery, visual patterns, sounds, or anything else that helps you remember to be more thorough. Any thinking model that gets the job done is valid. Furthermore, using different thinking models along with memorized test heuristics helps me analyze them, enhance them, and learn more about my own work. It also means my implementation looks different from someone else’s, based on my own experience, insight and skill. Now, I don’t just use test mnemonics to help plan, I use them during test analysis and planning, and during test execution.

EDIT Update: I SLICED UP FUN is a popular mnemonic I created. I developed it first for myself, then shared it with my team, then with everyone else. It has become quite popular on mobile software development projects the world over.

Are you curious about using mnemonics in your work? If so, imagine me peering over my glasses at you, expecting you to make them your own.