David McFadzean and I wrote an article for Better Software magazine called The Software Development Game, which was published in the September/October edition. We describe applying game-like structures and approaches to software development teams. I’ve been asked for another article on applying game-like approaches to testing, so look for more in this space.
In the meantime, here are some of my current thoughts.
How is software testing like a game? If we think of software development as a game, or a situation involving cooperation and conflict with different different actors with different motivations and goals, software testing fits as a game within the larger software development game (SDG). While the SDG focuses more on policy, practices and determining an ideal mix of process, tools and technology, software testing is often an individual pursuit within a larger game format. There are two distinct styles of playing that game today: scripted testing and exploratory testing.
In the scripted testing game, we are governed by a test plan and a test case management system. We are rewarded for going through a management system and repeating test scripts and marking them as passed or failed. We are measured on our coverage of the tests that are stored in this test case management system. It is very important to stakeholders that we get as close to 100% coverage of the tests in that system as possible. This is a repetitive task, in a highly constricted environment. Anything that threatens the test team goal of high coverage of the tests within a test case management system tends to be discouraged, sometimes just implicitly. The metrics are what matter the most, so any activity, even more and better testing will be discouraged if it takes away from reaching that objective. “You, tester are rewarded on how many test cases you follow and mark as pass or fail in the system. If there is time after we get this number, you may look at other testing activities.”
In the exploratory testing game, testers are given some degree of self-determination. In fact, the latest version of the definition of exploratory testing emphasizes this:
“a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”
The game player in exploratory testing is rewarded on the quality of their work, their approach, and the quality of the information that they provide. Exploratory testing works by being adaptable and relevant, so traditional ideas of metrics are often downplayed in favor of qualitative information. If a tester changes their mind when they are working on an assignment, and they have good reason to do so, and can make their case defensible, they are rewarded for changing things up because it helps make the product and our approach better. Coverage is important, but testers are rewarded on using multiple models of coverage (and discovering important new ones) rather than following the regression tests and counting that as complete coverage.
When I think about exploratory testing, I am reminded of the game Nomic where changing the rules is considered a valid move. Instead of a group effort like Nomic, a tester has the freedom to change their own course, and to help improve the approach of the overall team by demonstrating excellent work. “You, tester are rewarded by excellent testing, and your skill and ability to find and report important problems. We don’t care if you work through one test, or a thousand tests to get that information. The information you provide and how it improves the quality and value of our products is what we value.” It is deeply important for exploratory testers to be able to adapt to changing conditions, to uncover important information to help the team create value with their product, and for many, to get endorsement and the respect of their peers.
Now think of game play activities. Both scripted and exploratory testing approaches involve repetitive work. In the gamification space, we can explore these activities, and how we can apply game concepts to enhance the work. One area we can explore are video games. Imagine we are are playing a massively multiplayer online role-playing game (MMORPG). Some players like to perform repeated tasks that are unrelated to individual and shared game objectives. They like to repeat tasks to unlock features related to their character, such as acquiring new outfits or accessories for their character.
Other players are very achievement goal oriented – they try to reach individual goals and gain more points and unlock more achievement-based features in the game, and learn that when they co-operate with others, that they can achieve even more. One player type gets rewarded more for repeated individual tasks, while the other gets rewarded for trying to create achievement or quest value for themselves and others. The two types blend, as any player will have a subgoal of changing the appearance of my character, or acquiring accessories or currency to enhance quest or other achievement game play. People who are less adventurous and are more caught up in acquiring personal character attributes will also take part in quest events.
If you play these games and try to collaborate, you will often find other players who are at different points on this spectrum. It can be frustrating when your team members are more interested in building up their own character’s numbers than in helping the team as a whole. For example, if you are off on a raid, and one or more team members constantly stop to pick up health or currency points, your team will be let down and may have to regroup. It can be difficult to work together, since the perceived value of the game and the reward structures don’t match. One group want personal metrics and accessories, while the other care about non-character pursuits and the respect of their peers more.
In the software testing game, it is difficult to try to play an exploratory testing game in a system that is rigid, restrictive, and rewards testers for playing a scripted game. Conversely, those who are used to being rewarded on acquiring coverage counts in test case management systems can be very threatened when they are now rewarded on the quality of their testing skills and the information they provide. Testing tradition and common practice tends to think of testing as one sort of game, of which a test case management system is central. What they fail to realize is that there are many ways of playing this game, not just one. I want to explore this other alternatives in more detail through a gaming lens.
Testers and test managers, there is a lot more to our world than scripted testing and test case management tools. They are part of an ancient game (from at least the 1970s), which is struggling and showing its age in the face of new technology and new approaches to work that mobile technology and other styles of work are bringing. Modern testers work hard to play a game that fits the world around us. Are you measuring and rewarding them for playing the right kind of game, or the one you are most used to? Like the video game character who has gathered metrics and accessories, are you gathering these at the expense of the rest of the team’s goals, or are these measures and rewards helping the entire software development team create a better product?
Watch for more in this space as I explore software testing from the perspective of gamification, game theory and other game activities more deeply.