Session-Based Testing Using Instant Messaging

Here’s a scenario: a programmer friend is creating a new web-based product for his company. He wanted someone to augment the existing testing that was already underway, and hired me to add more firepower. It is a distributed environment, which means that everyone in the office telecommutes. We decided to start with me doing an initial session, via instant messaging. We blocked out a time for the session, determined a focus (“mission”) and I started testing.

As I tested, I typed what I was doing in the instant messaging (IM) client, along with questions, concerns, crashes and bug ideas. He followed along through the application as I tested, asked clarifying questions as I discovered problems.

After ten minutes, I had generated enough ideas for that mission. He logged 6 bugs in his fault-tracking system. I had discovered 5, he had discovered 1 himself while following me along and monitoring the server and database.

I’ve done session-based testing over IM quite a lot with different distributed projects, like open source projects such as Watir and Session Tester, and with some of my clients that use telecommuting, or are completely virtual. We’ve used and adapted some of the ideas from SBTM to help provide more structure around the actual test sessions.

Using session-based testing with IM seems to work best in pairs. Once a focus and rough timebox are determined, test setup is out of the way, both pairs initiate a session through an IM client. One drives by testing, and the other saves the conversation locally, and creates notes, logs bugs, investigates underlying behavior, etc.

This style is natural (we communicate a lot with IM clients anyway) and doesn’t interrupt the natural testing flow if one person acts more as the primary test idea generator/executor and the other acts as the primary scribe. If you want to add some variation to your testing approach, give it a try.