In Part 1 of this series, I introduced some ideas on building a physical test lab for testing mobile apps. In part 2, I expand on those ideas.
Storage
You will want a clean, dry, secure storage area for devices to be kept in when not in use. I’ve used a drawer that locks with toolbox liner at the bottom to protect the cases and keep them from moving around. Here is an example: toolbox liner. The devices are small and disappear easily, so secure storage is a must.
Set up a sign-out sheet with each device noted by OS, version and serial number. Also create tags for each power/sync cable for each device so they can be kept track of if they leave the test lab. I used a label maker for this. There is nothing worse than losing your hard-earned test devices to other groups, or not having devices around when you need them. Cables disappear the most, so you need to make ownership for your lab visible. Think of the washroom key for a service station on the side of a busy road – the small key is often connected to something larger to make it more visible to help prevent it from getting lost. I always chuckle when I see a key on a chain with a hubcap attached, and I try to make my lab cables visible in some easily identifiable way.
Power Station
You will need access to power to charge the devices so they are available for testing when you are. It’s incredibly frustrating when you are getting ready to test on a device and find out that the battery is dead. If you can’t get power into your device storage area, set up a charging station in the lab.
Set aside counter or desk space near a power source, and add the following:
- at least one power bar to plug in multiple devices
- several power cables for the devices
- toolbox liner or something similar on the surface to keep them from moving around
Lab PCs
You will need computers to manage the devices for testing, troubleshooting and other tasks. Consider at least one machine for each platform family. Here are four popular platforms. Your team will have their own mix of devices that they are developing for. Get the most powerful workstation you can afford for each platform; mobile app development tools can be large and complex.
iOS (Apple)
You will need at least one Mac for emulators and device management, and pulling screenshots and other information from the devices. Install XCode for emulator use, Also consider installing the Network Link Conditioner to help simulate different network connection conditions with an emulator. Install the iPhone Configuration Utility for getting stack traces, managing licenses, etc.
Android
You will need at least one machine with the Android SDK installed for managing Android devices and emulator use. Many developers use this within the Eclipse IDE, so talk to them about the correct combination of tools for your shop. The operating system doesn’t matter for Android development.
Windows Phone
If you are supporting Windows Phones, you will need a Windows machine with at least Windows 7 and the Windows Phone SDK and related tools.
BlackBerry
The new BlackBerry SDK supports several languages and implementations, so you’ll need to find out which version to install, and what other libraries need to be installed. I don’t think OS matters, but I have seen installation instructions recommending at least Windows XP or Vista.
Wireless Broadband
Data plans or other contracts for monthly wireless broadband access are important to test. I would get a reasonable plan for each device, from at least two local telecom providers. Try a larger provider, and a smaller provider (like a discount provider) to get more variation in wireless broadband technology. There is a huge diversity in what different carriers use to deliver this to the end user, so try to get some variation
Other
You will need internal test email addresses, and phone numbers for each smartphone for testing. Also get at least one developer license for each platform your team supports for your test lab. You will need test ids with the App Store on Apple for making purchases, or installing apps from the app store. You will likely need to do this with other stores as well such as Google Play and others for each device type you have.
Looking for ideas on how to test mobile devices? Check out my I SLICED UP FUN mnemonic to help generate ideas.
Nice article. One thing I would add…
For mobile devices with Wifi I like to set up a Linux based Wifi router. I then flash the router with DD WRT (http://www.dd-wrt.com/site/index) or something similar. You have to be sure the router you have is supported and follow the instructions very carefully. Flashing the router incorrectly can turn your router into a brick. i.e. useless as a router.
The next thing you want to do is set up a proxy on the router so that all wireless data going to the router goes to a computer connected to the router. Finally, you want to have something like Fiddler2 or Charles running on the computer, listening to the port you set for the router proxy.
The end result is that the data will go from the wireless device, to the router, to the computer and into Fiddler2. You can than save the session. Later you can take the session from Fiddler2, parameterize it and use something like curl to play it back. Now you have automated device testing using real life data.
Thanks Darrell, great idea to share.
What about products like http://www.soti.net/Mobicontrol/ which allow you to “remote in” into handsets which could be sitting pretty much anywhere?
I know there are companies out there that run test automation against the remoting window, exercising/testing their handsets that way from their workstations.
Handsets are able to be remoted into though USB cable, Bluetooth, WIFI and 3/4G connections.
I am not in anyway connected with SOTI, just interested in the capabilities and the potential impact to testing Mobile Apps.
Hi Roland, thanks for the comment. Those services are good for basic functional testing, but they don’t really help with the mobility side of things. If you look at the I SLICED UP FUN coverage models, those services only help in some of the areas that can be important for mobile app testing: http://www.kohl.ca/articles/ISLICEDUPFUN.pdf
They have their place to help us extend our reach, but I wouldn’t count on them as a primary test lab.