Recently, we were lucky enough to be given the opportunity to build a device lab for our department. Stephanie Rieger (@stephanierieger) covers the many reasons why organizations need to get their hands on real devices for testing. As mobile becomes an increasingly important outlet for our content it behooves us to make sure that content actually works on them. Over the years we had picked up a few devices but this was a chance to “do it right” and we jumped at it.
We took a fairly pragmatic stance when deciding what devices we should get for our device lab. We evaluated devices based on the following criteria:
I think we ended up meeting our goals on OS and screen dimensions. Our lab now includes devices running Windows Phone 7, webOS, BlackBerry 5, 6 & 7 as well as various flavors of iOS and Android. If you’re just starting your device lab I would rank devices as so:
I think you can fill in as necessary after that based on your audience. Depending on where you are in the world you may want a Symbian-based or Bada-based device. If you know you have a number of important administrators with older BlackBerries pick up a 6.0 device. Android devices offer a lot of options for testing responsive designs with their varied screen widths and heights. Obviously, you might want to pick up an iPhone for testing its Retina Display. More is better but you don’t have to kill yourself getting the perfect combo of devices. Especially if that’s what’s keeping you from jumping in.
And, remember, you can always check out emulators for any of the devices you can’t get your hands on or you can look at web-based device testings services like BrowserStack. Ryan Seddon (@ryanseddon) put together a good post that covers some of the options in the latter category.
Once you’ve decided what type of devices you want then the obvious next step is getting your hands on some. I think their are four ways to go about this:
Trolling eBay and Mobile Karma should show that you don’t have to break the bank to start building a device lab. Especially if you limit yourself to devices 1-3 from my list above. An iPod Touch (refurbished) from the Apple Store and a Fascinate and Thunderbolt from Mobile Karma are, combined, only $438. And there are no contract costs with those Android phones!
Using your new devices won’t be as simple as turning them on, connecting to a WiFi network, and firing up the browser. All of them would/should have been reset to their factory defaults meaning you’re going to have to activate them but without a cellphone plan. Not every device likes this. For example, with the Palm Pre 2 you’ll have to get it to run in dev mode via entering a code in the emergency number screen. Other devices will have other ways to skip activation (e.g. using the volume keys). You’ll also want to come up with one central account (most likely Gmail-based) that can be used to create accounts across all of the various app stores.
Activation issues can be overcome… just don’t be surprised by this step.
And, please note, you don’t have to provide a credit card when creating a centralized iTunes account. You can download free apps without one on record so you can skip that step if that’s an issue. Then you can use that account for all of your iOS testing devices.
When I asked on Twitter, “Got any questions about setting a device lab?,” the number one question was related to how to share purchased devices with other departments. To be honest, we haven’t gotten that far yet. We’re currently working on how to make sharing within the department easy. After that we can look outwards. I have a feeling we won’t be loaning out devices. Does that sound mean? Sure. Is it more practical for us? Yes. This doesn’t mean that folks couldn’t visit us to test of course. The bonus of that is that they then have some experts to help them troubleshoot any problems.
Another question was “When should we retire a particular device?” At this point I think their is something to be learned from testing even if it’s on an older device (e.g. BlackBerry 5.0). If a design is completely screwed up on an OS like that and your stats show almost no visits from that operating system/browser then it’s OK to ignore but, otherwise, the more you can cover the better. And, again, their is always something to be learned from older platforms.
Unfortunately, devices are only part of the cost in setting up a lab. This is especially true if, like us, you want to set-up a mobile device lab. Where “mobile” means allowing the device lab to easily move around to different cubes or offices. You should think about picking up the following:
We’re still in the process of fleshing this part out so I don’t have numbers but it shouldn’t be overwhelming budget-wise.
Getting devices into the hands of your developers and designers is only the first step in integrating device testing into your website design routine. In order to get folks to buy-in it has to be easy. Very, very easy. This is one of the reasons we want the cart. We hope that, by using the devices in their familiar environments (e.g. their own cubes), it will encourage them to test more. The other point is that what we’re really interested in on these devices are the browsers. In my next article I’ll talk about what browsers you might want to target and how you can use a combination of Adobe Shadow, SHIM, and ipfw to make it dead-simple for developers to robustly test and tweak their mobile designs in those browsers.
Have your own tips for setting up a device lab? Any other questions I can answer? Please submit a comment!