How many devices do you use in testing?

[Ronan Healy]

I’m guessing it depends on what type of testing you are doing and what platforms you are using.

This could turn into a game of one upmanship here but how many would you use generally?
How many is too many or is there never enough devices to test on?

I thought this image from html5rocks was quite good. Who tests on A Game Boy???

[Kasper]

Most of the times I just work my laptop and one or more (virtual) servers / clients. But I did test in some hardware dependent projects.
Test I did for a mobile operator pretty much looked like the image, but I also tested several types of ATM and several types of selling stations for a lottery.
It all depends on the work at hand.

[Michael]

[Comment Removed by Admin] See Charter

[Kim]

Generally I am testing on a windows machine and try to cover several browsers.
For some testing activities we also use our own mobiles or a tablet.

This is far from ideal at the moment. I am dreaming of a cross device test lab like in that image. 🙂

[Simon]

I tend to pick my testable device list based on real user numbers. For example, 33% of users use Windows 7 / Chrome 42.

If testing a site that’s Google Analytics (GA) enabled then it’s extremely useful to check GA and make a call on the device combos to test on. This means checking desktop, tablet and mobile device combinations and ranking them accordingly.

It’s also worth considering the following.

If the device is desktop/laptop:

– Operating system e.g. Windows 7, Windows 8.1, Apple OS X Mavericks, Apple OS X Yosemite
– Browser type and version e.g. Internet Explorer 9, Firefox 33, Safari 8, Chrome 42
– Perhaps device make/model e.g. Dell Inspiron 7200, Apple 13″ MacBook Air

If the device is tablet/mobile:

– Device make/model e.g. Apple iPhone 5S, Apple iPad Mini, Samsung Galaxy S V, Microsoft Lumia 640, Sony Xperia Z3
– Operating system e.g. iOS 8.3, Android 5.0, Windows Phone 8.1
– Browser type e.g. Safari, Chrome, Firefox, Internet Explorer, Samsung browser
– Perhaps device orientation e.g. Portrait, Landscape

Of course a line has to be drawn somewhere and that’s often difficult to draw. Sometimes it’s a case of ‘well what physical devices do we have available to hand from our device locker’. Alternatively, take a % of users that you are willing to support i.e. we’ll support 80% of users so let’s cut off the device list at 80%.

Avoiding virtual device emulators helps as nothing beats using real devices to test new features on. Of course this does require continual investment, to purchase each device keeping up to date with the latest based on those user numbers.

[Ronan Healy]

Came across this blog post recently about Facebook mobile testing lab. The lab has about 2,000 devices in total on 60 raqcks with 32 phones to each rack and each rack having it’s own Wifi.

I think that is probably the largest number of devices used for testing.

[Archana]

I usually test on my Windows laptop and use responsive mode of browsers to simulate multiple mobile devices.

There are also websites with a lot of devices available which you can use for testing on multiple devices.

[Author]

Posts