The 10 Commandments of Usability Testing for Mobile Apps

Mobile usability testing and quality assurance might not carry the same guts or glory as designing and developing apps, but testers are the unsung heroes of the mobile app world. When usability testing is done right, the finished app is a pleasure to interact with, and the user will look forward to opening it up again and again.

On the other hand, inadequate UX testing and QA can hinder your mobile app success and spell doom for your final product. Bugs, crashes and other UX fails will ensure that the app lands right in the wastebin.

To make sure you’re giving your users the awesome experience they crave, make sure you follow these 10 Commandments of Usability Testing:

UX Testing Commandment #1: Thou Shalt Not Skimp on Testing

Petre Moroca is a mobile usability testing expert at DICE https://dice.fm(@dicefm), an app that allows users to find concerts, festivals and events with no ticket booking fees.

“It’s so important to know your target audience and the devices and screen sizes that we intend to support,” says Moroca. “For example, I start by verifying the oldest device and OS combination I have available, and then I work my way up to the latest mobile device/OS combination. That gives you a pretty clear picture of usability issues that users on the older devices would experience.”

“Whatever you do, do not assume that you know how your app will look on a particular device,” Moroca adds. “I would rather know with certainty that a feature works as expected on all available devices. Even if that means pushing back on deadlines, it’s just not worth going with the uncertainty.”

Sometimes that might mean having an uncomfortable conversation with your PM, but it’s worth it. “I have seen how people skip checking older devices when the project manager is breathing down their necks to release the app,” says Moroca. “Don’t cave into the pressure! It’s better to take some heat from your project manager than to take it from users who leave bad reviews on your app.”

UX Testing Commandment #2: Thou Shalt Test For Accessibility

Sharon Rosenblatt reports to us from Accessibility Partners, LLC(@Access_Partners), which provides accessibility consulting to companies looking to make their products more usable for people with disabilities.

“As we test usability within my company, our focus is on accessibility for people with disabilities,” says Rosenblatt. “It is important, and in many cases legally mandated, that technology must be accessible for people with disabilities.”

In case that last sentence made you panic, read up on our beginner’s guide to accessible mobile UI design — chances are, you’re already covering off on some of these points in your mobile usability testing.

“We test apps with engineers that have disabilities: whether it is blindness, poor vision, deafness, or an inability to use or hold mobile devices, we try to check for everything,” says Rosenblatt. “We also encourage developers to use assistive technologies, like screen reading software, magnifiers, speech recognition like Siri, and more, to emulate the experience of a user with disabilities. After all, an app can’t be accessed if it can’t be used by the large demographic of people who identify as having a disability.”

Check this infographic. 

UX Testing Commandment #3: Thou Shalt Obey the Law of the Red Route

Bryan Clayton is CEO of GreenPal (@YourGreenPal), a sort of Uber for lawn care.

“The Law of the Red Route is based on a heuristic that comes from a mass transit bus in London. The bus is only allowed to drive on streets with a red stripe down the middle. This is called the red route, and it allows all other traffic to get out of their way — but it’s the only way that they can go on,” says Clayton.

For those on this side of the Atlantic Ocean, picture any city you’ve been in that has had a trolley car (if you’ve ever been to the Jersey Shore, you might have heard the iconic “Watch the tramcar please!” recording alerting you to get out of the way of the tram’s path on the Wildwood boardwalk).

“This analogy can be carried over into app design, because we must design a red route in our apps as the one path that the user can take — and none other,” says Clayton. “We then must design our apps around that red route by killing all of the ancillary routes that a user can take.”

One of the best ways to make sure that the user stays on the Red Route is to nail down the user onboarding experience (here are seven apps that excel at user onboarding, and what they do right). By thoroughly QAing the user journey and onboarding experience during usability testing, you’ll be eliminating a lot of potential end-user confusion, while also ensuring that you’re getting the desired action from your user — which could include following a CTA, purchasing an add-on, posting a review and other paths that ultimately benefit the app.

UX Testing Commandment #4: Thou Shalt Use the Right Tools

There seem to be two problematic philosophies when it comes to investing in new tools for your business. Sometimes, you run across project managers who are hesitant to invest time or capital expense into purchasing and learning a new tool that has the potential to streamline and improve the team’s workflow and output exponentially. On the other side of the coin, you’ll run into managers who want to try every shiny new tool that comes out without considering whether it will really improve their process, or just be a burden on the designers, developers and testers.

The right approach, of course, is somewhere in between. Performing mobile usability testing on a brand new app using outdated tools isn’t just a waste of your time, but it can result in a poor experience for your users (and ultimately, a loss of potential business). It’s imperative that project managers check in with their team to evaluate any pain points in the usability testing process, and seek to remedy that by changing the process or considering a tool to speed up or automate a particular task.

Don’t know where to start? Check out these 11 tools that take the sting out of mobile usability testing, all vetted by mobile app experts.

UX Testing Commandment #5: Thou Shalt Test On Actual Devices

There’s a corollary, of course, to using all the right tools during testing: you also shouldn’t let emulation and automation completely take over your testing process. No piece of emulation software can perfectly replicate the experience of interacting with a touch screen on an actual physical device running a certain operating system, and no automated tool can replace the discernment of the human brain (at least until we have replicants as good as the ones in Bladerunner).

With that said, of course the analog “meatspace” world of brains and smartphones and tablets has its own limitations. It’s impossible to have access to every device in circulation running every possible operating system — especially if you’re developing for Android, which doesn’t push OS updates across all devices like Apple does, and has multiple manufacturers pumping out devices with a wide variety of specs. (Unfortunately, Android fragmentation continues to be one of thebiggest mobile UI design challenges that still don’t have easy answers). Even if you had budget to afford such a huge pile of devices, testing on each of them would be a laborious time sink.

So what’s the compromise? If you work at an agency, one way to overcome this hurdle is to send a spreadsheet to everyone in the organization, where they can volunteer their phone for testing. In this spreadsheet, break out columns for the type of device and the OS it’s running, and then select a decent sample of older and newer versions for testing.

If you’re going it solo, you might want to invest in older used devices you find for cheap on sites like Ebay or Craigslist. Alternately, you could make it known to your friends that you’re collecting old devices, and are happy to “rehome” their smartphones and tablets once they’ve moved on to greener pastures. Again, select a handful of devices from the collection you’ve amassed for testing — better yet, enlist your friends as beta testers, and collect their feedback! Which brings us to our next point…

UX Testing Commandment #6: Thou Shalt Honor Thy Beta Testers

Just as writers should always get a second set of eyes to edit their work, so too should mobile usability testers enlist the help of beta users. No matter how sharp your UX testing skills, everyone’s eyes begin to glaze over after hours of pouring over the same code and UI, and it’s crucial that you see how your app performs in the hands of a diverse array of users. The more, the better.

UX Testing Commandment #7: Thou Shalt Document

Documentation isn’t the sexiest aspect of mobile app development, but it’s still massively important to your usability testing process. Without adequate documentation, you run into nightmarish versioning errors (more on that later), team members who are confused when a certain UI feature changes and countless communication issues that could be solved with a simple, “It’s right here in the changelog.” Speaking of change logs…

UX Testing Commandment #8: Thou Shalt Manage Change

One of the biggest mobile UI design disasters you can run into during usability testing is a big old versioning error. You see an issue, fix it in either the prototype or the code, and fail to communicate that you have made the change. Then, a developer submits a new version of the code, or a designer hands you an updated design, without the error that you fixed. Best case scenario, you remember the change you just made and remake it in the new version, documenting properly this time. Worst case scenario, you forget the whole thing happened, and then that bug remains in the UI, breaking the user experience only after your app is published.

Lock down your process, keep your changelogs up to date and try your best to avoid having multiple versions of your app or prototype in circulation at once. While it might seem like a timesaver when a deadline fast approaches, you might only be making yourself more trouble in the end.

UX Testing Commandment #9: Thou Shalt Iterate

If you read our article on the workflow management tools your mobile app business can’t live without, you’ve read the word “iterate” enough to make your eyes bleed. Modern app development hinges on the concept of iteration — that is, revising your product until it approaches (and eventually achieves) perfection in a rolling process, rather than sticking to a rigid linear project management process that doesn’t allow for changes up or downstream.

The whole point of mobile usability testing is to make the end product as polished as possible. It might take you a few cycles to hammer out every kink in the UX, but that iteration is one of the most important ways you can ensure a magical experience for your users.

UX Testing Commandment #10: Thou Shalt Test Thy Prototype

You don’t have to wait until after the developer puts the finishing touches on her last line of code to start mobile usability testing — in fact, you can test your user interface design long before it lands in the hands of your dev team. UX testing is one of the many ways smart businesses are using interactive prototypes to streamline their processes and get invaluable feedback on their most disruptive mobile app ideas.

[Proto.io is a mobile app prototyping tool used by entrepreneurs and startups to create fully-interactive realistic prototypes that look and feel like real apps.]

Note: We at Inc42 take our ethics very seriously. More information about it can be found here.