How to Work Virtual Reality into Mobile App Prototyping

In the 90s, virtual reality was going to transform the world. Movies like The Lawnmower Man, The Matrix, and Existenz created an image of VR as an almost supernatural force, able to create alternate worlds filled with danger, and powers that transcend normal human abilities. There were even a few early, consumer-grade VR tools like the Nintendo Virtual Boy and Sega VR Headset.

And then it all seemed to disappear.

In a lot of ways, those early days were like childhood for VR. We had big dreams (and fears), and pretty vague ideas about how it was going to work, and what it was actually good for. VR tools were crude and expensive, and the world wasn’t ready. It wasn’t just a matter of technology — we were just starting to understand how to develop an intuitive UI (and UX design was still very much in its early stages).

Now, however, we’re ready.

VR has exploded over the last few years. You can turn your phone into a VR tool with a $15 Google Cardboard display, or even do it yourself with some basic craft tools. Even powerful immersive headsets like the Oculus Rift, or HTC Vive cost less than a high-end Apple MacBook.

And unlike in the 90s, consumer demand is huge. According to Zion Market Research, the global virtual reality market will continue to expand at an astounding compound annual growth rate of 54.01% through 2022, when it will reach a value of $26.89 billion. Another study showed that VR app downloads from Google Play and the App Store increased 276% in 2016.

However, for developers and designers, it’s still the early days. There are plenty of SDKs and design tools, but for mobile app designers who want to whip up a functional virtual reality prototype without diving into the code, there wasn’t a lot out there — until our VR Components library, that is.

Where Mobile Virtual Reality Apps Are Now

There are two basic ways of doing virtual reality: with three or six degrees of freedom. Three degrees of freedom is the way virtual reality for Android and other mobile devices works now. The user can look in any direction, and interact with objects in the virtual space they’re standing in. Some dedicated VR controllers allow six degrees of freedom, meaning the app can also track user movement in a room, adding realism to interactions with the virtual space.

Why can’t smartphones do six degrees? While your phone is good at tracking changes in direction, it’s not able to track movements in physical space all that accurately. In fact, at the moment, VR controllers need a dedicated room with external sensors to achieve six degree VR, along with an external device to handle the processing.

Most mobile app prototyping focuses on three degrees for a few reasons:

• It’s widely available, so there’s a much bigger market.
• You can do it anywhere, not just in a specially configured room, tethered to a computer.
• You don’t need special (and costly) gear to use it or test it.

It’s hard to tell exactly how virtual reality will develop, but there’s a good chance that six degrees will remain a narrow niche in VR. One of the great things about VR is that it lets you escape the constraints of your physical environment, a benefit you lose if you have to configure an entire virtual space.

Virtual Reality App Development

The closer your prototype resembles your end product, the better. Simply because the cost of making changes to a product when it’s fully developed, is much higher than making changes to a prototype. Your app’s prototype should simulate the functionality, UI, and user experience of your app as closely as possible. In that way, the feedback you get is closer to what you would expect to get if you tested with users when the app was ready.

Having said that, your UI libraries should be up to date for Android or iOS. There needs to be support for gaze control so you can interact in complex ways with the app, even if you’re only using a smartphone as a controller. Interactions and animations need to be just as good, so you can nail the design elements in prototyping.

In a lot of ways, it’s even more important to get virtual reality prototyping tools just right. For one thing, you can’t fall back on tried and true design conventions that apply in mobile app prototyping. A simple interface that might look great on an Android screen isn’t necessarily going to translate when you have 360° and three axes to work with. It’s a good bet virtual reality apps will stoke the skeuomorphism vs. material design debate, and best practices will change.

You also can’t fall back on your instincts and a rough sketch. You need to simulate how it works in as much detail as possible, so you can get it right before you invest the time creating the final product.

Additionally, creating graphically rich, immersive environments is going to be much more important than it typically is in mobile app prototyping. You’re asking users to strap a screen to their face, which is inconvenient — they need to get more out of it than the exact same buttons doing the same things, but now floating in space.

A lot of what makes virtual reality interesting will hinge on interactions and microinteractions. If an object glows or casts a shadow on a flat screen, it’s just an effect. It can interact with other design elements, but it’s still somewhat removed from the user. In virtual reality, the user can actually interact with light sources. You can use distance to indicate priority, or have the user glance in different directions to complete different tasks.

This creates the opportunity for innovative designers to delight users in genuinely new ways. You could use actual simulated spotlights to highlight important tasks, or change the design of a room to respond to the user’s mood, or the nature of a particular task. Animated elements can shake to create a sense of urgency, or move towards the user to draw their attention to new information.

But you can’t test these elements together without prototyping user controls. What might seem like an intuitive user flow when you’re interacting with a screen might be an absolute nightmare when the user is controlling everything with their gaze. You need to be able to actually simulate the way the user interacts with the environment, or you could end up with an unfortunate wake up call later in the project.

And you need it all to work in a way that meshes with your 2D app development workflow. You shouldn’t have to learn a completely new set of tools, or mock up two different prototypes of an app if you want to support both VR and conventional interfaces — you should be able to design and test both within the same framework.

How to Prototype VR Apps in Proto.io

Proto.io has developed a VR prototyping device library that empowers designers and developers to simulate their own VR experiences, using the same intuitive Proto.io workflow they depend on for traditional mobile app prototyping.

Users can upload 360° photos and design walkthroughs in minutes. Proto.io Asset Flow integrates seamlessly with industry-standard tools like Sketch and Photoshop, making it every bit as easy to simulate a walkthrough in a fantastic virtual world as it is to tour the inside of a house.

Our VR Components library also makes it easy to simulate menus and other gaze-based controls, add animations and sound, and tweak your layout to make the best use of virtual space. And you can preview your prototype on any device that runs the Proto.io app, giving you incredible flexibility. For example, you can use your MacBook to build a scene on the web app, then test it and tweak the virtual reality version for Android, then demo it on Oculus Rift or another high-end VR headset.

Getting Started With Virtual Reality Apps

We’ve put together a few tutorials to get you started in VR. Whether you’re an experienced developer looking to get into VR, or completely new to VR prototyping, creating a basic VR environment is a great first step. In about three minutes, you’ll learn how to load up the VR container, add an image, and view it on your device.

Once you’ve done that, check out our on-demand webinar, VR prototyping without leaving Proto.io. First, we’ll show you how to make your first interactive walkthrough, using stunning 360° images. Then, we’ll teach you how to make a game menu, with buttons that expand, glow, and make sounds when you press them (you can also watch the game menu tutorial separately). In the process, you’ll learn a lot of skills that are useful for building other virtual reality apps, including:

• Making VR buttons that react to user gaze.
• Triggering scene changes.
• Using flat images and beacons to indicate different types of scene changes.
• Incorporating sound into your VR apps.

If you’re new to Proto.io, you’ll also get a feel for how everything works together. Although VR Containers are special in some ways, we’ve kept the workflow as consistent as possible. If you have experience with building apps, and you’re the type who only uses a manual as a last resort, it should get you on your way. Otherwise, our User Guide should have everything you need to know.

For more design and UI insights, we highly recommend browsing The UX of VR. There, you’ll find plenty of great videos, articles, books, slide decks, and other resources to make you an expert on Virtual Reality user experience.

This post first appeared on Proto.io blog and has been reproduced with permission.

Note: We at Inc42 take our ethics very seriously. More information about it can be found here.