Virtual Reality

Solving the “Hard Problems” of Hand Tracking in Virtual Reality

When Owlchemy Labs revealed its next game at Gamescom in August, there wasn’t much to say.

the the announcement was quite indescribable, with a trailer (embedded below) that revealed next to nothing and showed no gameplay. Instead, Owlchemy focused on two key facts: the game will be multiplayer and will support hand tracking.

Both of these elements are new to Owlchemy, a studio that has built a reputation for producing satisfying, native VR interaction systems that cater to all levels of gamers. A shining example of this is the studio’s best-known title, Job Simulator. Its user-friendly interaction system is probably one of the reasons it remains a best-seller on multiple VR platforms today, despite its 2017 release date.

Around the release of Job Simulator, the Owlchemy team was trying to crack the design code for an emerging technology: VR motion controllers. Five years later, the team finds itself pioneering another emerging form of VR input. “For us, a lot of that [work with hand tracking] feels very similar to how we felt when we first got our hands on the Vive. We’re like ‘Okay, let’s try to figure it out, there are some technical hurdles we have to overcome, but each release gets better.’ Today is the worst version of hand tracking you will play. Tomorrow, or in a few months, it will always be better than today.

While the team is reluctant to confirm that hand tracking is the exclusive input method for its new title, the team certainly doesn’t think much about controllers during development. “We’re big on accessibility,” says Andrew Eiche, Chief Operating Owl of Owlchemy Labs, seated across from me at a table on the show floor earlier this year at Gamescom. “We are building the foundation for manual tracking and at the moment our developers are expanding exclusively into manual tracking. The game does not have an integrated controller. We don’t let our developers use controllers [during development] because we have to solve the difficult problems.

Eiche believes that if Owlchemy can solve some of these “hard problems”, then hand tracking will progress further into a true primary input method for VR. Despite many titles offering hand tracking on Quest – both as a requirement or as a proprietary input mode – the technology still has limitations and there are few agreed-upon standards and implementation methods.

Job Simulator Quest 2

What’s most interesting about this new push from Owlchemy is that the studio is known for its precise interaction systems and yet, by all accounts, current hand tracking technology is generally defined by opposite. While it might be more immersive in some settings, it’s definitely less precise than motion controllers.

“We need to move the conversation about hand tracking forward, both outdoors and indoors. We want to commit to this front and put hand tracking outside of novelty. Eiche sees most existing hand tracking implementations as a cool feature that lives in the world of novelty. There are some exceptions – note Eiche Cubism like a personal favorite – but it’s more like a feature you use when you forget to pick up your controllers. “We try to say [that] it may be more than that. This can be your primary input method.

“It’s hard to predict an industry that has the adoption that we all hope it has. [while using motion controllers]Eiche reflects. “How do you get a billion headsets there? You can’t give someone an Xbox or PlayStation controller broken in half, saying ‘We’re going to blindfold you now. Touch X.’ You need a natural method of interaction.

Motion controllers have proven to be a useful bridging instrument in the recent period of virtual reality growth, but Eiche and the Owlchemy team see hand tracking as a way to spur growth even further. Eiche uses the natural interface of touch screens as a past example, driving the mass adoption of cell phones in the late 2000s. “We’ve had cell phones with keypads for a long, long time and they filled a very specific niche Keyboards haven’t gone away, in fact, we can even connect a keyboard to your cell phone right now, nothing’s stopping you But what really took smartphones from PDAs with a cellular connection to true adoption , is to discover this natural user interface [of touch screens].”

It could be that motion controllers are to the VR headset what the keyboard is to the smartphone. “I don’t think controllers are going to go away, but I think if you talk about how you get a billion people to play VR? They’re not controllers, you’re just not going to win this fight.

Cosmonious High New screenshot

With previous versions, Owlchemy reduces the friction of motion controllers by using a diegetic VR design – this means that everything, whether it’s a button, a switch or a menu, is part of the world Immersive VR they create for the player, ignoring controller buttons everywhere. possible. “We try to put as much as possible in [VR] world, because the more you see and touch, the better off you are. Instead of “Press A” there is a big control panel and you press the button. »

Moving to manual tracking is the natural next step, but it comes with its own challenges, many of which relate to the current limitations of technology. “You always know when someone has done a lot of hand tracking experiments when they do the side pinch. It’s like number one [indicator] … because [they know] the camera needs to see your fingers doing the pinch, picking it up… How do you throw [with hand tracking] great so people don’t think about it? So you don’t do the odd hand tracking throw.

There is also an inherently different experience of interactions when using hand tracking or motion controllers. “It looks like the difference between Duplo and Lego. In gamepad mode you have big mitts… and in hand tracking mode you have [got] this very fine control.

“With all your hands, you’re able to do things with small objects that are more interesting, which wouldn’t be interesting with a big hand. Picking up a small cube or a coin or something with an oven mitt hurts. Previous Owlchemy titles make objects larger than life – like crafting a sand dollar-sized coin – so it feels natural to pick them up with controllers. “But if we put a little cube or a sphere in there, because you have each individual finger [with hand tracking], It’s interesting. There are a lot of cool things you can do once you have a very precise and unobtrusive understanding of where your whole hand and fingers are when you curl them.

“There are a lot of cool states that exist between fully closed fits and fully open palm. Most things in life, when you pick them up, you don’t grasp them with the absolute pinnacle of your grip strength. It creates an interesting dynamic of interaction, where you can start thinking about that and what that means and how that plays out.

While the specific details of what Owlchemy’s next game will entail remain unclear, it’s the team’s passion for advancing technology that comes through loud and clear. Hand tracking is the next step for Owlchemy – that’s not up for debate. “We see the excitement [for hand tracking] and we see the potential. If we can build all of our business while it’s in this [preliminary] state, so when it hits, we don’t catch up.

“We think we’re about to start,” Eiche said with a smile. “It’ll be super.”

Leave a Reply