Put on a headset and enter a virtual world for a first person shooter game. Now connect yourself to an omni-directional treadmill and make that VR game fully physical, as you walk forward, backward, crouch and shoot in every direction. That’s what Virtuix president David Allan showed at the MEMS & Sensors conference during CES, detailing how he created a system that allows a user to jump, strafe and generally have fun… without falling over. “The system needs to hold you in place and understand your movements and gestures,” he said.

He identified the three challenges in creating the system: mechanical (to create safe use and enable movement); sensor fusion, to integrate the information from up to 16 sensors/MEMS; and wireless data transport, which requires very low latency to keep the illusion alive for the user.

“You’re blind to the real world the moment you put on the VR headset,” he notes. “One idea is to stand in the middle of a padded floor, or suspend the user on a leash or stand inside a giant ball. Our early ideas included putting the user in a giant bowl, or on a giant trackball or in a giant baby walker.”

A related problem is, once the user is in place, how to allow his feet to move across the platform. “First ideas included walking on air, air locomotion sandals, walking on water, even magnetic levitation,” he explained. The solution here was a concave base, “where the harness keeps you from falling over and low friction allows you to move your feet.”

Up to 16 wireless sensors live in the shoes, which then transmit data to a sensor hub and, from there, to the VR headset or a PC. Choosing the right wireless protocol for this data transport is a key and very challenging issue. “Concerns are size, throughput and power consumption,” Allan said. “The choices are Bluetooth Classic, BLE (Bluetooth Smart) and a proprietary system based on 2.4GHz.”

In testing solutions, Bluetooth Classic used too much power, although it enabled better throughput; BLE is much lower in power but didn’t provide enough throughput. “You need to look at human reaction time as a guide so latency isn’t perceptible to the human user, which is less than 150ms,” he said. “We had to design our own 2.4GHz system.”

He showed a sensor module a tad bigger than a quarter. “The most interesting part is sensor fusion,” he said. “A sensor fusion is what you need when you’re trying to use multiple sensors to solve a single problem. We use two types of MEMS sensors and two types of magnetic sensors and we put all that data to work with a ten-axis sensor fusion. Sensor fusion needs to take the raw data, combine it and detect activities instantaneously.”