Simple full-body tracking for end users is one of the most challenging virtual reality challenges yet to be solved. Facebook is trying electromagnetic sensors.
Cameras embedded in virtual reality glasses are not the optimal solution for tracking the entire body, since the body often covers itself or falls out of sight altogether. On the other hand, external cameras have a limited mobility problem: if you leave the camera’s field of view, you will no longer be tracked.
Therefore, a team of researchers from Facebook Reality Labs and ETH Zurich is trying to use electromagnetic sensors, since they do not require visual contact.
Whole Body Tracking: Sensors “solve”
This type of tracking requires a base station that generates a reference electromagnetic field, and a number of electromagnetic sensors from which the distance to the base station and orientation can be accurately measured. For research work, the team has developed its own tracking system with particularly light and compact EM sensors.
The tests were carried out once with six and once with twelve sensors. The fewer sensors required, the easier it is to use them, but then the accuracy of tracking the whole body also decreases.
The principle of operation of the electromagnetic tracking system. The figure on the right shows an installation with twelve sensors. The base station rests on the lower back
The researchers write that developing their own electromagnetic tracking system is not an easy task, another task is to deduce poses based on measurement data. If you use only six sensors, the AI model should fill in the gaps in the measurement data. For example, the spatial position of the forearms is no longer measured.
Slight deviation with six sensors
To do this, the team has developed a two-part solution: first, based on the training data, the pose is displayed, then the accuracy is optimized.
The researchers are satisfied with the results: with twelve sensors, the accuracy of the sensors deviates by a maximum of 31.8 millimeters and 13.3 degrees, with six sensors only slightly more – by 35.4 millimeters and 14.9 degrees.
The following video shows the electromagnetic tracking of the whole body in action. In the right part of the image, the avatar, including the derived poses, was placed directly above the subject, so that the real position and movement could be compared with the reconstructed one.
