Researchers from Facebook Reality Labs, so you can manage the applications of mixed reality using the object that you hold in your hands.
A research group consisting of members of the Facebook Reality Labs and the University of British Columbia (Canada), wants to facilitate your life in the future and make books and pens input device using hand tracking.
To use everyday objects as controllers
With the system “Gripmarks” everyday object will become the input device for a mixed reality. Thus, you won’t need a controller and instead you use the object, which hold in your hand.
This should work as follows: you take, for example, a jar of sparkling water in hand. AR goggles or glasses VR track the position of your hand, and the software compares the capture point fingers with saved drawings capture. If it detects the lever position, in this case, it computes a cylindrical object without the need to directly record and keep track of it.
On the other hand, you perform the gesture activation object, similar to the gestures on your smartphone. Thus, to determine the location and size of the virtual input field, which should appear on the object.
Now you have a control panel of media player or the calculator buttons directly on the object. In the end, you actually hold in your hand a jar and feel tactile resistance, while you also see a cylindrical remote control to input virtually or in virtual reality.
Various input methods have become possible with the help of Gripmarks
In addition to the two-handed control, it is also possible to input only one hand. As examples , the study group in its report, the spray bottle and the marker. If you hold the pen in one hand, swipe left and right or tap to confirm.
For example, with spray you are targeting a virtual button and push the lever of the sprayer to activate it. So far the system recognises five schemes capture: the capture book or wallet, of a cylindrical object, the shape of the gun, e.g. a spray bottle, capture of the marker and the subject of the Apple shape.
Yet Gripmarks Rworks only with the motion capture
The team demonstrates a new principle of the controller with the application of VR and the Oculus Rift’s, as well as the system hand tracking based on motion capture. So, if you hope to shoot from a toy gun as a controller VR in the Arizona Sunshinewill have to wait.
However, the research team wants to make its system more practical. In the next stage of development Gripmarks will only work with touch gloves and optical tracking inside-out.
Source
