Foveal rendering is an important basic technology of virtual reality and the next big step in the development of VR points. Now, researchers at Facebook have presented a method based on artificial intelligence (AI), which additionally optimizes the rendering process with low power consumption.
When foveal rendering only a small area of the field of view is calculated at full resolution, which currently is viewing and the user sees the virtual reality. What focuses the eye, is determined in real time by tracking the eyes.
The rest of the field of view is displayed with less detail, which saves a lot of processing power. Foveal rendering can provide even more sophisticated graphics and virtual reality to reduce hardware requirements. It is also needed for VR displays with higher resolution.
AI recovers images with less detail
Researchers from Facebook Reality Labs now introduced a method that allows to reduce power consumption of the peripheral region of the image and continue to be visually as close as possible to the original. System based on the AI Facebook researchers call DeepFovea. How does it work?
Generative-adversarial neural network (Generative adversarial network) first scans a large amount of footage, initially analyzing about ten percent of all of the pixels in the video still image. From this she then creates a less detailed version of the peripheral region of the image which is visually close to the original. Facebook researchers called this procedure “reconstruction of neurons“.
On the right the original image with focus on the head of a lizard. On the left image, reconstructed AI. The head of the animal is equally detailed in both images — in contrast to the wood surface in the peripheral region of the image.
DeepFovea suitable for games
DeepFovea should be able to compress the original video in real time with low latency 14 times. Thanks foveal rendering, the user will not notice any difference in image quality.
Another advantage compared to previous methods: an image that is compressed DeepFovea, less deviates from the source material than conventional rendering methods, as shown in the following example.
DeepFovea for example GTA. To the right of the source section of the image in the middle of the image, rendered DeepFovea the left image of the same section of the game that is compressed using conventional rendering.
Facebook hinted rendering using the AI is still at Oculus Connect 5. When technology is used in glasses Oculus, is not known. To date, none of the available points from Facebook do not offer the eye.
DeepFovea was presented at Siggraph Asia.
Source
