The Aipoly app is aimed at blind people and allows you to describe a scene from a photograph taken.
The Aipoly app features an intelligent vision that allows you to identify scenes. This software is rooted in artificial intelligence to carry out object recognition aimed at offering certain benefits to blind people. With this idea in mind, its creators have developed it, as a project within Singularity University, an academic center in Silicon Valley sponsored by NASA and Google.
Marita Cheng and Alberto Rizzoli are two of those enrolled at Singularity University, and the promoters of Aipoly. The idea was inspired by a blind friend of Rizzoli’s family. So that blind people know something more about the environment around them at a certain time, sometimes a person who sees and is by their side it describes things to them. It is about explaining what is happening or what is beyond the other senses.
Aipoly does just this: it allows you to take a photo of an object or a scene and the software narrates what appears in the image. It is about automate the work a person does when a blind person asks him about what is around him or in front of him. The application uses artificial intelligence to carry out the recognition process, more specifically machine vision and machine learning.
Operation based on data and artificial intelligence
When a user takes a photo, it is sent to the application’s servers. They produce image processing and once identified objects or scene, the information is returned in text form, such as a voice message. Processing is the key part. It is done through convolutional neural networks (CNN), which subdivide photography into several points of interest and associate each of them with specific objects.
From when the user takes the photo until he receives the spoken information can take between 5 seconds – with a fast WiFi connection – and 20, with slow mobile connections or for images that require arduous processing.
Aipoly can not only identify objects but the relationships they have with each other in an image. Thanks to the advance of computer vision today a system like the one used by the application can distinguish that in a photo there is a person riding a bike, instead of simply describing that a person and a bicycle appear.
At the moment the Aipoly team does not use its own algorithms, but is working on its own database from which to train the system. So far technology trained with around 300,000 images and it’s growing. Creators plan to focus on objects that may be useful to the blind, such as road signs.
Needless to say, the system has its flaws. Does not properly distinguish gender of a person or facial expressions, unless they are very marked.
Images: lecercle
