When humans hallucinate, it's usually considered a problem. When computers do it, it's science. Research divisions of Google's parent company Alphabet have recently been working hard on improving the way computers perceive and generate images. Last month, Google showed off RAISR, a tool meant to save bandwidth through heavy image compression and restoration. In a newly-published research paper, the Google Brain team now demonstrated its work in "pixel recursive super resolution," which uses deep-learning techniques to look at ultra-low-resolution images and try to recreate the original high-res pictures. Sound familar yet?
The "hallucination" process involves first giving the software a tiny image to look at. The code then makes guesses about what was originally there, based on information it's learned looking at other higher-resolution images. The results are far from perfect and often goofy or strange-looking, but they're still pretty impressive. The team subjected images of celebrities and bedrooms to their software, and then showed the results seen below to a test audience, asking them to tell which image was the downscaled high-resolution image and which was the upscaled low-resolution image. The Brain team fooled 10% of their test audience with the celebrity images, and 28%—almost a third— with the bedroom pictures.
The recreation is a two-step process. First, a "conditioning network" takes a tiny 8x8 image and tries to match it with other low-res pictures obtained from downsampling lots of high-resolution images. After that, the "prior network" uses a version of PixelCNN (another piece of neural network imaging software) to estimate where different parts of the higher resolution images would go—eyes go here, mouth goes there—using information taken from the many pictures it's previously looked at.
The results are impressive, but obviously imperfect, since they're still best-guess estimates. As the technology improves, however, it's easy to imagine it being used in multiple scenarios including facial recognition. The Brain team has a lot of work left to do to with concept, but it shows how far computers have come in understanding images and making sense of them.