Google Brain neural network learns how to ‘Enhance’

When humans hallucinate, it's usually considered a problem. When computers do it, it's science. Research divisions of Google's parent company Alphabet have recently been working hard on improving the way computers perceive and generate images. Last month, Google showed off RAISR, a tool meant to save bandwidth through heavy image compression and restoration. In a newly-published research paper, the Google Brain team now demonstrated its work in "pixel recursive super resolution," which uses deep-learning techniques to look at ultra-low-resolution images and try to recreate the original high-res pictures. Sound familar yet?

The "hallucination" process involves first giving the software a tiny image to look at. The code then makes guesses about what was originally there, based on information it's learned looking at other higher-resolution images. The results are far from perfect and often goofy or strange-looking, but they're still pretty impressive. The team subjected images of celebrities and bedrooms to their software, and then showed the results seen below to a test audience, asking them to tell which image was the downscaled high-resolution image and which was the upscaled low-resolution image. The Brain team fooled 10% of their test audience with the celebrity images, and 28%—almost a third— with the bedroom pictures.

The recreation is a two-step process. First, a "conditioning network" takes a tiny 8×8 image and tries to match it with other low-res pictures obtained from downsampling lots of high-resolution images. After that, the "prior network" uses a version of PixelCNN (another piece of neural network imaging software) to estimate where different parts of the higher resolution images would go—eyes go here, mouth goes there—using information taken from the many pictures it's previously looked at.

The results are impressive, but obviously imperfect, since they're still best-guess estimates. As the technology improves, however, it's easy to imagine it being used in multiple scenarios including facial recognition. The Brain team has a lot of work left to do to with concept, but it shows how far computers have come in understanding images and making sense of them.

Comments closed
    • DPete27
    • 3 years ago

    I can look at that top left face and tell you straight away it’s Justin Bieber. Computers are still dumber than humans. [add] …except maybe Justin Bieber.

    • Krogoth
    • 3 years ago

    Skynet coming online in 3, 2, 1……….

    • ronch
    • 3 years ago

    Hey, Zen also has some sort of neural network, right?

    • Mystiq
    • 3 years ago

    [quote<]The Brain team has a lot of work left to do to with concept, but it shows how far computers have come in understanding images and making sense of them.[/quote<] Shouldn't that really say: [quote<]The Brain team has a lot of work left to do to with concept, but it shows how far [b<] we've[/b<] come in understanding images and making sense of them [b<]using computers[/b<].[/quote<] 🙂

    • CScottG
    • 3 years ago

    No Way Out (with Costner and Hackman) is probably a better example..

    • the
    • 3 years ago

    This technique could potentially be better with video. Objects that change depth can be used as higher quality source images to enhance the edges of the path it takes. Essentially undoing anti-aliasing in motion to figure out the background.

      • UberGerbil
      • 3 years ago

      Yes, to some degree you can use the temporal dimension to compensate for limitations in the spatial dimensions.

      • brucethemoose
      • 3 years ago

      Motion interpolation already does that, to some extent. It takes parts of the background from the previous/next frames and pieces it together for the frames in between.

    • Voldenuit
    • 3 years ago

    Portrait #3 was obviously this: [url<]http://www.syfy.com/sites/syfy/files/styles/large/public/Helix_blog_eyepatch_nick_fury_02.jpg[/url<]

    • brucethemoose
    • 3 years ago

    Notice how they use beds and faces? That’s the real limitation of this method… It’s needs a large sample set of images similar to the one it’s trying to upscale, and time to “train”. That makes good at, say, upscaling a database of low-res mugshots or profile pics, but not so good at upscaling a video or random web images.

    And unlike CSI’s “enhance”, which can get a license plate number out of a pixely mess, Google’s code would just insert some random number.

    Still, neural network upscaling can pull off small miracles. Waifu2X does something similar for animation, and I’ve had amazing results with it.

      • VincentHanna
      • 3 years ago

      Oh god. Cringe. That show.

      This was worse back in 2001 when the show aired, but still…

      It’s also worth pointing out that there exists a great many pictures in this world of a great many things. Things including : vans, license plates, back alleys, dumpsters, tile floors, sofas, artwork, statues, lamp posts, man hole covers, etc…

      And if the right databases haven’t been compiled yet, they can be in short order.

        • brucethemoose
        • 3 years ago

        Yeah, but you still need that contextual information. So this could never work as a browser image scaler, as it can’t tell what database it would need to scale an image.

      • UberGerbil
      • 3 years ago

      [quote<]And unlike CSI's "enhance", which can get a license plate number out of a pixely mess, Google's code would just insert some random number.[/quote<]Which is not going to stop somebody somewhere from using that random plate number, or the random face that comes out when applied to a picture, as a means to "find" the vehicle or person of interest in a crime -- along the lines of what happened on Reddit in the immediate aftermath of the Boston bombing.

      • Wonders
      • 3 years ago

      [quote<]some random number[/quote<] This is not accurate.

        • brucethemoose
        • 3 years ago

        Some license plate number from another image is what I mean.

    • Voldenuit
    • 3 years ago

    Buckle up, buckaroo.

Pin It on Pinterest

Share This