Microsoft tries to patent room-filling immersive display

Microsoft is trying to patent the holodeck. Well, sort of. Back in early 2011, the company filed a patent application for an “immersive display experience” that combines primary and peripheral images to give users a better sense of their environment. A patent has yet to be granted, but the application is now online for all to see.

The application explains the relationship between the primary image, the peripheral image, and a depth camera that tracks the position of the user. The peripheral image is derived from the primary one, with corrective distortion applied based on the user’s perspective, the topography of the room, and even the color of the walls and upholstery. A projector is tasked with displaying the peripheral image, which will have a lower resolution than the primary one.

Rather than being the focus of the user’s attention, the peripheral image is designed to provide environmental awareness. As I’ve discovered using triple-screen surround gaming configs, even limited peripheral vision can enhance the gaming experience greatly. The lower-resolution image shouldn’t be an issue, since the player’s attention will be focused on the primary display.

True holodecks are still relegated to the realm of science fiction, but the system described in the patent application should be possible with today’s technology. Microsoft’s Kinect motion controller already offers depth camera functionality, and small projectors seem to be popping up everywhere. Adjusting the peripheral image to compensate for the environment may be the most challenging aspect of the entire system. Thanks to Ars Technica for the tip.

Comments closed
    • Chrispy_
    • 7 years ago

    Interesting side effect from the bit about peripheral vision being lower resolution.

    For those of us with Eyefinity setups and/or 4MP displays, I’d really like to see dynamic resolution:

    Take an FPS or driving sim; You’re mainly focussed on the area around the crosshair, or a point of the road 50 yards or more ahead:

    It would be nice to have the option of something like this:

    [list<][*<]a 1024x600 portion of the screen at 1:1 resolution with 8xAA and all the filtering it can handle.[/*<][*<]The rest of the main screen at half resolution[/*<][*<]trilinear filtering between the two resolutions to avoid a visible transition[/*<][*<]half resolution with no AA on the side screens.[/*<][/list<] The idea being that you can get the same framerate/fluidity from 3 screens as you normally would with one screen: The stuff in your peripheral vision is quite often whizzing past at a high rate, is something you don't even focus on for more than a quick glance, and normally has motion blur or similar postprocessing applied to it by the game engine anyway. Why waste two-thirds of your GPU's horsepower on stuff that you're barely looking at?

      • indeego
      • 7 years ago

      Because when you *are* looking at it the information is vital, and the difference can give wrong information. Uniformity (and fluidity, like 120Hz displays) to me seems much more important.

        • Chrispy_
        • 7 years ago

        I see your point, but I would be more than happy with side screens running at 1280×720, and it would be nice to see options that can be configured “per screen” in a multi-screen setup, rather than “per setup”.

    • DPete27
    • 7 years ago

    Psssh, I can use three of those Sharp 90″ TV’s, one front and two sides, to do the same thing.

      • dpaus
      • 7 years ago

      Not quite, but 6 of them arranged in a cube [i<][b<]would[/i<][/b<] give a true, full-immersion environment. A bit small, maybe. How tall are you?

        • derFunkenstein
        • 7 years ago

        and at truly blinding levels of light (seriously, 6 of those things lit up and pointing at the same center? wowza).

          • dpaus
          • 7 years ago

          Well, the one he stands on wouldn’t be bright for very long….

        • Grigory
        • 7 years ago

        Since they’re not square they can’t form all 6 sides of a cube. :p

          • dpaus
          • 7 years ago

          DPete just ran off crying. I hope you’re happy now!!

    • albundy
    • 7 years ago

    should be great for headaches and migraines

    • ludi
    • 7 years ago

    There are three independent claims in the application: 1, 7, and 16. If any of these fails on examination or challenge, the dependent claims that follow (2-6, 8-15, and 17-20) also vanish.

    As I read it, all three are discussing functions of a computer system, the things that appear specific are:

    Claim 1 — a computing system that tracks user position with a depth camera, keep the user in a shadow while rendering and displaying separate primary and peripheral display paths. The peripheral display is explicitly an extension of the primary display.

    Claim 7 — a computing subsystem that maintains primary and peripheral display rendering paths, where the peripheral display is explicitly of lower resolution.

    Claim 16 — a computing that follows the same structure of claim 1, except that it tracks depth and color position and distortion-corrects the peripheral display.

    [u<]What it probably means:[/u<] It sounds to me like they're trying to make a living room system that outputs the main display to an HD display and then supports the peripheral display path using (relatively) cheap office projectors. The depth camera, presumably Kinect, then maintains a blackout position wherever the user is standing so they don't get blinded by the projectors. Has anyone actually set up a system like this before? I've seen a position-tracker with selective blackout on a PowerPoint projector (so the speaker can walk the stage without getting blinded), but that was a single-display application, not multiple, nor would it qualify as "primary" and "peripheral".

      • lilbuddhaman
      • 7 years ago

      Thanks for the TLDR, since it was too long and I didn’t read. (too busy reading iphone 5 info of course!)

    • GatoRat
    • 7 years ago

    Given court decisions over the past several years, Microsoft has no choice. If you patent, or try to patent, every idea (and that’s what most of these patents actually are), no matter how absurd, you risk getting screwed over in a law suit.

    • mcnabney
    • 7 years ago

    It would really be much easier and cheaper to just use VR displays with very high resolution (570dpi is required for a display to be retina-like at 6″).

    So a flexible display that is 5″ tall and 12″ wide would take care of it. That odd shaped panel would be over 6000×3000 pixels to provide a resolution devoid of pixels. The display would probably cost a couple grand, but it would be easy total immersion, multiplayerable, and portable.

    • yogibbear
    • 7 years ago

    Does prior art from movies and books count? If so, this has been done over 9000 times before.

      • Sahrin
      • 7 years ago

      Not according to Apple; or apparently the US District Court.

        • yogibbear
        • 7 years ago

        Yeah I was watching the original (*cough* only) version of Total Recall the other day and when they answer the vid phones they use their finger and slide it down to turn it off, and across to turn it on.

          • BiffStroganoffsky
          • 7 years ago

          When the time comes, I’m sure Apple will sue the vid phone makers too.

      • Arag0n
      • 7 years ago

      The point is that Microsoft is patenting a concrete way to make it possible. How do you get the image in the walls? How do you handle the users position perspective, etc. Someone could come with a different implementation where the walls are screens and have totally no problems with that patent at all or use some kind of echo-device to locate the user in the room instead of Kinect. Still, if they make it real in the near-future I wont complain if they keep a patent for it. I don’t think anyone else was looking forward to make it possible in the near future.

        • Grigory
        • 7 years ago

        Well said! 🙂

      • Vulk
      • 7 years ago

      No. Many devices created in science fiction and movies have been patented and the artistic work DOES NOT count as prior art. Only another patent or physical working items count as prior art.

        • Grigory
        • 7 years ago

        Of course artistic work counts as prior art. If there are patents that violate that, they could easily be disputed. However, most of the times the patents protect HOW something is done (which isn’t covered in the artistic work), not THAT it is done.

    • blastdoor
    • 7 years ago

    But will there be a start menu? Because if not, then I’d rather just get three video projectors displaying win xp desktops on each wall.

      • blastdoor
      • 7 years ago

      Unrelated thought — has anyone ever tried to patent an algorithm for applying for patents? That would be cool beans.
      –patent pending

        • yogibbear
        • 7 years ago

        You should put ‘- patent pending’ after that else be forever sad that you didn’t!

    • superjawes
    • 7 years ago

    Apple tries to patent shapes.
    Microsoft tries to patent a room-display.

    Little crazy, but I have to applaud Microsoft for the effort.

    • blorbic5
    • 7 years ago

    This could be really cool for FPS games but how would you move your character around the map.

      • NeelyCam
      • 7 years ago

      You run in place.

      The real question is, how do you move backwards..

        • superjawes
        • 7 years ago

        I would prefer a controller, even a simple one, to control stuff like that. And a few extra buttons would certainly work better than trying to trigger everything with gestures.

        • dpaus
        • 7 years ago

        [quote<]how do you move backwards..[/quote<] Simply install Microsoft Small Business Server. In business, running in place [i<][b<]is[/i<][/b<] moving backwards...

        • Grigory
        • 7 years ago

        You run in place while sticking your ass out. The system picks up on your ass and moves your avatar accordingly.

          • dpaus
          • 7 years ago

          “Does this avatar make my ass look big?”

            • Grigory
            • 7 years ago

            With this system if your avatar can only walk backwards you know it is time to put down the fork. 😀

      • mcnabney
      • 7 years ago

      Sensors would know the direction and angle of your controller (probably a fake gun). The controller would have forward/backward/side2side buttons, plus the usual fire, alt fire, and other buttons to control interactivity.

      Here is the problem. Even going with 720p projectors, it would require a custom room (or at least minimal furniture and bare walls). The display/hardware cost for this would be $5k at best – likely many tens of thousands for early adopters.

      Don’t forget that using projectors would require you to stay in the center of the room, or you would block the image.

        • superjawes
        • 7 years ago

        I have a feeling that this isn’t for your average consumer. Probably specifically targeted for wealthier people/arcades. Getting into a general consumer space would probably take at least 10 years.

        • Sargent Duck
        • 7 years ago

        If you don’t have kids, it’ll be easier.

        I live in a townhouse and have the space downstairs (once I clear all the Christmas decorations out). I have 2 friends that each just bought a 4 bdrm house (despite the fact they’re only dating) and they each don’t have a lot of stuff, so they have 2 empty rooms.

        If you have kids and are middle class, this isn’t going to happen.

        That being said, although it’d be super cool playing a FPS in your own house, could you imagine playing a survivor horror game? Even when it’s off you’d be scared crapless to enter that room at nighttime!

Pin It on Pinterest

Share This