At Nvidia's Montreal 2013 event, top game engine developers Tim Sweeney of Epic, Johan Andersson of EA DICE, and John Carmack of id Software and Oculus are on stage together discussing the future of gaming technology. Here are our notes from the discussion.
John Carmack, Tim Sweeney, Johan Andersson
Moderator: Tony Tamasi
Tim: For last year, haven't had much time to program. Seeing Epic's transition into next gen. Building Unreal Engine 4. UE was just me for about 4 years, joined by two other programmers sto ocmplete. UE4 is over 200 people. Epic's upcoming project is Fortnight, an online game you can play without going into a retail store. Kind of Minecraft meets Left 4 Dead with a Pixar-style art direction. Also working on other things.
Johan: Growing Frostbite, a lot of different types of games using it. Cool to see lots of different teams and cultures make use of it. Also back at DICE, seeing say a water system that's usable across different games, tech that can be shared.
John: After we shipped Rage, I looked into state of VR. Was a promise in the 90s. We licensed Wolf, Doom, some but I was very hands-off. Thought they were losers. But 20 years have passed. Was shocked to see nobody had really done it well after 6 orders of magnitude of improvement in performance. But realized it was in reach. Decided to use a HMD as a hook for promoting Doom 3 BFG. Palmer's taped-together one was a better experience than anything else. Demoed it and good reception. Oculus was founded. A couple of months ago, I decided to move full-time to working at Oculus. I can look at some of this stuff and say I remember when we moved from 2D to 3D. Was significant. FPSes have been a persistent, dominant genre since then. Have dev kits out there. Will be a big deal. Been heads down, locked in a room, working on a lot of core technology issues. Feel like myself more than any recent time in a way. Engergized and excited about what we're doing. Gonna make a little bit of a dent, I think.
Tim: Today's games have lots of glitches. G-Sync lets us get around that. Devs have to get together an pound out all of the latency and jitteryness.
John: I fought the crusade for 60 FPS locked frame rate in the last gen. There were a lot of significant battles along the way. Game would have been better if we'd have compromised on frame rate, though we weren't willing to do that. G-Sync will let you make that compromised. Will unequivocally make for better games. Smoothness is important, but it can be legit and correct and valuable to user to sacrifice it for some things.
Way back on the 3dfx tech panel, I was complaining about tearing. Dude said to me, it's your fault. Due to the benchmark wars on Quake. But accepting tear was bad.
Dropping a frame in VR is like a kick in the head. You also really notice the tear lines. We recommend vsync is always on, but then the hitches are terrible. So something like G-Sync would be helpful. The persistence is also huge. I know you're not really touting it, but the low-persistence leg of it is important. Moving to between 90 and 120Hz will be an added benefit. Is that much better.
Valve's low-persistence demo was an A-ha moment for me. 60HZ was clearly not good enough, but I thought 120Hz would be. They were very positive that the persistence of the display was an issue. When I saw the demo, realized it was better when you could turn low-persistence on and off. Was thrilled to be proven wrong.
Tim: VR applications. Having been to Valve and worked w/Oculus folks. Closely related to what NV is doing with G-Sync. Dealing with visual flaws, some of which are incredibly subtle. Reminds you that what you're doin gis tryiing to replicate world.
What about in 5-10 yrs?
John: 30 years ago state of the art was the movie Tron. What used to take hours to render then we can do in real time on a wristwatch today. Comparisons like that are relevant. Five years from now, 4K displays on tablets and HMD. Another order of magnitude perf on GPUs. Could do that 4K triple-display on one GPU. Other things, network will improve. There's a trillion $$ of economy pushing these things. We can always turn the cranks, it'll always get better, but te interesting thing is what you didn't see coming.
Johann: I have a pretty good sense of visuals in the next few years, but one of the hardest problems isn't the visual techniques, but would like to be able to solve how we create our game worlds. Massive challenge as an industry. Not clear whose problem that is. Our goal is to create amazing game experiences, have lots of games enjoy it. I don't know how we do it in five years if we continue on current trajectory.
John: Game content creation, does it become like fab creation where only three companies in the world can do it?
Johann: May have to break things up, more companies, more proceduralism.
John: Proc has been the next big thing for 20 years.
Tim: Content creation challenges are really severe. When you take the longer term view, in several generations NV will kind a way to put 100 tflops into your pocket. Eventually have dual 4K displays in format of Oakley sunglasses. Eventually Carmack said earier, you eventually have a superpower.
John: It's not immediately obvious, but we are tied to mobile development product cycle. A little frightening, because the displays won't be available very long. Is a big commmitment to update our product every year.
Johann: We've been experimenting with VR with our game worlds for a while. Seeing the mechanics. Those are the difficult questions. We have a good feel on the rendering side, but animation is a challenge. Takes a lot of time analyzing these things, getting people familiar with how it changes things. Surpirsing how much it changes.
Q: What do you guys think about Steam machines and SteamOS?
John: Valve approached it at the beginning of Steam, and we basically said, Are you crazy? Would be nuts to tie yourself to this little digital distro platform. I'm afraid I'm at the same point now. Making your own OS, are you crazy? Maybe they will be right. Seems a little dicey to me. Given their track record, I can't be scornful.
Tim: Linux right now is #1 consumer OS. Valve isn't like MS or Sony launch, big event where 10s of Ms of players show up right away. Thing they will play a strong long game. There is a lot of fear being tied to platforms tied to MS or Sony or others. Control over e-commerce prevents things devs want to do. This won't be absolutely controlled by Valve like MS and Sony. I think it will push others, like Sony, to pursue an enlightenen path. Have to judge this over the next decade.
Johann: Was initally skeptical about SteamOS, but commitment to making it an open platform is good. Initially very positive. Also, tech is exciting. Also quite a lot of challenges.
Q: Next challenges now that tearing and lag are solved?
Tim: This solves monitor problem as well as it can be solved. Oculus and stuff is what you need to go beyond it.
Johann: Higher FPS isn't where we are now. Want to get there, run every game at 100 FPS with variable refresh. Taking that leap forward requires a lot more performance. We need to get that from everyone. Think it'll be quite interesting for others.
John: I think the pictures being made in modern games are pretty damned good, but my belief is it's time to start pushing for higher frame rate lower latency. Could say it's a connsieur thing, and there may be different classes of consumers, but I'd like to see more efforts to improving the latency and the all-around feedback look. We know where the better pixels go. It goes to movie rendering. Then we Avengers scenes rendered in real time, but there's other things we can do with that horsepower that might be more valuable.
Tamasi: Voodoo graphics was 45M pixes/sec. We're talking about GPUs today that are many orders of magnitude. It's inevitable you'll have equivalent of Titan SLI rig attached to your face.
Q: Will you be able to forget about target frame rate in a game and assume its' there?
John: This is something that should be broadly adopted. This is just the right thing. IN the space of five years I would hope this is ubiquitous.
Johann: There's a limited number of things a monitor manufacturer can do. Once people see it, I think there will be a lot of movement to something like this,
Tim: Yeah, the rest of the industry needs to hardware up. Every display device, every platform, we need to adopt thins technology. INcredibly apparent the difference in quality here.
Johann: You go to retail stores, all of the screens are in a showroom mode that cranks up all the colors. Instead, show this varable refresh rate and it will look nice.
Q: What about for games you're working on now?
Tamasi: Pretty much every game comes with settings that push the limits of everything.
John: There was a lot of pain from the artists not wanting their favorite particle effect to be cut. You had to be pretty brutal to hit 60Hz. Not clear next time since you'll have to deal with the next gen consoles that don't have this. Rage would have been a richer experience with some moments where there's more stuff onscreen. Will be nice if nobody has to fight through those battles, make a smooth game without having to fight through that. The raster is a very harsh taskmaster.
Q: Can you talk about how your were influence by others on the panel?
John: Issue at id was years between releases. Was looking at some of the Gears of War stuff, talking to artists about how they're wrapping seams, makes for a very polished world. Amazed by current games with lots of people making huge titles. Could not have conceieved. When I look at Gear or BF, I'd proud to be a peer.
Johann: Both John and Tim have been an inspiration to us. Having very large worlds being textured like that. In BF, we have a virtual texturing system that's somewhat procedural. Massive inspiration, had a team working at DICE on Mirror's Edge with Unreal. Tool set inspired us to make our own toolset better.
Tim: I grew up playing older computer games. When Doom game out, I gave up on programming for a year because this was some unimaginable withcraft. Took a while to get back into it. First game I built, ZZT, and it came with a level editing tool. Kids grew up playing my game. I thought ZZT was a tradeoff that couldn't be matched in the 3D graphics era but Notch did exactly that. Cool to see these innovations come along surprisingly. Nvidia's tech here could have been done any time in the last decade, but stars aligned now and we see the dramatic improvement it provides.
Q: When do you think you'll have games like BF4, Gears, Rage on a mobile platform?
John: I work a lot on an Android platform. Still a huge gap. Phones and tablets still aren't where an Xbox 360 is. Still a lot of challenges aside from horsepower. Shield is pretty potent, but you still have a lot of dropped frames due to power managment. G-Sync would be a wonderful thing to have on those mobile platforms. As far as seeing the latest Battlefield on mobile, it's still a long way off. Probably will see a slight decrease in the growth of mobiel GPU power. Current torrid pace of growth will probably slow. I'm going to be really happy if Nvidia can capture some share in mobile since Nvidia devrel, tools, and driver team that's probably better than everybody else in the industry together. Would be nice to get that support on the mobile GPUs.
Johann: We do a lot of research on mobile. We are getting there. There are quite a few questions there. Even if you could play the same game on a mobile device, the game types and biz models are different. We have to so many teams trained to work in PC, console model for a long time. Think we'll start seeing hybrid type of games, a little more of console experience on tablet. Some may be really awkward and some may be good. There will probably be a Halo moment where something works really well.
Also memory systems are an issue, even on laptops, but it's our fault if we can't find ways to make games on those platforms.
Tim: I think NV coming in with Kepler is a nice. Driver quality is lacking in the industry. Quality is there but the overhead is severe. The only mobile platform we can run our next-gen game platform on is Nvidia's they're working on. HW in other devices is capable of the GPU performance, but they're losing it all in software. Looking forward to increased competition on that front. Major opportunities are tuning on the whole system level.
Are CPUs, memory bandwidth expanding fast enough to keep up with a GPU architecture like Kepler?
John: They're doing such a good job with that stuff so that I don't even need to pay attention to it.
Johann: Interesting question what's the right balance of resources for a mobiel SoC for gaming. Will take some time to have all of that settle down. Is good to have competition. Will likely be more companies in the future--and then less. Is a massive open space there.
Tim: Mem BW seems to be in good balance, but once you move to large display like a television, you'll want more. A lot of techniques we're using for things like HDR don't work on mobile due to BW limitations. Things like die stacking, new memory architectures are going to be interesting.
Carmack: Lemme take a bathroom break.
Jen-Hsun: Let me say thanks. You didn't just make our launch. You made our year.
Ok, they're back onstage and talking about the future of PC gaming. All three seem to think the PC has a good future.
Carmack: There's another platform that's relevant to G-Sync, and that's cloud gaming. G-Sync could be useful for that since the cloud platforms all tear horribly. Streaming still hasn't taken off in a big way, but I think it's inevitable it will take some share of market.
Johan: It is getting awkward to have all these different types of devices at home. Would be nice to have a central server at home, be able to play games on whatever device or monitor. Could see same things in an office, also.
Q: 4K vs. G-Sync, what is more important to gaming?
Carmack: 4K tradeoffs are iffy now. G-Sync will make things better. Want to see this absolutely ubquitous. It's a shame the 4K monitors in the back are juddering and tearing now.
Sweeney: We'll be using 4K monitors for building games, but I don't think it's the right output resolution for gaming until your GPUs become more powerful.
Johann: I use several displays now. A single curved 4K display would be nice.
Carmack: I want spherical section screens so everything focuses on the lens properly.
Q: What do you think of Mantle?
Carmack: Should Nvidia have a response to Mantle? Unequivocally no. You already have good low-level access through extensions Nvidia has rolled out. Now, AMD has talked many times in the past about close-to-metal architectures and it only became interesting because of their dual console wins. Devs will be making systems architectural changes that favor those. Some benefits AMD can reap from it. Some implications for Steam. But looks like Microsoft isn't going to embrace it. If I were still doing all of the major tech coding for next-gen game stuff, I probably wouldn't embrace that.
Tim: There are some good ideas in Mantle. There's some overhead in DirectX and OGL. They date back to old hardware built on a different model. I hope it helps the OpenGL committee and MS shape their APIs. I do want a low-level API, but not five of them. I don't think it's a good idea to have different things on all these platforms.
Johan: Yeah... (laughter) The idea is that we solve some of the long-term problems we have on the PC, getting console type robust, stable performance. We do that with DX and GL, too, but it's interesting to open up these architectures with which we're intimately familiar. I see it as a success right now given the amount of discussion and enthusiasm and even the opposite of enthusiasm. It's been a bit stale in the PC graphics space. MS switched focus for a while. There's a lot of movement now, and it's good to have a lot of avenues to experiment. I completely agree that we don't want a future if Nvidia were to do their own API, and Intel... MS and Khronos do a good job.
Tim: Johann, what is the cost of implementing Mantle on the PC?
Johann: Too early to say. We're just making sure our game works. When it comes to bottlenecks, it's like whack-a-mole. You fix one, and you find another.
John: The big thing is no API makes a dramatic difference if you make a good game engine. Still, would be nice to take over some of the GCN async engines.
Johann: We're still learning and experimenting. And Mantle will not replace all of DX or GL. They're complimentary.
Q: Can you tell us how much the paid for you to use it? (Groans)
Johann: They wouldn't have to pay for me to use it. I've been advocating for this kind of thing for a while.
Tim: So it's good for a research and development platform, and it's good to push MS, but I think it would be bad if we had to use five different APIs.
Johann: Sure. And it can become an open API.
John: But then it wouldn't be a low-level API.
Johann: I disagree somewhat. The way modern GPU work is very different from DX and GL. Like bindings, for example. The way Nvidia has been doing that, mobile developers agree that's the future.
John: My point about.. unified memory is absolutely here, and we need to get documentation about how it all works.
Johan: And everything is becoming SoCs. We need to understand how it all works and where everything goes. One of the best use cases for GPU compute is for graPhics. You want to use it both tightly coupled between GPU and CPU, and then put low-latency stuff on CPU and other stuff in background on GPU.
OK, they are geeking out too quickly to keep up. Ack!
Q: Will you be playing the games you're developing on Nvidia hardware since you like G-Sync?
Johann: Tough to say. Nvidia has good GPUs. They don't have Mantle. Hard to say. There will be first-mover advantages for companies do things.
John: Will G-Sync be licensable?
Tony: We've haven't precluded that. We're working on it.
Tim: We use mostly Nvidia hardware. We buy most of it!
John: I've got a LightBoost monitor on my desk and I'll probably update to G-Sync.
Q: What do you want to see next in the future?
John: We're building it. One of the great things about doing this.
Tim: It's a great time to be a game developer. Industry is in an awesome place. Can now build without retail, publishers. Just build a game. Pace of innovation is impressive. Next set of games will be build around a different set of infrastructures than in the past.
Johann: I agree. Lots of platforms, and we don't yet know exactly how each will fare.
Tony: Thanks to John, Tim, and Johann for coming to Montreal.
And we're done!