Slashdot Log In
Ray Tracing for Gaming Explored
Posted by
Soulskill
on Fri Jan 18, 2008 09:01 AM
from the pretty-pictures dept.
from the pretty-pictures dept.
Vigile brings us a follow-up to a discussion we had recently about efforts to make ray tracing a reality for video games. Daniel Pohl, a research scientist at Intel, takes us through the nuts and bolts of how ray tracing works, and he talks about how games such as Portal can benefit from this technology. Pohl also touches on the difficulty in mixing ray tracing with current methods of rendering. Quoting:
"How will ray tracing for games hit the market? Many people expect it to be a smooth transition - raster only to raster plus ray tracing combined, transitioning to completely ray traced eventually. They think that in the early stages, most of the image would be still rasterized and ray tracing would be used sparingly, only in some small areas such as on a reflecting sphere. It is a nice thought and reflects what has happened so far in the development of graphics cards. The only problem is: Technically it makes no sense."
Related Stories
[+]
Hardware: Real-time Raytracing For PC Games Almost A Reality 292 comments
Vigile writes "Real-time raytracing has often been called the pinnacle of computer rendering for games but only recently has it been getting traction in the field. A German student, and now Intel employee, has been working on raytraced versions of the Quake 3 and Quake 4 game engines for years and is now using the power of Intel's development teams to push the technology further. With antialiasing implemented and anisotropic filtering close behind, they speculate that within two years the hardware will exist on the desktop to make 'game quality' raytracing graphics a reality."
[+]
The Importance of Portal 222 comments
Team Fortress 2 and Episode Two may have been more anticipated elements of Valve's Orange Box offering, but it's the charmingly small Portal that's been getting a lot of attention in the last few days. MTV's Multiplayer blog thinks the game has the move of the year, and the Gamers with Jobs site offers up a convincing argument why Portal represents a significant step forward for storytelling in games: "Portal is an object lesson in interactive storytelling. We in the media are so fond of shaking our heads, scratching our beards and looking for the "art" in videogames. Well it's time for us all to shut the hell up. This is it. It's in this finely crafted, lovingly rendered piece of short-story literature. Honestly, I'd be surprised if the authors themselves see it as the accomplishment it is. It's a simple set of mechanics, a few pages of sound-booth dialog, a handful of textures and repetitive level designs. But then, a novel is only made up of 26 letters, black ink and white paper. And most artists of lasting brilliance don't recognize the importance of their own work. And how many now-revered musicians and painters died unknown and broke?" If you still haven't heard it, Jonathan Coulton's 'Still Alive' (the ending theme to Portal) has been in my head for over a week now. Just try to get it out of yours.
[+]
Mobile: Intel Researchers Consider Ray-Tracing for Mobile Devices 120 comments
An anonymous reader points out an Intel blog discussing the feasibility of Ray-Tracing on mobile hardware. The required processing power is reduced enough by the lower resolution on these devices that they could realistically run Ray-Traced games. We've discussed the basics of Ray-Tracing in the past. Quoting:
"Moore's Law works in favor of Ray-Tracing, because it assures us that computers will get faster - much faster - while monitor resolutions will grow at a much slower pace. As computational capabilities outgrow computational requirements, the quality of rendering Ray-Tracing in real time will improve, and developers will have an opportunity to do more than ever before. We believe that with Ray-Tracing, developers will have an opportunity to deliver more content in less time, because when you render things in a physically correct environment, you can achieve high levels of quality very quickly, and with an engine that is scalable from the Ultra-Mobile to the Ultra-Powerful, Ray-Tracing may become a very popular technology in the upcoming years."
[+]
NVIDIA Doubts Ray Tracing Is the Future of Games 198 comments
SizeWise writes "After Intel's prominent work in ray tracing in the both the desktop and mobile spaces, many gamers might be thinking that the move to ray-tracing engines is inevitable. NVIDIA's Chief Scientist, Dr. David Kirk, thinks otherwise as revealed in this interview on rasterization and ray tracing. Kirk counters many of Intel's claims of ray tracing's superiority, such as the inherent benefit to polygon complexity, while pointing out areas where ray-tracing engines would falter, such as basic antialiasing. The interview concludes with discussions on mixing the two rendering technologies and whether NVIDIA hardware can efficiently handle ray tracing calculations as well."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Adaptive techniques: make or break (Score:4, Interesting)
Re:Adaptive techniques: make or break (Score:5, Insightful)
"How will ray tracing for games hit the market?" (Score:5, Funny)
Now hear this (Score:5, Insightful)
See, the two are incompatible because the purpose is different. With games, the idea is "How realistic can we make something look at a generated rate of 30 frames per second". But with photorealistic rendering the idea is "How realistic can we make something look, regardless of the time it takes to render one frame."
And as time goes on and processors become faster and faster, the status quo for what people want becomes higher. Things like radiosity, fluid simulations and more becomes more expected and less possible to do in real time. So don't ever count on games looking like those still images that take hours to make. Maybe they could make it look like the pictures from 15-20 years ago. But who cares about that? Real time game texturing already looks better than that.
Re:Now hear this (Score:5, Insightful)
Re:Now hear this (Score:4, Insightful)
I don't have to look at the damn graph to tell you that what people are going to want is this [blenderartists.org]
And what they are going to get is this [pcper.com]
And, they should just be happy with this [computergames.ro] (which, is pretty awesome)
My point is that real time photorealistic rendering will never catch up with what people expect from their games. It will always be behind. If all you want is mirrors, then find a faster way to implement them at the expense of a bit of quality.
Re:Now hear this (Score:5, Insightful)
Yes, what can be produced will still be behind what people want or expect. But ray tracing will be less far behind than rasters in the near future.
All of this is according to TFA; I don't know much about this from a technical standpoint.
Re:Now hear this (Score:5, Informative)
One of the biggest hurdle in game graphics is geometry detail. Normal mapping is just a hack to make things appear more detailed, it breaks down in some situations. Raytracing will allow *much* higher geometry detail than rasterisation. Better reflection, refraction, caustics and so on are just gravy.
Re:Now hear this (Score:5, Interesting)
Furthermore, rasterization requires tricks (many would call them "hacks") to make the scene approach realism. In games today, shadows are textures (or stencil volumes) created by rendering more passes. While they look "good enough", they still have artifacts and limitations falling short of realistic. Shadows in raytracing come naturally. So do reflections, and refractions. Add some global illumination and the scene looks "real".
Rasterization requires hacks like occlusion culling, depth culling, sorting, portals, levels of detail, etc to make 3D engines run realtime, and some of these algorithms are insanely hard to implement for best case scenarios, and even then you're doing unnecessary work and wasting unnecessary ram rendering things you never see. Raytracing only renders what's on the screen.
That being said, I don't think raytracing will completely replace rasterization, at least not right away. Eventually, some games may incorporate a hybrid approach like most commercial renderers do today (scanline rendering for geometry, add raytracing for reflections and shadows). Eventually, 3D hardware will better support raytracing, and maybe in another decade we'll begin to see fast 3D engines that use ray tracing exclusively.
Re:Now hear this (Score:4, Insightful)
It's like arguing that we should go back to raycasting because it can render a textured cube many times faster than a 3D rasterized engine could.
You're being rather shortsighted.
Re:Now hear this (Score:4, Interesting)
Are there any reasons why current GPU designs can't be adapted for hardware assisted raytracing?
Re:Now hear this (Score:5, Informative)
No, they have a graph that very clearly shows that raytracing while using a binary tree to cull non-visible surfaces has a performance advantage over rasterizing while using nothing to cull non-visible surfaces. Perhaps someday a raster engine will regain that advantage by using these "BSP Trees" [gamedev.net] as well.
This isn't what we need in games (Score:5, Insightful)
I guess one has to state the obvious in that by moving to a process which is not implemented in silicon, as with current graphics cards, the work must necessarily be done in software. That means it runs on CPUs and that's something Intel is involved in where as when you look at the computational share of bringing a game to your senses right now, NVIDIA and ATI/AMD are far more likely to be providing the horsepower than Intel.
But really, even if this wasn't a vested interest case (and it may not be, no harm exploring it after all) - the fact remains that we don't actually need this for games. Graphics hardware has gone down an entirely different route whereby you write little shader programs which create surface visual effects on top of the bread and butter polygons and textures. This is a well established system by now and has a naturally compressive effect. It's like making all your visual effects procedural in nature rather than giving objects simple real-world textures and then doing a load of crazy maths to simulate reality. It works very well. Rememeber a lot of the time you want things to look fantastical and not ultra-realistic so lighting is some of the challenge.
Games aren't having a problem looking great. They're having a problem looking great and doing it fast enough and game developers are having a problem creating the content to fill these luscious realistic-looking worlds. That's actually what's more useful, really. Ways to aid game developers create content in parallel rather than throwing out the current rendering strategy adopted world wide by the games industry.
Re:This isn't what we need in games (Score:4, Insightful)
If the parallelization trend continues like it's progressing now, manicore CPUs are probable to arrive before 2010. Also, both AMD and Intel appear to be undertaking steps in the direction of enthusiast-grade multi-socket systems, increasing the average number of cores once again. Assuming raytracing can be parallelized as gread as TFA makes it sound, rendering could just return to the CPUs. I'm no expert, but it does sound kinda nice.
Re:Raytracing scales up far better... (Score:4, Insightful)
People keep saying this, that raytracing scales up better than rasterization. It's simply not true. Both of them have aspects that scale linearly and logarithmically. They do scale differently, but in a related sort of wy.
Raytracing is O(resolution), and O(ln(triangles)), assuming you already have your acceleration structures built. But guess what? It takes significant time to built your acceleration structures in the first place. And they change from frame to frame.
Rasterization is O(ln(resolution)), and O(triangles). Basically, in a rasterizer, we only draw places that we have triangles. Places that don't have triangles have no work done. But the thing is, we've highly pipelined our ability to handle triangles. When people talk about impacting the framerate, I want to be clear what we're talking about here: adding hundreds, thousands, or even a million triangles is not going to tank the processing power of a modern GPU. The 8800 Ultra can process in the neighborhood of 300M triangles per second. At 100 FPS, that'd be (not suprisingly) 3M triangles per frame.
Modern scenes typically run in the 100-500K triangles per frame, so we've still got some headroom in this regard.
Cheers.
Further Reading (Score:5, Interesting)
http://realtimecollisiondetection.net/blog/?p=38 [realtimeco...ection.net]
Re:Further Reading (Score:4, Insightful)
In short, I don't buy the summary article's viewpoint because at times he can be confusing or ambiguous with respect to his "proof." I like the parent's linked article, because the author of that article at least provides something computationally meaningful to think about.
How far we've come in just 15 years (Score:5, Interesting)
I do remember that someone found some shortcuts for raytracing, and I wonder if that shortcut is applicable to realtime rendering today. From what I recall, the shortcut was to do the raytracing backwards, from the surface to the light sources. The shortcut didn't take into account ALL reflections, but I remember that it worked wonders for transparent surfaces and simple light sources. I know we investigated this for our business, but at the time we also were considering leaving the industry since the competition was starting to ignite. We did leave a few months early, but it was a smart move on our part rather than continue to invest in ever-faster hardware.
Now, 15 years later, it's finally becoming a reality of sorts, or at least considered.
Many will say that raytracing is NOT important for real time gaming, but I disagree completely. I wrote up a theory on it back in the day on how real time raytracing WOULD add a new layer of intrigue, drama and playability to the gaming world.
First of all, real time raytracing means amazingly complex shadows and reflections. Imagine a gay where you could watch for enemies stealthily by monitoring shadows or reflections -- even shadows and reflections through glass, off of water, or other reflective/transparent materials. It definitely adds some playability and excitement, especially if you find locations that provide a target for those reflections and shadows.
In my opinion, raytracing is not just about visual quality but about adding something that is definitely missing. My biggest problem with gaming has been the lack of peripheral vision (even with wide aspect ratios and funky fisheye effects). If you hunt, you know how important peripheral vision is, combined with truly 3D sound and even atmospheric conditions. Raytracing can definitely aid in rendering atmospheric conditions better (imagine which player would be aided by the sun in the soft fog and who would be harmed by it). It can't overcome the peripheral loss, but by producing truer shadows and reflections, you can overcome some of the gaming negatives by watching for the details.
Of course, I also wrote that we'd likely never see true and complete raytracing in our lives. Maybe I'll be wrong, but "true and complete" raytracing is VERY VERY complicated. Even current non-real time raytracing engines don't account for every reflection, every shadow, every atmospheric condition and every change in movement. Sure, a truly infinite raytracer IS impossible, but I know that with more hardware assistance, it will get better.
My experience over the years was ALWAYS with static images that were raytraced. They looked great, but it wasn't until I experienced raytraced animations (high res, many reflective and transparent layers with multiple light sources and a sun-source) that I really saw the benefit and how it would aid in gaming.
The next step: a truly 3D immersive peripheral video system, maybe a curved paper-thin monitor?
Re:How far we've come in just 15 years (Score:5, Funny)
Holy Grail? Maybe not. (Score:4, Informative)
in the player's best interests, natch (Score:5, Insightful)
There, fixed that for you.
Raytracing the shiny first-intersection makes a lot of sense, even if it doesn't sell more CPUs. Sure, some day we will all have stunning holistic scene graphs that fit entirely within the pipeline cache of the processor, but it's not yet time for that.
Every change in developing a game engine requires changes in the entire toolset to deal with how to produce assets, how to fit within render time limit budgets, and how to model the scene graph and the logic graphs so that both are easily traversed and managed.
In the meantime, we have a pretty nice raster system right now, with a development strategy that provides for all those needs. You might not think that fullscale raytracing would upset this curve, but I'm not convinced. What do you do when a frame suddenly is taking more than 1/30sec to render, because the player is near a crystalline object and the ray depth is too high? How do you degrade the scene gracefully if your whole engine is built on raytracing? We've all played games where things like this were not handled well.
I contend that game AI is sometimes more advanced than academic AI because game developers are results-oriented and cut corners ruthlessly to achieve something that works well enough for a niche application. The same goes for game graphics: 33 milliseconds isn't enough to render complex scene graphs in an academically perfect and general way, it will require the same results-oriented corner-cutting to nudge the graphics beyond what anyone thought possible in 33ms. If that means using raytracing for a few key elements and ray-casting/z-buffering/fragment-shading the rest of the frame, game developers will do it.
Re:in the player's best interests, natch (Score:4, Interesting)
I contend that game AI is almost always laughably bad (or pretty much non-existent). I realize Mass Effect doesn't exactly win a lot of points for its AI, but the problems very nearly ruined a AAA-developer/large-budget game. I remember one point where, out of battle, I was telling one of my squad to go somewhere. There was pretty much one feature in the room - a wall intersecting the direct path to the target point. Getting around this wall would require one small deviation from the direct path. Instead of walking around the wall, the character just stood there "I'm going to need a transporter to get there" or something.
I can't imagine how the "AI" could have been implemented in order for that kind of failure to be possible (and common - I had repeated problems with this through the game). I assume they must have just cheated vigorously on the "follow" logic, as - if they'd used the same system - you'd be losing your squad around every corner.
Really, though, none of the maps were that complicated. The "navigable area" map for the problem location couldn't have had more than 200 or so vertices (it was a very simple map, one of the explorable bunker type areas). That's few enough that you could just Floyd-Warshall the whole graph. But, more generally, a stupidly naive, guessing DFS (that capped at 3 turns or something) would have worked just fine too. I can't think of a solution or algorithm that would fail the way their system did constantly. Mind-boggling.
Stepping back a bit, this shouldn't even be a consideration. There are simple, fast, robust algorithms that could handle this kind of trivial pathing problem without putting any strain on CPU or occupying more than a couple pages of code. That they don't have a better solution says that they (and most of the games industry, in my experience as a player) value AI at very close to 0.
General purpose CPUs: a REALLY bad way to do this. (Score:5, Interesting)
Here's a debate between Professer Slusallek and chief scientist David Kirk of nVidia: http://scarydevil.com/~peter/io/raytracing-vs-rasterization.html [scarydevil.com] .
Here's the SIGGRAPH 2005 paper, on a prototype running at 66 MHz: http://www.cs.utah.edu/classes/cs7940-010-rajeev/sum06/papers/siggraph05.pdf [utah.edu]
Here's their hardware page: http://graphics.cs.uni-sb.de/SaarCOR/ [uni-sb.de]
Not ray tracing, radiosity (Score:5, Interesting)
It's amusing to read this. This guy apparently works for Intel's "find ways to use more CPU time" department. Back when I was working on physics engines, I encountered that group.
Actually, the Holy Grail isn't real time ray tracing. It's real time radiosity. Ray-tracing works backwards from the viewpoint; radiosity works outward from the light sources. All the high-end 3D packages have radiosity renderers now. Here's a typical radiosity image. [icreate3d.com] of a kitchen. Radiosity images are great for interiors, and architects now routinely use them for rendering buildings. Lighting effects work like they do in the real world. In a radiosity renderer, you don't have to add phony light sources to make up for the lack of diffuse lighting.
There's a subtle effect that appears in radiosity images but not ray-traced images. Look at the kitchen image and look for an inside corner. Notice the dark band at the inside corner. [mrcad.com] Look at an inside corner in the real world and you'll see that, too. Neither ray-tracing nor traditional rendering produces that effect, and it's a cue the human vision system uses to resolve corners. The dark band appears as the light bounces back and forth between the two corners, with more light absorbed on each bounce. Radiosity rendering is iterative; you render the image with the starting light sources, then re-render with each illuminated surface as a light source. Each rendering cycle improves the image, until, somewhere around 5 to 50 cycles, the bounced light has mostly been absorbed.
There are ways to precompute light maps from radiosity, then render in real time with an ordinary renderer, and those yield better-looking images of diffuse surfaces than ray-tracing would. Some games already do this. There's a demo of true real-time radiosity [dee.cz], but it doesn't have the "dark band in corners" effect, so it's not doing very many light bounces. Geometrics [geomerics.com] has a commercial real-time game rendering system.
Ray-tracing can get you "ooh, shiny thing", but radiosity can get to "is that real?"
Nice introduction, but wrong conclusion (Score:4, Informative)
In fact, his points actually show why a hybrid is perfect: most surfaces are not shiny, refractive, a portal, etc. Most are opaque - and a rasterizer is much better for this (since no ray intersection tests are necessary). He shows pathological scenes where most surfaces are reflective; however, most shots do show a lot of opaque surfaces (since Quake 4 does not feature levels where one explores a glass labyrinth or something).
Yes, if a reflective surface fills the entire screen, its all pure ray tracing - and guess what, that is exactly what happens in a hybrid. Hybrid does not exclude pure ray tracing for special cases.
Ideally, we'd have a rasterizer with a cast_ray() function in the shaders. The spatial partitioning structure could well reside within the graphics hardware's memory (as an added bonus, it could be used for predicate rendering). This way haze, fog, translucency, refractions, reflections, shadows could be done via ray tracing, and the basic opaque surface + its lighting via rasterization.
Now, I keep hearing the argument that ray tracing is better because it scales better with geometric complexity. This is true, but largely irrelevant for games. Games do NOT feature 350 million triangles per frame - it just isn't necessary. Unless its a huge scene, most of these triangles would be used for fine details, and we already have normal/parallax mapping for these. (Note though that relief mapping usually doesn't pay off; either the details are too tiny for relief mapping to make a difference, or they are large, and in this case, traditional geometry displacement mapping is usually better.) So, coarse features are preserved in the geometry, and fine ridges and bumps reside in the normal map. This way, triangle count rarely exceeds 2 million triangles per frame (special cases where this does not apply include complex grass rendering and very wide and fine terrains). The difference is not visible, and in addition the mipmap chain takes care of any flickering, which would appear if all these details were geometry (and AA is more expensive than mipmaps, especially with ray tracing).
This leaves us with no pros for pure raytracing. Take the best of both worlds, and go hybrid, just like the major CGI studios did.