I got to the Textures session a bit late because I went to Nvidia's "OpenGL 4.0 for 2010" session, and the session went over by 20 minutes. It was a good overview of OpenGL 4.0 / 4.1. Nvidia (Mark Kilgard, more precisely) continues to spread mis-information about the compatibility profile, and that's really frustrating.

I missed all of By-Example Synthesis of Architectural Textures and everything except the results of Synthesizing Structured Image Hybrids. The results in the later were very cool. I especially liked the synthesized pirate flags.

Every year there's some piece of cool tech at SIGGRAPH that I want to implement. I think Vector Solid Textures is it this year. It's a very clever method of compactly storing volumetric textures as a sort of voxelized SVG. The complex step, of course, is generating the texture data. Any algorithm that utilizes sub-algorithms with names like "L-BFGS-B minimizer" and "teleportation" is bound to be frightening. lol.

The Mesh Colors presentation started with an excellent overview of all the problems with 2D texture mapping. If he had stopped there, I would have called it a useful presentation. Of course, he didn't stop there. The idea behind mesh colors is that a set of colors are stored for each triangle (one per vertex, one per edge, and N per triangle) along a tessellation grid (basically). At run-time the UV value is used to weigh the nearest colors. The frustrating bit of the presentation was the massive amount of hand waving (shown in the photo below) about the implementation. I suspect the details are in the paper. The hair farm shirt was a nice touch.

It also seems like there is a lot of area for future research. The filtered performance is pretty poor. I had a brief discussion with Cem about using hardware tessellation (to a level that matches the mesh colors pattern) to solve some of the filtering issues. It seems like it could work in some cases, but it presents problems of its own (i.e., potentially a lot more work for the rasterizer).

I may have to implement this algorithm too.

There were a bunch of cool things in the poster session, and I took the opportunity today to talk to some of the authors. There were several useful posters about soft-shadow algorithms, a couple about image enhancement, and a couple about ambient occlusion. I especially liked Curvature depended Real-Time Ambient Occlusion (sic). They use a preprocess to determine the concave curvature at each vertex. At run-time, this is interpolated, and the interpolated to determine the AO. The interpolation here gives a better result vs. per-vertex AO in the same way that interpolating light parameters gives better results than interpolating per-vertex light calculations.

My two favorite posters were GPU Ray Casting of Virtual Globes and, I kid you not, A Physical Rendering Model for Human Teeth. I plan to add both of these to my VGP curriculum. The tooth shader is useful because it shows a case of a lighting model developed for a specific type of surface.

The other poster that really caught my eye was the WebGLU poster. It sounds like Benjamin is doing a lot of the same things for WebGL that I'm doing for desktop GL. He and I talked, and there may be some colaboration in our future.

There are some things to really like about Open Shading Language. Being able to arbitrarily connect shaders (via matching inputs and outputs) in a DAG is something that I've been talking about doing in OpenGL for years. Pulling data out by specifying an arbitrary expression on shader outputs is also genius. In OSL the data gets dumped as an image, but I still see use for this in GL.

The implementation of automatic differentiation is really, really smart. I suspect that something like this could even be implemented in our hardware shaders. This would allow higher quality derivatives in some cases. I guess I need to read the Piponi paper from journal of graphics tools.

And they now use LLVM.

Since this system has texture mapping support in this system, this might make an interesting test bed for my thesis work. Hmm...

REYES Using DirectX 11 wasn't too interesting to me. During the Q-and-A period, someone (the previous presenter, in fact) asked what could be changed in either DirectX or the hardware to make this either easier or faster. Being able to insert compute shaders directly into the pipeline (to eliminate the intermediate buffers, synchronization, etc.) would have helped a lot. This isn't the first time I've heard a request for this.

The WebGLot: High-Performance Visualization in the Browser guys are doing some really cool stuff on top of WebGL. We'll need to use this app to test WebGL on our drivers!

The Gazing at Games: Using Eye Tracking to Control Virtual Characters course didn't feel like a course. It felt like a presentation of a survey paper on what people have done related to eye tracking in games. I got references to a bunch of papers that seem interesting, but that's about it. I do like the idea of "smart text." Text (dialog, etc.) is displayed until you look away from it. I had to leave early to go to the OpenGL BoF, so I probably missed the best parts. shrug

The highlight of the OpenGL BoF was, of course, the announcement of GLU3 as part of the OpenGL SDK. Getting a round of applause was cool.

One of the trivial questions was, "What ARB extension has not been shipped in any publicly available driver?" Answer? GL_ARB_shading_language_include. Damnit!!! Okay, we need to add that to our compiler todo list. Ugh. It's a cool feature, and John Kessenich, Jon Leech, and I spent a lot of time working on it. In all fairness, GLSL needs a much better module mechanism than it currently has. I have some ideas, but they'll have to wait until our new compiler actually ships at least GLSL 1.30.

The Importance Sampling for Production Rendering course was interesting, but a lot of it was review. I hadn't noticed that the first presenter in the course was also the author of the GPU-Based Importance Sampling from GPU Gems 3 that I use in VGP352. The bits about multiple important sampling and the bits about resampled importance sampling for shadows from the second presenter were new to me. There was a question from the audience about the Lafortune BRDF. Matt's response echoed what I say in VGP352 every year. He says that the Lafortune BRDF is not used at IMD beause it is "too uncontrollable for many artists."

I don't really know anything about REYES, so most of Micropolygon Ray Tracing With Defocus and Motion Blur was lost on me. The hyper-trapezoid BVH was interesting, though. Real-Time Lens Blur Effects and Focus Control was similarly a loss. The foreground occlusion image may find a place in VGP352 when I talk about depth-of-field effects. OptiX: A General Purpose Ray Tracing Engine was mostly an advertisement for Nvidia's ray tracing engine. There were a couple interesting bits about their compiler architecture, but meh.

To this point, I wish I would have gone to the An Introduction to 3D Spatial Interaction With Videogame Motion Controllers session instead.

That changed with Reducing Shading on GPUs using Quad-Fragment Merging. One of the opening tidbits will annoy keithp to be sure. "...one of the better ideas of real-time rendering [is] multisample anti-aliasing." In any case, their method of merging quads (2x2 pixel blocks processed by the fragment shading hardware) is fairly straight forward, but still clever. They track facing and connectivity for covered pixels in each quad. Multiple quads that have connected triangles with the same facing are merged. This can save a lot of processing.

Much to my joy, the Making of TRON: Legacy session got moved to Tuesday evening... when it didn't conflict with anything that I wanted to attend! w00t! Yes folks, that is the line...

EIGHT MINUTES of footage from the movie. A new trailer (I want one of those lightcycle throw pillows). Full on awesome.

A couple interesting bits:

• Daft Punk is doing the music, and they have a cameo.

• Some of the concept drawings from the original movie that couldn't be done on computers at the time was used. For example, the original lightcycles had the rider on the outside. There was no way the computers of 1982 could render that, so the design was changed.

• It was filmed in 3D, but, in the words of the director, there's no "spear poking you in the eye" business.

In the Q-and-A session I asked a semi-smart ass question. That's how I roll. "My biggest disappointment with the film industry came when I was 8 and the original TRON wasn't nominated for an Oscar because it "cheated" [by using computers]. Come January, is it going to be lightcycles or broomsticks? TRON or Harry Potter?" Yes, I know the Oscars are in March. I was nervous. Unfortunately, their answer was way, way too serious. "Ultimately that's up to you, but we think our work will stand on it's own." meh.

UPDATE: One thing I forgot before, there was an off-hand mention by the director that some effort was going into doing a "re-imagination" of The Black Hole. That was such a creepy movie! A re-make could be awesome... or horrible.

This year I was finally smart enough to register at the conference the day before it starts. Yay me. That made is so much easier the first day.

I wasn't expecting a lot from envLight: An Interface for Editing Natural Illumination, but I was pleasantly surprised. There were a couple of surprises in their results. I keep meaning to implement some sort of parameter editing interface for my BRDF demos, and I would have done it quite differently before seeing this paper.

I understand the method in Interactive On-Surface Signal Deformation, but I don't really get how it would be implemented in a real system. I guess that's what I get for not reading the paper before attending the presentation. I would have read the paper if I had remembered to bring the DVD drive for my laptop. That's the one annoying thing about the X200s... no built-in optical drive. In any case, it seems like this approach could be used to great effect in a demo. I imagine a scene where, say, the shock wave from a heavy object hitting the ground causes shadows on the ground to get pushed away.

My first Avatar presentation, PantaRay: Fast Ray-Traced Occlusion Caching was impressive. I guess rendering scenes with 10M to ONE BILLION DOLLARS... er... one billion polygons needs some new acceleration techniques. And a petabyte of "live data" needs some kick ass I/O. There are so many different optimizations implemented here that I think they could have gotten multiple papers out of it. Seriously. Just the layers of optimizations in the spatial index calculation should have been sufficient. Showoffs!

On a whim, I went the SIGGRAPH awards presentation. The cool thing there was a visualization app that shows, on a map of the world, where every technical paper ever published at SIGGRAPH originated. It will supposedly be posted on the web, so I'll link it here soon. The talk by Don "The Tornado" Marinelli really made it worth it, though. You can tell that he comes from a drama background. They also said that some of the content would be posted on YouTube, so his talk may get linked here later.

The word(s) of the day: deferred shading. Screen Space Classification for Efficient Deferred Shading was the first of the bunch. The basic technique is to read back the G-buffer to generate a batch of polygons to draw. The screen is divided into tiles, and the tiles are classified by attributes (e.g., in shadow, direct lit, etc.). On both PS3 and Xbox360 it cuts the frame time by about half. They talked a bit about the optimization of the polygon submission. Since the polygons are tile aligned, it seems like some sort of quadtree could be generated and rendered using point sprites. I'll have to think on that a bit more.

"Boom. 60 is better." lol. How to Get From 30 to 60 Frames Per Second in Video Games for "Free" uses the Z-buffer and the velocity buffer from the current frame and the color buffer of the previous frame to generate an in-between frame. The author was inspired by MPEG motion vectors. Here, it's a big hack. It has a bunch of hacks on top of it to fix artifacts. The best part is the way dynamic objects are "erased" from the frame (this is only necessary with deferred shading). I like it.

Split-Second Motion Blur covered some additional work from Split Second. They render a per-pixel 2D motion vector ID. This ID is then used in a blur post-process. In addition, they update the texture sampling derivatives based on the motion vector. Cheap, easy, good results. I'll definitely add this to the motion blur techniques covered in VGP351 next time it comes around.

Over the last couple years systems for indirect lighting in real-time have been all the rage. Pretty much all of them have used deferred shading. A Deferred-Shading Pipeline for Real-Time Indirect Illumination is shock no different. However, they're algorithm has good quality and speed. The other algorithms that I have seen pick one or the other. They incorporated a clever way to include occlusion for bounces, but this costs a lot of memory. I guess that's the trade off. The method for mipmapping the G-buffer was pretty clever. At each downscale they average the attributes from the largest object. This is sort of a bilateral filter in reverse. It's still screen space, so it has some drawbacks:

• losing indirect light from an object that goes off screen

• no support for mirrors reflecting objects behind the camera

Also, if I hear "current console hardware can't" one more time, I'll puke... or just roll my eyes again. I knew realize dynamic branching was bad on PS3, but I didn't it was so bad on Xbox360. sigh...

I wrapped up the day with Live Real-Time Demos. As Eric Haines notes, they were pretty cool. Thinking back to my days as a demo coder, it brings a tear to my eye to see demos on the big screen at SIGGRAPH.

The 0.9 release is now available! I'm calling this 0.9 because not all of the functionality is available in the C interfaces. Once that is resolved and a couple other GLSL utilities are complete, I'll call it 1.0.

There is a tag in the GIT repo for the release. The glu3-0.9.1 tag is the "real" tag. I pushed the glu3-0.9 before doing make distcheck, and, of course, there were errors.

md5sums:

75032c9aaf2d279738ce3940383439ff  GLU3-0.9-20100721-bin.zip
63ccdbdb8d6799b9a530b661f1d9d0f7  GLU3-0.9-20100721-src.zip
862b22f67dd7d6f087ff6ba497653621  GLU3-0.9-20100721.tar.bz2
6266c6944128eec4149c8b8101e290df  GLU3-0.9-20100721.tar.gz

I've been pretty busy over the last few months, so I haven't had a chance to really comb through the SIGGRAPH program. There are several good overviews at the Real-Time Rendering Blog.

My plan feels full, but there is still time left for posters and the expo. I think I'm doing it right. There are a couple courses and some of the Avatar talks that look interesting on Sunday, but, seriously? Sunday? FAIL.

Monday

• Lighting & Material Design (9:00AM - 10:30AM)

  There are several presentations about PantaRay in conjunction with all the sessions (at least a dozen!) about Avatar. PantaRay: Fast Ray-Traced Occlusion Caching in this session looks good.

• Split Second Screen Space (2:00PM - 3:30PM)

  All four papers in this session sound interesting. I'm especially interested in Screen Space Classification for Efficient Deferred Shading and Split-Second Motion Blur. Unfortunately, I don't see a preprints for either one online.

Tuesday

• Importance Sampling for Production Rendering (course) (9:00AM - 10:30AM)

  I've been wanting to include some coverage of this topic in VGP352 for the last couple years. I already talk about GPU-Based Importance Sampling from GPU Gems 3, but I'd like to have a bit broader coverage.

• GPU Rendering (2:00PM - 3:30PM)

  Reducing Shading on GPUs Using Quad-Fragment Merging and Real-Time Lens Blur Effects and Focus Control look to be the highlights in this session for me.

  As usual at SIGGRAPH, there are big periods of nothing for me, and big periods where there are 37 things I want to see. Also at 2:00 is An Introduction to 3D Spatial Interaction With Videogame Motion Controllers. Since this session runs until 5:00PM, I might just drop in late.

Wednesday

• Textures (10:45AM - 12:15AM)

  I like the looks of all the papers in this session. By-Example Synthesis of Architectural Textures

and Synthesizing Structured Image Hybrids are probably the most interesting to me. However, Vector Solid Textures and Mesh Colors also look good.
• APIs for Rendering (2:00PM - 3:30PM)

  I posted about Open Shading Language a few months ago. I'd like to hear more about what they've done... especially since I've been off in compilerland this whole year.

• Perception, Presence & Animation (2:00PM - 3:30PM)

  I might try to drop in for Using Blur to Affect Perceived Distance and Size.

• OpenGL BoF (5:15PM)

  Duh.

Thursday

• Large Steps Toward Open Source (panel) (9:00AM - 10:30AM)

  I feel obligated to attend this panel.

• Games & Real Time (10:45AM - 12:15AM)

  Both Practical Morphological Anti-Aliasing on the GPU and Curvature-Dependent Reflectance Function for Rendering Translucent Materials are high on my list. I think the later paper had a poster last year. Whether it was them or not, there was an cool posted about a similar technique.

  I may need to bail to go to The Making of "TRON: LEGACY".

• Global Illumination Across Industries (2:00PM - 5:15PM)

  GI has always been fascinating to me, and I'm so happy to live in a time where real-time GI is just about within reach.

Several coworkers and I have been working on a compiler project for the last six months or so. We're implementing it in C++, and, for most of us, it's our first real C++ project. We're using various flavors of visitors¹ for a lot of our IR processing. In quite a few places we have visitors that just do stuff but have no value. To invoke them some code that looks something like:

    ir_validate_tree v;
    v.run(instructions);

Often these are wrapped in a function. In other places we have things like:

    ir_swizzle_swizzle_visitor v;
    v.run(instructions);

    /* Do something with v.progress */

The latter form is always wrapped in a function that just returns v.progress.

I find the first form to be particularly ugly. Wrapping it in a function makes using it less ugly, but the ugly is still there, lurking in the shadows.

It occured to me today that at least the first form could be "fixed" by using the constructor as the run method. Instead of the code above, there would be a constructor that had the code from the run method, and callers would just do:

    ir_validate_tree(instructions);

Something similar could be done with the second case, but that would look like:

    progress = ir_swizzle_swizzle_visitor(instructions).progress || progress;

My conundrum, as a C++ newb, is whether or not this is a common idiom. Would someone familiar with C++ (at least more familiar than I am) look at that code and know what was going on? Or would they just think the author was a clueless newb?

1: We primarily use hierarchical visitors.

Folks may remember from XDS 2008 that I was working on a rewrite of Mesa's GLSL compiler. To make a long story short, let's just say that effort fell over and sank into the swamp.

Last December, I started building another one. I had called the working directory of the first compiler glsl, so this one, being take 2, was called glsl2. After I had been working on it for a couple months, Eric Anholt and Ken Graunke (who has since joined our group at Intel) started working on it with me. We've come a long way.

I expect we'll take some flak for working in semi-secret. I'll take the blame for that one. The first attempt at rewriting the compiler was widely publicized and turned into a disaster. In spite of it being widely publicized, I don't think anyone ever pulled the tree. I decided that I wanted to just keep my head down and work on this one. The announcements can come when there's actually something to announce. I think we're close enough to having something to announce to make it worth the effort.

At this point, the front-end (parsing, semantic checking, generation of the initial high-level IR) is mostly complete. We support all of GLSL 1.20 and most of GLSL 1.30. We've temporarily punted on switch-statements, the float (vs. vec4) variants of the shadow sampler functions, and the "offset" versions of all the texture look-up functions. Each of those is going to require a bit more thought to implement correctly.

Carl Worth has written a preprocessor for us, but it hasn't been integrated into the tree yet. As was mentioned on the Mesa mailing list, without the preprocessor we're passing over 90% of the piglit tests.

There's still a lot left to do.

• Linker: The single biggest thing remaining is the linker. The compiler and the IR are structured such that we can actually have a real linker (instead of just concatenating multiple sources and recompiling), but that means we have to write a real linker.
• More optimizations: Our primary target is i915-class hardware. In order to get GLSL to run on hardware that doesn't have branches, there are a bunch of optimizations, like loop-unrolling, that you have to do. We still need to do some of those.
• Preprocessor: We still have to integrate Carl's preprocessor. I believe that is being done this will still.
• Memory management: Any objects that are no longer needed get dropped on the floor. We leak like a screendoor on a submarine. This was an intentional design decision. I had planned to eventually write some sort of slab allocator. At the end of compilation, the entire slab would be deleted. Eric and Carl have since convinced me to just use talloc.
• Codegen: Eric is writing a back-end that will generate Mesa's existing low-level IR from our IR. I believe this is getting close to being done enough.
• Mesa: Once all that stuff is done, we need to get our stand-alone compiler in the Mesa tree. We're expecting it to live in a branch for a few months while things stabilize. It should make for interesting times.

The primary repo is my GIT tree. Eric, Ken, and Carl all have their own trees, and I pull from them when their stuff is ready.

git://anongit.freedesktop.org/~idr/glsl2

For those interested in help things along, the biggest thing we need are more test cases. We're especially interested in shaders from real applications that are known to work correctly on other drivers. We have a bunch from gstreamer and other projects, but more is better. If you have some shaders and don't mind releasing them under a piglit compatible license, please send them our way.

Last night Rose and I went to see w00tstock at the Aladdin Theater. It was freakin' awesome. If you live in Chicago or Minneapolis (the next stops on the tour), you need to go see it when it comes to town. I have to admit, other than Adam Savage and Wil Wheaton, I hadn't heard of any of the acts before the show. I was a bit skeptical about the whole thing. Alas. It was 4+ hours of win. I know, it's billed as "3 Hours of Geeks & Music," but it went way over.

Since they expressly released the content of the show under a Creative Commons NC-BY license, the whole show has probably been posted, bit by bit, to YouTube by now.

Even though the show went hours over time, I stayed until the very end for autographs. (Note: I'll post pictures later.) The autographs were being done from the bar area where the Aladdin usually serves pizza. The line went across the lobby, up the stairs, across the balcony, down the other stairs, and out on to the side walk. I was fourth from last. I get up to the front, have a little chit chat with Paul and Storm, but I'm really waiting to meet Adam. Of course, by the time I got there, I couldn't think of "my favorite episode." It came to me this morning. Three words: exploding cement truck.

Instead, all I can think of is Tori whacking his knee. That episode drove me nuts! They kept replaying that clip over and over again. Every time I got this shot of sympathy pain in my left knee... that I damaged in a skiing accident. So, that's what I told Adam about. As soon as I mentioned that episode, his facial expression changed. "I wasn't at that shoot," he tells me, "but I soon as I saw that set-up I said, 'What the hell are they thinking?'" And then we had a decent conversation about it. It was really cool.

Am I the only one terrified by the thought of RMS making educational games for children?

Over the weekend my wife and I went to see How to Train Your Dragon. Holy crap. It was really good. For the type of movie that it is, I think it was just about perfect. Coming from me, that's saying something. I can pretty much always find something to complain about.

Not only was the movie great, but the rendering went above and beyond! I was expecting the fire to be the impressive rendering feat, but it was the water and the clouds that impressed me. Of course, it is hard to top the fire from The Half-Blood Prince. I think the best shot was towards the end of the movie. The Vikings had arrived at an island by ship. One of the dragons was going breathe fire over the ships. The camera cut to a shot from under the water looking straight up. It's hard to explain, but it was freakin' cool. It's about 3 seconds after the image below.

And yeah, the fire is pretty damn sweet in that shot.

About a minute into the trailer are some good shots of the water (surface of the lake) and the clouds.