Folks may remember from XDS 2008 that I was working on a rewrite of Mesa's GLSL compiler. To make a long story short, let's just say that effort fell over and sank into the swamp.

Last December, I started building another one. I had called the working directory of the first compiler glsl, so this one, being take 2, was called glsl2. After I had been working on it for a couple months, Eric Anholt and Ken Graunke (who has since joined our group at Intel) started working on it with me. We've come a long way.

I expect we'll take some flak for working in semi-secret. I'll take the blame for that one. The first attempt at rewriting the compiler was widely publicized and turned into a disaster. In spite of it being widely publicized, I don't think anyone ever pulled the tree. I decided that I wanted to just keep my head down and work on this one. The announcements can come when there's actually something to announce. I think we're close enough to having something to announce to make it worth the effort.

At this point, the front-end (parsing, semantic checking, generation of the initial high-level IR) is mostly complete. We support all of GLSL 1.20 and most of GLSL 1.30. We've temporarily punted on switch-statements, the float (vs. vec4) variants of the shadow sampler functions, and the "offset" versions of all the texture look-up functions. Each of those is going to require a bit more thought to implement correctly.

Carl Worth has written a preprocessor for us, but it hasn't been integrated into the tree yet. As was mentioned on the Mesa mailing list, without the preprocessor we're passing over 90% of the piglit tests.

There's still a lot left to do.

  • Linker: The single biggest thing remaining is the linker. The compiler and the IR are structured such that we can actually have a real linker (instead of just concatenating multiple sources and recompiling), but that means we have to write a real linker.
  • More optimizations: Our primary target is i915-class hardware. In order to get GLSL to run on hardware that doesn't have branches, there are a bunch of optimizations, like loop-unrolling, that you have to do. We still need to do some of those.
  • Preprocessor: We still have to integrate Carl's preprocessor. I believe that is being done this will still.
  • Memory management: Any objects that are no longer needed get dropped on the floor. We leak like a screendoor on a submarine. This was an intentional design decision. I had planned to eventually write some sort of slab allocator. At the end of compilation, the entire slab would be deleted. Eric and Carl have since convinced me to just use talloc.
  • Codegen: Eric is writing a back-end that will generate Mesa's existing low-level IR from our IR. I believe this is getting close to being done enough.
  • Mesa: Once all that stuff is done, we need to get our stand-alone compiler in the Mesa tree. We're expecting it to live in a branch for a few months while things stabilize. It should make for interesting times.

The primary repo is my GIT tree. Eric, Ken, and Carl all have their own trees, and I pull from them when their stuff is ready.

git://anongit.freedesktop.org/~idr/glsl2

For those interested in help things along, the biggest thing we need are more test cases. We're especially interested in shaders from real applications that are known to work correctly on other drivers. We have a bunch from gstreamer and other projects, but more is better. If you have some shaders and don't mind releasing them under a piglit compatible license, please send them our way.

i915-class?!
As in like the i945 in my laptop?! That would be interesting (in a good way)... This might speed up Flash http://blogs.adobe.com/penguin.swf/2008/05/flash_uses_the_gpu.html . Would it also provide enough support so super new OpenGL tutorials like http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Table-of-Contents.html would be supported on i915-class hardware?
Comment by Anon Fri 11 Jun 2010 06:49:23 AM PDT
Benefits
What are the benefits of the compiler rewrite? Will it make the codebase more compact/streamlined? Will it enable support of higher GLSL versions? Will it improve performance (if so, how much have you measured?)?
Comment by Anonymous Fri 11 Jun 2010 03:33:27 PM PDT
RE: i915-class?!
i915 can't do full GLSL, but it can handle the subset (mostly changes regarding loops and if-statement) in OpenGL ES 2.0. We want to enable that.
Comment by IanRomanick Sat 12 Jun 2010 10:38:22 PM PDT
RE: Benefits

The current compiler is not very good. It was written by Michal Krol as his final project at university. Given that, it's pretty amazing how well it works. However, the list of things that need to be fixed is long and runs from the frontend through the backend. Michal and I had a long discussion about it at XDS 2008. He really wanted to rewrite it, but he was (and presumably still is) too busy with other work.

I don't want to make any performance statements since we don't even have the compiler in Mesa. However, the current architecture should make it quite a bit easier to write new optimization passes. This is quite difficult in the current Mesa compiler. We're basing the architecture largely on "Advanced Compiler Design and Implementation" by Steven Muchnick.

Comment by IanRomanick Sat 12 Jun 2010 10:48:18 PM PDT