RenderAnts - Interactive REYES Rendering on GPU

This has been floating around the net recently, something that is actually quite impressive for what it does as well as what it could be. Interactive REYES rendering on a GPU, which really in a sense has been something a lot of people and studios have been looking for.




CG Society Link

BlenderArtist Link

Authors page:
http://www.kunzhou.net/

Paper:
http://www.kunzhou.net/2009/renderants.pdf

Video:
http://www.kunzhou.net/2009/renderants.wmv

There have been other similar types, such as Pixar's LPics which was featured after Cars was released, as well as Lightspeed which ILM had developed during the Transformer production. However the difference was these used GL shader equivalents of RSL shaders, so they both did not really use a Renderman based rendering. Both were very impressive though.

Gelato was also something that had been designed for such a purpose but was discontinued after a few years, certian tools did have the ability to convert basic RSL shaders into it's own shader language so in a sense it was sort of a start of what could be. Larry Gritz, the same person who had developed the first non Pixar REYES renderer, BMRT, had developed Gelato. So maybe that was another reason for Gelato being non REYES based considering the legal issues between Gritz and Pixar in previous years.

RenderAnts is a GPU based REYES rendering system, using RIB and RSL code to render the resulting image from the GPU, rather than the traditional CPU software we currently use. The ability to get fast rendering feedback is always a great thing, the only current way to do this is to render smaller sized images along with turning down the detail features of REYES, or do something like Crop rendering which will only render a certain region. This does in fact make an image render faster but if you are concerned about details, or lighting changes, having to render out a new image just to see if something works or not is quite a painfully slow task. This is why RenderAnts is a huge deal. It is not because of the fact that Elephants Dream was used to showcase the speed difference of normal CPU based rendering versus GPU, though it was pretty cool to see that. Elephants Dream was used mainly because it is Open Content, these were fully animated scenes that can be used for any reason within legal bounds.

What makes it so interesting for us, the Blender to Renderman users and developers, is that Mosaic was used to export these scenes out. This is why open source Blender to Renderman is important, it can be used for research, not only production. It is far easier and cheaper to use Blender, Mosaic and Aqsis or Pixie to showcase some new 3D research where you have access to the source code and can make your research possible, than it is with closed source commercial software. At best you can make a plugin for Maya if you were to make something like say a new form of fluid simulation that used a custom RSL volume shader. You would also only be able to do this on one system, while with open source you can have several copies spread out over a network, even at home.

This is the first time Mosaic has been officially used and cited in a published research paper.

If you watch the video make sure to notice that this NOT real time, it is fast but it does not have the ability to render at even 1 fps. At best it does take a few seconds, the few that do look fast are more like camera placement changes or lighting changes. Anything really drastic does seem to take a bit longer to render. However considering the same frame using the same data would take a considerable amount of time for PRMan to render does say quite a bit. What this also means is that this is not to replace current software for final frame rendering, at least not for a while. The best use for such a system is for previewing during production, the little changes that artists and TD's make for instance. Something so tedious like shader development would cut such time in half, making 30 renders of minute changes in the shader is a very time consuming task. It is not hard to imagine that this will be used by the big boys very soon, it is also only a matter of time before a commercial adaptation of this is released in the next few years.

We just have a nice warm feeling knowing that our work here has helped in this, we were used first. THAT is something.

Aqsis 1.6 and Project Widow

Ack! I have been very behind! I recently moved (again) and am also in the process of remodeling a house as well so my time has been limited, obviously when BlenderNation reports news before we do. Not to mention the link to this site as well. Anyways off to the subject at hand.

Aqsis 1.6



Aqsis has undergone some serious changes since version 1.4 and a lot of it has been to improve it's speed and stability. Copied directly from the press release :

General optimisation and performance has been the primary focus for this release, with improvements including:

• Avoiding sample recomputation at overlapping bucket boundaries.
• Refactored sampling and occlusion culling code.
• Enabled "focusfactor" and "motionfactor" approximation by default, for depth-of-field and motion blur effects respectively.
• Improved shadow map rendering speed.
• Faster splitting code for large point clouds.

In addition, key feature enhancements have been made with improvements including:

• Multi-layer support added to OpenEXR display driver.
• Side Effects "Houdini" plugin.
• New RIB parser, supporting inline comments and better error reporting.
• Matte "alpha" support, for directly rendering shadows on otherwise transparent surfaces.
• Refactored advanced framebuffer (Piqsl).
• Texturing improvements.
• Enabled "smooth" shading interpolation by default.

Now to get the point. One of the main additions to Aqsis, the MultiLayer OpenEXR, was from the request of the team that is working on Project Widow. The reason for this of course is because Blender's Compositor can use this directly, much like the way it can with it's own EXR render. This was to facilitate an easier workflow later on during the composite stage, rather than have a mess of multiple image sequences for each and every single AOV render we wanted. Also because of the talks between the Widow team and the Aqsis team, Mosaic was also built to handle this very function. In the latest CVS version of Mosaic there is a much larger menu selection of display drivers available than in previous versions. So Blender, Aqsis and Mosaic all work hand in hand in various stages of the pipeline now, rather than just rendering. Since we used Aqsis for preview renders as well, it was important for us to have the speed and stability. The Piqsl framebuffer was also a request from us working on Widow, we wanted to have the ability to scroll through images using the arrow keys rather than clicking on each render, this saved us a lot of time when working on previews and rendering dozens of images. We also tested Aqsis quite a bit through out the process, though now that it is fully released we can use the "production stable" version rather than the daily builds or sources.




Above is an example of the AOV multi layer EXR renders



Composite Nodes


During the months of pre-production of Widow, all of us would gather in an IRC chat room and discuss ideas that we wanted from Aqsis, also to get feedback over how to work with this or that in the rendering end. Planning for a renderfarm had begun and was tested over the summer, even building a new script tool so that DrQueue could use Mosaic batch output. We also had to design a lot of the assets from Aqsis in, by that I mean the process of figuring out how to make Blender work with what we wanted. There were some ideas scraped simply because of the limits currently imposed by the Python API.

So now that we have covered that...

Project Widow



This short has taken a LOT longer than planned, the idea was to get this done in 3 months starting in May of this year. It is now October. So yes things are way behind but that does NOT mean that it is stopped. At the moment it is at a standstill because there are so few of us working on it but also I have had a lot of real life situations that prevented me from devoting as much time as I want to it. There also has been quite a bit of technical issues as well. Our propsed "Arachnid" system was not stable enough to be considered as workable, it was just not perfectly solid as we had hoped. So now we have decided to use SVN once again and that is still being worked on (issues with speed mainly), the other hosts I had looked at did not offer near enough space for what we needed, so we will be using a private server located in Wisconsin belonging to a personal friend of mine.

One of the main issues we had encountered was texture maps. Sometimes when the map is pointed to a file that is not relative, it will not be found and thus not rendered. This became frustrating to the point that it was decided that all surfaces aside from the spider model will be Renderman shaders rather than a collection of images. This also supports our cause since Blender can do texture maps quite well on it's own but when it comes to displacements nothing beats Renderman. As there were to be quite a bit of it in the short it only made sense to showcase what Renderman can do quite well rather than just say "Hey it can render!" So a lot of work has been going into designing shaders that can take advantage of Mosaic's power, not just look good. Such as using the Blender material system to control the shader parameters so that different models can share a shader but each have it's own look and feel. The train above is such an example, the main body of the train itself is using one shader but the color and subtle pattern differences are controlled by the base Blender material. The only other shaders that do not share this are the wheel assemblies, but even those are also controlled in their own way by their base Blender material. In all the entire short maybe uses 12 custom Renderman shaders, including the displacement shaders, the rest are all Mosaic's power.

Blender 2.50 and the future of Mosaic

This is something that needs to be addressed as the timeline to the next version of Blender gets shorter. Mosaic as it is in it's current form, will not work with Blender 2.50. This is due to the use of Python 3 for the reworked Blender. However all is not lost since the Blender devs have started to work on the much requested Render API that we have been waiting for. This means Mosaic will need to be rewritten from scratch all over again, something Eric is not too excited to undertake since he spent the past year putting much effort and work into what it is now, though we do know that when the time comes it will need to be done. This is good news though since this will allow Blender users to render everything that can currently be done only in Blender (such as particles, animated curves and soft bodies). Currently Mosaic can output about %90 of what Blender can do natively, this is due to the limit of the data that Mosaic can access in Python. This of course is not a Mosaic only issue, ALL render exporters have this limit in Blender (with the exception of possibly Yafray). One of this sites goals was to prove to the Blender devs that having that external renderer support was a good thing, this will offer users a choice to use something they know rather than use just Blender's internal. Again we do not want to say Blender's internal render engine is bad, it is quite an amazing piece of coding and one of the best open source renderers out there. The issue mainly is choice rather than function and since most visual effects and animation studios use Renderman for the final frame rendering it would only make sense to have that option for Blender, thus making it more appealing to the high end market. This site itself has gotten the attention of many such studios and in the process some have even started to use Blender to Renderman for their own evaluation or even actual work.

So what does this mean for the future of this site? Well that is something we have a year to figure out. I do know that things will be changed, ideas are already being drawn out for the site itself though I do know this blog will be used in some form or other. I think our goal of public awareness has been achieved, that is obvious when Pixar, LucasArts, Blizzard, Dreamworks and more have stopped in on more than a few occasions. BlenderNation, BlenderAritst, CGTalk and even Blender.org have directed traffic here every single day. This site has gotten Animux some attention too, people who come here have gone to that Linux OS to check it out and in some cases are now working with them on various projects, including myself.

We have come a long way that is certain but we also have a long way to go.

Fisheye lens distortion using MOSAIC and Aqsis

Greetings again Blender heads and RenderMan junkies :)

Well I've been hard at work with MOSAIC and recently needed a good test case for the volume shader I'm currently working on. It just so happened at that moment someone over at BlenderArtists inquired about lens distortion so I decided to do a test project and developer blog demonstrating lens distortion but also testing several area of MOSAIC. I've been asked to copy that blog to share with all the good people here also....


From the original developer blog here http://sourceforge.net/apps/wordpress/ribmosaic/

I've recently had someone at BlenderArtists ask whether MOSAIC could do lens distortions. Well since that's not a standard feature of Blender I have no built-in solution for this, however knowing this could be done I figured I'd tell him that RenderMan could do it easy and MOSAIC can set it up. As I was writing the reply I realized that I hadn't actually tried this myself and I'm currently needing an good project to test the new volume shader so... thus was born this fisheye project :) This post is not intended as a tutorial but just an overview of how I achieved this effect, I'll also include the blend if anyone wants to play with it.

Well the first thing I did of course is look around for examples of techniques, and as it so often turns out there's several different ways to do this. If looking for just mild image distortion/displacement then the simplest solution is to just use an image shader to process the image (the same as the post process filter in Blender's compositor). If looking for something more extreme such as the 360 degree fisheye effect then more radical steps are required. One approach is to use a warped plane in front of the camera and use raytracing with the surface normal refraction to fake the lens effect. Another technique, which I prefer, is to render the camera as a cube map in separate passes and then combine the maps into an empty beauty pass with an image shader. This approach can see distortions all the way around the camera, has fine tuned control over image quality and camera rotations and lens distortion and can even be animated with the same maps as long as camera translation doesn't change!

I found several example shaders but was surprised to find one written by my friend Chris Foster in the Aqsis example folder! I only made a few small changes to the shader for creative control:

• I added a rather wide filter to the lens mask to blur its edge


• I added the ability to flatten the distortion so the lensing effect can be pulled in and out of


• I added the ability to rotate the forward vector in the cube lookup on x,y and z axises

The idea behind this technique is simple: use a cube map pass from the camera's position to generate cube faces, then in the beauty pass lookup into the cube faces with the image shader to project the warped perspective on the frame. The first step is to build a standard scene, I decided to use checker displacements on a ground plane and columns around the camera to emphasize the effect. I also decided to test faked soft shadows in a larger more complex space and also to include my partially rewritten volume shader to produce more daigonal light streaks for effect (the finished shader will be included in next CVS update). Next I added the fisheye image shader and created a shader fragment.

First off there's several tricks I had to play on the beauty pass:

• Enable an empty scene layer. This is so nothing is rendererd except the image shader otherwise you'll waste time rendering object not seen.


• Create and select a RIBset on the camera with the fisheye shader enabled. This is so the beauty uses the fisheye shader but we can force the default RIBset with no fisheye effect for other passes.

Next I created a User Autopass to use as the cube renders with a few filter options applied:

• Blank the "Layers Scene:" filter. This is so we can specify what layers to use in this pass, otherwise it will use the beauty's empty layer.


• Set "RIBset Bypass:" to DEFAULT. This makes sure the camera used from the beauty pass is not using the fisheye lens.

An autopass is necessary instead of just using a global env pass because better filtering can be achieved in Aqsis using textures then an env map. In this custom pass I made several custom scene RIBset's named as numbers from 0000 to 0005. Then I set this pass in the Project tab to use "RIBSETPASSES" so MOSAIC will export the numbered RIBset's as separate passes and ignore the DEFAULT RIBset. This is so I can use the same scene setup for each cube perspective. Then for each RIBset I used the "Show Autopass Settings" to setup the following:

• For each RIBset I use one of each of the cube Camera: Perspectives (Object: nx, Object: px, Object: ny, Object: py, Object: nz, Object: pz). I use the Object instead of World perspectives so the cube faces are relative to the cameras orientation.


• Setup each "file" display to point to ./Maps/ and use the name of each cube face, as "ny.tif", "py.tif" ect.


• Setup the "Texture" dialog to convert each file display tif into mipmapped tex, as "ny.tif" - "ny.tex"ect. This is not strictly necessary but produces much better results with the image shader ;)

So at this point what's happening is the user autopass is exporting each RIBset using the active camera from the beauty scene with one of each cube face directions and optimizing them into one of 6 images. Now all that has to do done is pull up the fisheye shader in the shader editor and put in each of the images from the cube pass and adjust "thetaMax" to the distortion angle and render the beauty pass.

However since the cube faces can be reused in a simple rotational animation with minimal render time I decided to take things further and do a 20 second animation. Also since the cube faces are static I decided to try really high quality occlusion maps, shadow map with faked soft shadows, DOF and volume atmosphere shading. This is because the addition time needed to calculate these passes on the first frame are more then made up for by the really fast render times of the animation in the imager pass (it only has to grab the cube faces and calculate lookup direction and lensing). I also thought it would be interesting to synchronize settings across the pipeline from Blender to MOSAIC by adding animated composting effects and by using the same camera controls to drive multiple shader parameters in MOSAIC. In particular I'm animating the "lens" control on the camera and hooking that to the thetaMax shader control but also grabbing the same lens data and modifying it in a python token to control the lens distortion parameter and finally using the frame count in another python token to feed y axis rotation in the cube lookup. As a finishing touch I've animated a spectral lens distortion effect in Blender's compositor, this could fairly easily be done in the image shader but this gave me a chance to try sychronizing animation in Blender's compositor with RenderMan :) Anyway Here's the video, project file and a few frames of the animation from my gallery...

Here's the youTube video...


Here's a direct download of the mp4...
http://www.dreamscapearts.com/Public/fisheye.mp4

Here's a frame at 100 degree lens at 0 degrees rotation...


Here's a frame at 200 degree lens at 90 degrees rotation...


Here's a frame at 360 degree lens at 180 degrees rotation...


Here's a frame at 360 degree lens at 180 degrees rotation...


And if anybody want's to play with the blend here it is too.
NOTE: I embedded a modified version of MOSAIC that includes the Object:py-nz camera perspecitves that is not in CVS yet so you'll need to run MOSAIC from the text editor!!
http://www.dreamscapearts.com/Public/fisheye.blend

That's it, thanks for reading :)
Eric Back (WHiTeRaBBiT)

SIGGRAPH 2009

I decided that with 2 weeks until this years SIGGRAPH convention that I should make a post about some of the behind the scenes talks between myself and the Animux devs. Since a lot of Blender to RenderMan integration has been done on the Animux distro, Mark Puttnam (the founder of Animux) has asked for some screencasts and imagery so that he can show this during his presentation.

So I decided that instead of just showing off normal screenshots of the default Blender startup screen, why not show something with some punch to it? Such as the one below, a simple test render done for Project Widow in early June.




So the idea is to setup and render at least static frames of Project Widow, if not maybe 10 seconds of animation of the spider, the tunnel system and the train. This not only would be the first official test of the pipeline, it will also serve as a small technical demo for the SIGGRAPH presentation. Most likely this preview will not end up in the final production short, much like Pixar did for the first Finding Nemo trailer. What this will do is not only draw attention to Project Widow itself, it will also draw attention to the whole Blender to RenderMan idea. SIGGRAPH has traditionally been the place to present "proof of concept" ideas and papers, showcasing new technologies and amazing artwork. So it only makes sense that after years of pain staking work trying to get where we are now that we at least begin to present our efforts at the biggest CG event in the industry.

The downside is that I am not able to attend this, I cannot afford it. I am not upset though since what we are showcasing is more of how Blender to RenderMan works with one OS - Animux, rather than how it can work for everyone. It just so happens that Animux is delivering us to SIGGRAPH. We could not ask for more publicity than SIGGRAPH anyway. If anyone does manage to get to New Orleans this year, please stop by the Animux Birds of a Feather meeting, take pictures too!

Monday, 3 August

Animux: Free Software for Animators
Animux is an absolutely FREE animation toolset that is used to handle all the tasks of pre-production, production, and post-production stages of a high-quality animation project.

Monday, 11 am - 1 pm
Ernest N. Morial Convention Center
Room 264
Mark Puttnam