Variable Rate Shading in VRED: Quality Where Needed

RICCARDO PIERUCCI: Hello, and welcome to my presentation. Today, I'll be talking about Variable Rate Shading in VRED: Quality where needed. Here you can also see the date of the day I'm going to present this. And I'm Riccardo Pierucci, Technical Account Specialist at Autodesk.

So thanks for joining to this recording here today. I will then begin with the presentation. First of all, there's the Safe Harbor Statement. Summing up, I just wanted to tell you, please don't make any purchasing decisions based on this presentation. And that's the essence of it.

And for the rest, you can have a look and read through it for yourself. And then let's begin. That's the agenda of the presentation today. I will give you a brief introduction about myself, and also about VRED. I will tell you what variable rate shading is, in general, which advantages it gives you, and how VRS works, technically.

So always, if I say VRS, that's the abbreviation, right, for variable rate shading-- the main topic that we're talking about today. Then I will talk about the hardware requirements for VRS, how to set-up VRS in VRED, and at the end, I will do a quick wrap-up about what we talked about. And yeah, those are the subjects. Let's begin.

Let's go first with the introduction. So that's me. As I was saying, I'm working at Autodesk as a Technical Account Specialist, and I'm living in the Alps and working for the Munich office. There you can also see my email address. If you would have any questions, anything afterwards, feel free to contact me.

I was born in Italy, in Florence, and I'm half German and half Italian. And I currently live, as I was saying, in the Alps. I studied 3D art and animation in Munich at Quantum, which was a part of the SAE Institute School of Audio Engineering. I specialized in game design.

So basically, that was a private study that I did, which was based, generally, to prepare us for several different areas. We could work in the automotive industry, after that in the gaming industry. Even though it was specialized in gaming, it was preparing us if we wanted to become concept artists, or go into publicity or any other sector.

It was broad, and you could basically choose in which direction you wanted to go and then pursue that. So we had specialists there-- good hardware and specialists giving us good advices working in the industry. And I've been working as a 3D artist in the video game sector in Berlin, but mainly in the automotive sector in Munich and Stuttgart in Germany for the last nine years.

And the most time I was a visualizer working with VRED, and also a Poly Modeler with Maya. So I did, also, a bit of rendering with Maya, but that's not my focus. I was mainly modeling and working within the design departments, and with designers from the OEMs and working with VRED, as I was saying, visualizing, doing renderings, creating real-time scenes, and configurators, and doing data prep and such things.

I'm married with my Chinese wife for six years now, who comes from the icy north north of China. Say icy north because it gets up to -30 degrees Celsius. So really, really cold, and it also stays that way for a few months sometimes. And I was quite impressed by that.

And my interests are cars, video games, I do also video game streaming on YouTube, Twitch, and I like to watch different TV shows, and to do all kinds of different sports like inline skating, running, swimming, surfing. I like water sports in general, and yeah, I like to try out new sports, just having some variation.

OK, so that was it about me. Let's go on with the introduction to VRED. VRED is a great tool for real-time and Raytracing visualization. It's a professional software, which is used to render pictures, render out videos, and it's often also used for real-time presentation and configurator.

So what I saw, for example, it's often used within for internal reviews at the OEMs, and car OEMs, and design reviews where the artists, for example, present the current status of the CAD data to their managers. And this is, for example, one thing that it's often used for.

But in general, it is quite intuitive to use, but also powerful as it has a really realistic render engine with Raytracing, and you can achieve great realistic results with it. So you're having those two modes. You can work in OpenGL real-time, or you can work in Raytracing.

OpenGL is more for quicker results, and you're you're having the real-time, the Raytracing, which is more for realistic results and also-- which needs more calculation power. There are many ways in VRED to improve your performance and quality.

So to gain performace, for example, you can use LODs, the level of detail. Like, you can see here on the right an example on moving an object to which you're closer. And when you're closer to an object, it will blend in the version of it, which is higher tessellated. As you can see, it has way more triangles in there.

And on bottom, then if you go further away, it blends in a version of the object that is having less triangulation, less detail, but is still really good enough, right? So in most cases, you're getting performance through that, because at that point, if you're further away VRED needs to calculate less polygons.

And this is one of the ways you can gain performance. There's also DLSS from NVIDIA-- deep learning super sampling-- which is basically taking an image which is of lower resolution, and then upscaling it to a higher resolution using AI. And that's a technique often used in games, for example, which is always being developed further, and getting better and better.

You're getting better quality output out of it, and you're getting better speed out of it. So I'm talking about a huge performance advantage here. And there's, for example, backface culling where you can tell-- if you turn that on in VRED, it will then not calculate the faces which are on the back and not visible, therefore, you can also save performance and there are many more functions like that within VRED.

To achieve a better quality, you can also create Light Maps. So shadow and Light Maps, so that's also known as baking lights or shadows. That's a thing that VRED can do. And compared to other software, it can also bake in the lights. So it's taking that Raytracing image, the realistic calculation of the light with its bounces and so on, and it basically pre-renders those fixed system into the object, so that you can also use those for real-time in OpenGL or Vulkan.

And yeah, you can also use Raytracing with full global illumination-- that would be the highest quality. There are different quality settings you can set-up, and with that you would then get the best quality possible. You would also see realistic reflections of several things like, for example, if you're having the mirror, which is reflecting into the body of the car, you could see such things which you cannot see within real-time.

And what we're going to talk about today is variable rate shading, and it allows you to do both-- to increase the quality, and also to increase the performance-- and that's what we'll be talking about now.

So what is variable rate shading? It's a technique which allows you to define different render quality on a material base. That's how it works in VRED. And so it allows you to define how many samples per pixels should be rendered.

On the right here, for example, you see an example where you're having four VRS samples in the middle of the horn, and on the outer part of the steering wheel we're having one VRS sample per 16 pixels. So a way worse quality. And as you can see-- I hope you can see it also on video-- having a really great quality here in the middle, and here it basically blurs out and gets worn out a bit, right?

And you have this flexibility to decide where you need quality, and where you need more performance maybe. And it is an OpenGL specific rendering feature, which is not available in Raytracing. And it's included in VRED 2021 up to the newest version, which would be now 2024.1, which was just released shortly.

And yeah, those are the main things to know about. So which advantages does VRS give you? You can vary quality and amount of processing power required for different parts of an image.

As I was saying, let's say we're seeing here the example on the right. You would now want the rim to be in focus, and like the profile here of the tire. And then you would set-up a higher quality here to the center. And like you can see, hopefully, on the top, a lower quality to the body because you don't want to focus on it, right?

And that way, you could also not only decide where the quality goes, and where the quality is less, but you could also decide at the same time, right, where to direct the PC to focus on the things that you want for processing power, and also leave out the things which are not important at the same time.

It can also increase rendering performance if you set-up the quality to a lower quality overall. Also, over the image, if you're having a few focus points, it can be focused on those ones, and at the end, overall, have a better performance depending how many of those better quality materials you're having in the scene, of course.

And it can reduce rendering quality with little or no reduction in presentable output quality. So there would be, basically, a performance improvement free of charge, right, as I was explaining. As you can set that up, you're quite flexible and can get a lot of gains through it. Yeah. And you can also get rid of more effects, or other so-called, I call them impurities, sorry if that's not the right word for it-- increasing its quality.

So I will show this to you later on a picture and also in life, but also there is, for example, a bug where you are seeing, basically, the wireframe on top of materials with a high contrast, and therefore, you can set it up. Let's continue, then, with the next.

So let's have a look at a few shading examples here. We already saw them on the rim and on the steering wheel. And there's one sample per pixel here on the left, as you can see. And then there's one sample per four pixel already getting worse quality, and here one sample per 16 pixels. And those are good examples where you really see on detailed textures, this makes a big impact.

You probably would use a normal quality of one sample per pixel on those, or higher, because as you can see, it's really visible and you should consider to use higher quality for those displays and so on. Of course, if they would be just a thing not to be considered in the background, you could still consider to put it on the lowest quality. But in the most cases, I would advise against it.

And if we look here onto the stitching, you can see here that at this size you can almost not see a big impact in quality. But you can see it on the leather, right? So here you're having more detail, here it's getting less detail. And here, it's basically washed out on the one sample per 16 pixels. You cannot see any details at all anymore.

So one thing I wanted to say is thank you to NVIDIA for allowing me to use their pictures. Those are all referenced here, as you can see, with picture provided by NVIDIA. You can also click onto those and see where they're coming from, where their source is. Just wanted to mention that.

So let's continue with how does VRS work technically. So we're having the regular shading on the left, and you're having a raster rate of 1 pixel by 1 pixel, and a shading rate with 1 pixel per 1 pixel. And that's how it would look normally, right, with if you wouldn't have any VRS turned on.

So now what you can do here-- you can decouple the restoration frequency from the shading frequency, which you can see here on this picture on the right. So basically, you can see where before you were having 1 pixel by 1 pixel, you're now having 4 pixels rendered out-- so basically, 4 pixels rendered out together, which would lead to lowering the quality and basically, united here on the right.

And that would be an example of having lower shading rate, and basically joining them together. And this is an example of how it works. Here another example, how it's working in games. You can see, we're having different rasterizing qualities here on the left, where blue is the highest and it goes down here to 4x4, where the sampling errors get bigger and bigger.

And in a game, to explain this technique a bit better, you would then have here, the focus points-- like the horizon where you would look to the most times with a higher quality. And the car, obviously, which is also one of the focus points of the scene. Then you would slowly go lower in quality in the areas around the car.

And then on the parts which are on the sides, or in the back where you're mostly not looking, and whereas, also, a lot of motion blur, right, you wouldn't see the detail because it's moving quickly and you would just see this blur effect. It doesn't need to be that high of a shading rate, and therefore, you can set it up in this way in games, which is quite useful. Going from high quality to a lower quality depending on the surroundings.

Here you can also see other examples on how it can be used. It can be a flexible area like here, adapting to the situation. So it mustn't be like we saw on the slide here before, it mustn't be fixed areas of the screen. The area itself can also be adapted to the situation to a scene.

Let's say, for example, the camera is moving faster than you could have a bigger area adapted, and then if you would move slower, you could see everything more clear. It could make it-- sorry, the other way around. It could be a smaller area, and then it could be a bigger area for when you're moving slowly, right, where you have the time to see every detail.

Here, you see just a normal fixed area where we're having here in this example eight times shading rate, really high, and the normal shading rate on the border. And here on the right we're having an example of the Foveated rendering in action.

So there's, for example, the PlayStation VT, right, or there are also other head mounted displays which are using this. So you would then have a camera inside of your VR goggles, which would follow the eye. And that way you could really save a lot of performance, right? You can think about it, and the goggles, the computer, only needs to calculate the area where you're looking to, and everything else would be, then, the more it's not in your focus of the eyes, it would then get worse.

And as you can see, if we compare it to any other way it's being used, the area would be way, way smaller and you could save a lot of performance here. So just be aware of it. And those are other examples on how it's being used in other areas. And here, you can see another one of Foveated rendering where it would also adapt to your eye, right?

In the middle, the highest quality. Here, middle quality, and the lowest at the surrounding. And here in VRED, it's based on each material. So you would have a lot of different materials all over the scene, and you could set up any material you like with the quality you need. So really specific on each one of those materials, you're able to set it up.

And that way you're most flexible, right? That's an advantage which VRED has. OK, so then let me take a sip of water, and let's move on, then, to hardware requirements for VRS. OK, so in general, at least in NVIDIA, VR Ready Turing based graphics card or newer is required to use.

So if you're having a not supported card, you won't be using to an older card. You won't be able to use VRS, and also, AMD cards are currently not supported in VRED for VRS. And in that case, if you have any of those older graphics cards or AMD cards, you will then see that the feature will be grayed out and you won't be able to use it.

But if you're uncertain, you can go and check it out. I will show you later where to find the VRS option, and that way you will know for sure if it's supported or not. And for general viewer system requirements, you can follow those links. So we're having here the system requirements for VRED 2023 and for VRED 2024, which is the newest.

I will click on those to quickly show you what I'm talking about. So if you click on those, you'll get on this link and you will see the system requirements. And I just wanted to make you aware of it that those are really the minimum requirements you're seeing here.

If your company can afford to get better hardware it's, in my opinion, often better to pay a bit more at the beginning to get better hardware, and then save this later in rendering time and work time because everything works faster, right? Just always consider this.

It is not always the best way to think about saving hardware, but you always need to think that the work time of the employees is often higher, right, than buying hardware. And therefore, I think, in my opinion, it would be the wrong place to save. But just as an advice, think about it.

So what you can see here in the hardware, we're having a minimum of a 2 GHz, 64-bit Intel or AMD multicore processor. And recommended is an Intel Core Xeon processor or an AMD Ryzen, or Ryzen Threadripper. So those are just the product series that are recommended, as you can see.

Then there's memory-- 8gb of RAM, or 60gb of RAM or more are recommended. I can only tell you that in the automotive industry they're working mostly with 64 or 128gb of RAM because the scenes, if they get complex, they can be really big and have a complex structure. And therefore, the more, the better.

And now that, especially, the prices for memory are cheap, for RAM memory, I would advise to get more. If you need to, buy it now. And yeah, you can also see here the drivers which are required at least here. RTX 516.1 for NVIDIA and the Linux drivers, as you can see here. And there's also a link to the NVIDIA website here.

And what I wanted to show you, especially there's a certified hardware here. Just to finish this up, you can use Windows is supported, as you can see. Windows 11, Windows 10, and certain versions versions of Linux. And also there's optional hardware, like a few HMDs which are recommended here below, right?

If now we go under Certified Hardware, you will see here fine-tested hardware, and then go under VRED, because that's what we're looking for, right? And then you could select here the version you're searching for from 2022 to 2024. You could select the Windows version you're using. And then you could also filter by AMD or NVIDIA graphics cards.

And yeah, here are all the graphics cards which are tested listed. So it's a good point to see if you, for example, want to buy a new graphics card and you want to be really sure that it's working well, and tested with VRED, you can look that up here. And you would also see the driver version on which it was tested.

So if you would experience any issues with a new driver version, you could always roll back to that version on which it was tested. Just be aware that on AMD cards there are several functions not supported, which are implemented in VRED. So for example, VRS, as I was saying, wouldn't be supported. Also, DLSS, this upscaling technique which can be quite handy.

And there are more features, which are NVIDIA specific and only on NVIDIA cards. And therefore, won't be available on AMD. Just consider that. Besides that, I'm not allowed to make any proper recommendations, but so you know at least.

OK, let's move back to the presentation. OK, now we looked into that, and let's move over to the next slide. There is the recommended system components by NVIDIA. I just wanted to bring that in. If you're searching for recommended explicit graphics cards, or CPUs or HMDS, those are the ones which are being recommended by NVIDIA for VRS, or also for VR use.

Just be aware that those are for VRS in general, right, and not for VRED for VRED. Please refer to the official system requirements if you want to know for VRS in general, what is a good hardware. And it might be, in the most cases, I think, it it might be fitting also for VRED, but no guarantee on that-- you can also look into here onto explicit hardware.

All right, just be aware that they're not matching exactly, right? OK, so now let's move to how to set-up VRS in VRED. So to be able to use variable rate shading, you need to turn it on first-- like you can see here on the right. Please be aware that those screenshots are taken with the newest version, with the VRED 2024.1.

It might look a bit different, the color, especially of the UI, differs from version to version. And you would then here go under Visualization, Realtime Antialiasing, and then go under Variable Rate Shading. And there, you can see you're having two modes. I mean, basically the Off toggle, or you can turn it on with the Modes screen and materials, or the Mode Materials.

And if you now would have an older graphics card which is not supported, or an AMD graphics card in your system, you would have this function completely grayed out, and you wouldn't even get this additional menu opening up. And that way you would know, OK, it's not supported by my hardware. OK, that's the point, where to find it.

And to go into the screen in Materials Mode, here the global quality for the image is defined by the Realtime Antialiasing settings that you're seeing here on the right. Those are the ones which are having the most impact to that mode. And for materials, the global quality for the image is native 1 by 1 shading in Standard, which means one sample per pixel.

And be aware that the Material settings in the Material Editor will always override any global setting which was set-up. So if we look into that, you can see here, you first, as I was saying, need to activate the VRS Mode. And if you're having this screen in Materials Mode, you can just set it up here, right, and it will then turn it on all over the screen.

And if you want to have it in the Material Mode, you can set it up by material. And not only-- I mean, only over all the screen, and you would then go into the Material Editor, and here under select Material, first of all, of course. Then go under Common at the real button here. And here you could then set up the sampling, then the shading rate.

As I was saying before, here you have the choice between the samples. So one would be having the same settings as Off-- you wouldn't notice any difference. There would be 1 samples per 4 pixel and 1 sample per 16 pixel, which is the lowest quality. And if you would need higher qualities, you could then go up to 2 samples, 4 samples, 8 samples on the bottom.

And yeah, those higher sampling rates can be used to get rid of the more effect, as I was saying. And also, this more effect is happening mainly with carbon woven cloth, for example, or textures with visualized in VR, where the resolution might also be lower. And mainly on materials where you have a certain pattern which is getting repeated all over the bigger area, and then this weird more effect is happening, right? And then you would need this higher sampling rate to get rid of it.

And these modes all require Realtime Antialiasing to be enabled with at least the same multi-sampling rate, which is chosen here. So the lower shading rates, as we were discussing before, are just for better performance, but contain lower quality. For example, for things that are in focus of the scene.

And yeah, as I was saying, one sample is the same as off. And now what I mean by this sentence here, they require Realtime Antialiasing to be enabled with at least the same multi-sampling sample amount as chosen. So you would then have-- sorry, it's the variable rate shading, I need to correct that-- variable rate shading sample amount.

If we go in the next slide here, you will see that you also have Settings. If you go on Edit and then Preferences, and then go on to Render Options here, if you click on Visualization Advanced here on the right, you will see that you're having those presets, basically, for we go back for those settings here.

For the Realtime Antialiasing levels here, you can set them up each, one by one. And if you now would like to have, for example, in the high quality settings, so those eight samples. To be able to see those eight samples set-up, you would have to also turn the Realtime Antialiasing to be enabled, right, at least that level.

So you would need to have the high set up here, where we're having eight samples for being able to use eight samples in the material in VRED. And that's, basically, how it's working. And that's all you can set-up here. Those settings would be the variable rate shading that you can also change to your needs, right?

And they go up to a maximum of eight samples, at least with my graphics card. That's what I found out. It could be that later graphics cards will have also higher settings. I don't know exactly. OK, and to show you the carbon material example I was talking about this is the famous moire effect I was talking about.

And that's how it looks like when it's turned off, right? You see these ugly, kind of, circles. And if you would now put the material onto eight samples and Realtime Antialiasing turned on high, then you would get this clean result. And this, I will also demonstrate to you inside of VRED here.

Let me go into the scene that I have open. So in this case, I will just use the M-hotkey and press the Left Mouse button to take this carbon material that we're having here, to apply it here to the hood, to have a bigger surface, right? So at the moment, let me go back. I'm having Realtime Antialiasing on Medium. Let me turn that off, so that you see here this moire effect in its full beauty, right?

You see those circles wander around? And now if I would have the mode set to-- let me go in Realtime Antialiasing and Variable Rate Shading, Screen and Materials, and now we'd go into Realtime Antialiasing and on High. As you can see, the effect vanishes because the quality is really high. And that way it would be working right away. If you are, as I was saying, here in the Mode screen and Materials.

Now, what happens if I switch to Materials? It goes back here because in this mode it's only controlled by the materials, and it gives you a bit more control. And you can then choose in which surroundings to have which quality better and easier. And in this case, I would then now click on that with the Shift and Left Mouse button onto this hood, so it selects the material here.

Let me go on Modify Asset to be able to edit it, and in Common here, I would be able to set it up to eight samples. And as you can see, now it would be vanishing, and that's how it needs to be set-up in Material Mode. And that's, basically, the difference between the two modes.

The one is, overall, controlled with this Antialiasing mode, and the other one is controlled by those settings within the Material Editor, right? And it's not over all the scene. Now, let me go back here because-- let me set it up because it's really hard and it costs more performance. Be aware of that.

Let me go under Realtime Antialiasing Medium just for presentation purposes, and onto Off here. OK, let me Control-Z to get back. And to give you another example here, what I will quickly do is take one of the lights that I'm having in the scene-- or quickly create one.

Let's take at this slide. OK, it will be then created. It's here. OK, let me move it on top. Let me enlighten, turn it around, first of all. Turn off the Cast Shadow on Shadow Material. That looks better for us now. OK, let me move it.

Let me see, what do we-- yeah, there's the-- OK. Let me move it in front of the steering wheel, and then I will present this to you, in life what we've seen before with different qualities so that you also know how to use it, basically. Let me jump there. Let's go here closer. Let's make this circle a bit bigger. Oh.

All right, there we are. OK, let me set the intensity up to, let's say, I don't know, 20, which is quite bright. Now, let me disable the visualization of it, Primary Visibility, OK. And then turn off the visibility here. And what I would do now, for example, in this case, if I would have this view, right-- let me say I will quickly hide the headrest so we can see that better.

Now, I would go here. And if I would say, OK, I want to have again here, the higher quality with one-- with the setting of VRS and turn to the standard here-- it already looks good. Let me go into the Visualization, and then again to turn on to Materials Mode. And now, I would basically take this material here, I would Control-D it to duplicate it.

Oh, no, wait. Now, it copied the-- wait a second. Let me take it right here, not in the switch, it's better. Control-D it and call it Low Quality. I would then have this one with the high quality. OK, then I would set up this one to, let's say, yeah, I can leave it off that because it is the same results.

As you can see, it looks good, and you can see all the details. And then I would take the low quality and assign that only to this part. And here, I would then go to 1 sample per 16 pixels. OK, and as you can see here, you're getting the same result that I was presenting before. If I go around a bit, you can see it better.

Here you're having all the details on the focus point, where you want to-- let's say you want to present a new horn, the most modern horn, and on the outside you don't want to focus on. You don't want to present it to your superiors in this case, for example. And then you would have this lower quality here, and that's how it looks like.

OK, this is one example on how you could use it. And there can be many others depending on the case, in which you're in the user case and what you want to achieve, right? OK, now, let me show you a last example. Let me close this and move on to another scene.

So be aware, there's currently a bug, which is known, which is still present here in the newest version of VRED 2024.1. And in this case, as you can see, even though I'm having here the visualization set up on Realtime Antialiasing High, that's especially when it's happening.

If you're having this setting on High wherever, there is a bug in VRED where on high contrast materials, like here, this brushed metal, where you can see those brushes and a really high reflectivity, you would then start to see the wireframe, right, that you can see here.

If I turn it off, you can see those-- I hope you can see this also on the recording-- those lines. And there's also the possibility to then turn on, here to go into Variable Rate Shading, set it on Screen and Materials, right, because it's taking the 8 samples right away. And as you can see, it's perfectly, perfectly vanished and it works.

It's a good workaround. For as long as it won't be fixed, it's a good and quick solution that you can use if you have a hardware that is supporting VRS. And maybe one last thing, which I didn't mention so far. If you go into Realtime Antialiasing, if I turn it off just to explain to you, maybe, for the ones who don't know what aliasing is.

Aliasing is this effect that you're getting if the image is not smoothed out on the borders, right? You can see those kind of stairs here, or those pixelated parts you can see here on the sides. And if you turn on Antialiasing, it will give you a better result and smooth those out, and you will get a more homogeneous picture overall.

Yeah, so that was the main presentation. Let me, maybe, do a quick resumption of all, what we went through. Let me jump back to the beginning here. OK.

So let me talk about what we talked about, what variable rate shading is. Just remember, it's an OpenGL specific rendering feature. We talked about that it's a quite flexible technique you can use, as you want, to use the performance you need, where you need it, and to save performance and quality where you don't need it.

We saw how big the impact was. We understood-- I explained to you how it works technically, how it's being used in other software, or in general overall in many other software, how it's being used on a material base in VRED. We talked about the system requirements, the official ones, and the one that NVIDIA is recommending.

And I showed you how to set it up, and how to understand which different modes there are, and how it can be set-up here in Visualization here, in the Variable Rate Shading settings. And if you don't see it, it won't be working with your hardware.

Which advantages it can also have to set them up on the quality settings on a higher and onto a lower basis. You can set up the Antialiasing settings, and how it can be used, actually, within VRED. And how it can help you to get rid of those moire effects, and to get a better quality overall.

Yes, and that was it for today. Thank you very much for attending to this session. I'm happy that you listened to this recording. And if you have any comment, any question, feel free to reach out to me.

I hope you learned something today about this option, this technique, you can use within VRED. And I hope you can make good use of it for your future. And yeah, thanks again.