Build the Matrix City of Machines Using USD Across the Maya Tool Sets

SEBASTIAN ORTEGA: Hello, and welcome to this Autodesk University session. Today we'll show how to use USD inside of Maya in order to create and render complex scenes using the Maya tool set, so Maya USD Bifrost USD, and Arnold. I'm Sebastian, I work in the Arnold team at Autodesk. And on my side, I will start by doing first a technical overview of how all these different plug-ins work inside of Maya, how they interact together. And after that, you will see practical examples, more detailed use cases, and about how to get the images that you want.

So most of what we will show today is done with software that was already released, but we will also be showing some tools that haven't been released in Maya yet, so for that, please read this Safe Harbor Statement. Be aware that all this is work in progress, and do not make any purchasing decisions based on it.

OK. Most of you must already know that USD is the scene description format meant for content creation, and that has been developed by Pixar. It is meant to handle complex production scenes. In collaborative workflows, it's designed to be extensible. And so how do we support USD in Arnold, it's all exposed in the project called Arnold-USD, which is open sourced on GitHub at this address that you can see. Everything is public, so the source code the tickets, the triage, the roadmaps, so I welcome everyone to take a look, post the features requests, report bugs and so on. And if you look at it, you will see that USD has been a very, very active area of development in the Arnold team in the last few years.

So in the list of things that Arnold USD can do, an important one that we'll cover here is that it makes USD a file format for Arnold which means that everything that can be represented in Arnold, or everything that can be rendered in Arnold, can also be represented in the USD description. So for that, we've needed to do some extensions to USD to represent all the Arnold-specific nodes and that happens to what USD calls schemas. So we declare all the Arnold schemas in USD, all the Arnold shaders, and that allows us to have exactly what we need for USD.

We can then load the USD files and, for example, render them in Arnold. And we can also save them, and so we can offer USD files with exactly all the Arnold data that we want and then we can even render them in standalone. So the standalone command line for Arnold is called kick, and so kicking the USD file will really give you the final rendered image.

OK. So another open source project is Maya USD. We also welcome you to participate to this project. And what Maya USD allows you to do, for example, is to take a Maya scene with Maya native objects and export this as USD. So you will have a USD file with all the Maya nodes and you can then either reuse it in Maya or use it in other 3D applications that support USD. And what is interesting is that the Maya USD export framework is extensible, so we can customize it to add what we need for it.

Then, this USD file, you can either import it back to Maya and, which will recreate all the objects inside of Maya, but this is not what we want to show here. We want to stay as much as possible in the USD framework and so we're going to use something called the USD Proxy Shape, which is a single node in Maya, but inside of it, you will have the whole contents of the USD scene. So you can go in the outliner, you will see everything from the USD scene, you can select them, see them being highlighted in the viewport, see the attributes in the Attribute Editor, edit what you want. You can also have the Arnold manipulators to move and hide objects, et cetera.

And so it really feels like all these are Maya objects, but it's actually just one single node inside of Maya which makes it more lightweight. You can create USD primitives at any point of the hierarchy through the outliner, and that also works with custom schemas, and you have this window called the USD Layer Editor that you can see at the right. That shows all the layers in the current scene, so the USD scene is typically made of lots of different files, which is called the layer in the USD lingo.

And so five, four different assets are for different stages of the pipeline, one for modeling, one for Lookdev one for lighting. And this window really shows you all the files and allows you to, for example, disable, enable some of them so that you can change what you want to see. And also, when you do your edits, you can choose in which of these layers the edits are going to go. So here, for example, you select one specific layer. You move objects or hide them, and then if you want to save back your USD scenes, all these edits are going to go in this specific file.

So how does Maya USD work with Arnold? Well, we say that Maya USD can export USD files and that Arnold USD also can export USD files. So now, in Maya USD, you have this option called Plug-in Configuration that is meant to customize the export, and when you set it to Arnold, you will have all the Arnold data that you need on top of what Maya USD has exported. So you have the Arnold nodes like aiSkydome Light, aiStandin, et cetera. You have all the Arnold shaders that are going to be saved in the Arnold-rendered context, and you will also have Arnold attributes on top of built-in nodes. Because, in Maya, when you have meshes or lights, et cetera, you always have an Arnold section with specific attributes for Arnold which are really important in production scenes.

And finally, also, you have the Arnold render settings, which are needed if you want to render your scene in standalone using kick. We also expose the Arnold schemas so that Maya USD can find them and so that you can create any Arnold primitive at any point of the stage. And the most important one, probably, is that if you load a USD scene in the Proxy Shape and click Render, it's going to render your scene in Arnold as it is in Maya USD as it shows in the viewport. And that happens because the USD scene is shared in memory between Maya USD and Arnold, so whatever you do in Maya USD, if you hide objects, move stuff, et cetera, it will be reflected in the Arnold rendering because it's really the same data that is seen by both. So that's really important to have a good fidelity of the render, and also it's efficient this way because the scene is all in memory.

So far, we have a workflow where you can have traditional scenes in Maya with the modeling, with the Lookdev done in Maya Hypershade. You can save out your assets as USD. You can assemble them and create your shots, your layouts, do overrides, do your lighting. So you can already do a lot of different things, but in production scene, one thing that is really interesting in particular these days is to create data procedurally, and that's where Bifrost comes into play.

So you probably know that Bifrost is a tool for visual programming. Yeah, it's a tool for visual programming to create simulation, fluids, instancing, everything in a procedural way. And so you have a Bifrost graph, you have all your nodes, and you can create very complex things with that. It's all integrated in Maya, it renders in Arnold, and it's extensible through what we call compounds.

So you can create a compound that is going to include a more complex Bifrost graph inside. And using these compounds, and a bit more, Bifrost was extended to become Bifrost USD. So this time, it's the same graph but the nodes are actually representing USD data and they kind of expose the USD API. So you can use two nodes to create USD nodes geometries or you can load the USD file in to edit, assemble your scenes, and you can do all this procedurally, and all this is integrated with Maya USD. So it will create a USD Proxy Shape and you will see in the outliner the exact hierarchy of what you have created, because once again, the memory is shared between Bifrost USD and Maya USD. And as you can guess, if you render it, it's going to render the same way in Arnold because it's, once again, the same USD data that is shared in memory across the three different tools.

And yeah, so now you really have a framework where you can either create your scenes in Maya in a traditional way or create them procedurally. You can do instances, you have a lot of very interesting tools to assemble your scene, as you will see later. And for Look-dev, currently, the best way is to create your shading trees in Maya Hypershade and then export them to USD.

But then you can edit the shader attributes, as I've shown before, or if you really want to do big changes, you would have to re-import the shaders back to Maya and re-modify them in Hypershade, which is not ideal, which is why we have been working on a new tool to be shaped in Maya in the future that hasn't been released yet, which is called Lookdev X, where you can really load shading trees in USD, edit them, save them out, et cetera, as a complement to everything else that I've shown.

You can create Arnold shaders, Arnold-specific shaders, but you can also create material X shaders that will render in Arnold but that could also render in other renderers that supports the material X. And that's all for the theoretical part, now I will leave it to John Paul who's going to show all this in more practical workflows.

JEAN PAUL GIANCARLO: Well thank you, Sebastian, for that great technical overview. My name is Jean Paul Giancarlo. I'm going to be presenting how I built a matrix city of machines, as a use case, and that was built procedurally using all the USD tools available inside of Maya.

This example, again, we're going to be showing how to create procedurally a city. We're going to also see how to create modular USD prototypes and get some composition arc like variants, also in a non-destructive workflow. And it's also using the opinion-based layer system, so this is really one of the advantages of USD. It's just a non-destructive way of work.

OK, so just the agenda. As you may see, we have the technical overview, and now we're going to see a little bit of how you can create, edit, load and work with USD inside of Maya, so we're going to be assembling layout, show some assembling layout workflows. I'm going to show you very quickly how that Maya USD plug-in works, and then finally, we're going to procedurally build that city using Bifrost.

So in order to create a USD model, Maya USD, the import/export workflows, that really let artists import USD data as intrinsic metadata or export intrinsic metadata as USD data, so artists really can use Maya's tools and workflows to create and export models for the USD pipeline so they can then re-import it to make further edits if necessary.

So with this assembly layout workflow, I'm going to break it into three tasks. First, we're going to be creating a model, which is pretty much stats, data translation, so the import of data. And we're going to take the USD data on disk and make it Maya data in memory, and vice-versa. So from there, we were just going to generate use the asset. And really, it's just going to be a bunch of assets created in USD which then you can pull from. To define different variants, we create a set. Then, when we create the set procedurally, using that USD data that we created on disk. And those are the three tasks that we need to do in order to create a layout workflow, and this one we're going to be seeing today.

So first, I'm just going to click Play on this video that I prerecorded where we're going to be seeing the export, import and work, how to work with USD inside of Maya.

[VIDEO PLAYBACK]

- In order to export an object that's USD, all you need to do is select and go to File, Export Selection. Now you see you have File Type, USD. I assume I see there's a few options, to keep, include options. You can also set the plugin configuration to Arnold and you have the ability to export as ASCII or binary. You just give it a name of the [INAUDIBLE] scope. They have different motions like subdivision methods to keep geometry information; Color sets, UV sets, skeletons. skin clusters, length shapes, display colors. You can also say if this USD Preview surface or material x preview surface. Set to save animation data and all this stuff. I'm going to export selection, replace that one, and done.

There are different ways you can bring in a USD file from Maya. You can either go to Create, Universal Scene Description, Stage from File. Let's go find out. So you see, you have a prim path, low pay loads, et cetera, and I'm going to go find that specific file. So, let's save it here, USD, and that gives us this particular object, as you might see, with the actual name as a layer. So that works pretty quickly, as you might see.

Another way you can bring in information, you can delete this now, is via Bifrost graph. So for that, you can create a new graph. I'm going to delete that input because I don't need it, and I'm going to create an add to stage node. That add to stage node gives us the ability to basically bring the USD file in.

So you see, you have some inputs and some information on this side here that you can use. For this time, I'm going to pretty much just bring up prim definition. When I create a node, you will see, by right-clicking on that, it gives me this particular dev or X form. If I go here, you will see in here, it's actually seen. Make sure you have shapes enabled as Bifrost graph shape.

So if I now create an output, you will see it will give me a USD proxy 1. So if I expand that, you will see that it's seen as translate. It's foreign data, but it's a translate. So now, all I need to do is, go back here and change that and give it a name. This one's going to say kill pod, and that will be renamed automatically here. Now, next thing is to create a reference definition. So I can create a reference and then a layer. Now all I need to do is go find my object again. USD. See, it's back to the same thing. As you see, this here now is a unshareable layer, so you are doing everything inside of Bifrost.

OK. All right. So last thing, I'm going to delete all this stuff, collapse that, and we're going to unhide the pod. Now, we can also have the ability to create a scene stage with new layer. That creates an anonymous layer and it gives us this stage, Shape 1. So now what we can do is, we can grab all the subjects and duplicate two as my US data to stage one.

Now you see that appears here. I can isolate that. That comes in USD form. All I need to do now is repeat the steps, just creating that, add in my cables, and the rest of the geometry. OK. Now that that's done, I can hide that object. And you will see, the information here is kept, all that information is here, and now I have a few options.

You can see that the Maya data leaves in gray alongside with the USD foreign data. Now, what I can do, I can select all the objects in the screen. I have the ability to change things, like rotations here. I can do all these operations in USD format. But say I wanted to edit. I wanted to match this transform to that other transform, all I need to do, in this case, is grab that and edit as Maya data. Same with this one, going to find it and edit it as Maya data.

As you can see, I have now the ability to edit it. So now I can do or perform any changes, like for example, modify faces, and do whatever I want. But I also have the ability now to match, say, transforms like, say, modify and match rotations. Once that's done, I can just merge to USD, and that goes back into this part. So that's some of the operations that you can do inside of Maya as USD as you work normally.

You also have the ability to snap a line. So if you want to snap to a specific object, say I want to bring this sphere and snap it here somewhere, I have the ability to do so. So you can see, it's quite fast. See that there? So snap two points, it works quite nicely. It is quite fast.

So one last step, once you're finished editing, is just to save the session layer. So at the moment, it's an anonymous layer. So what you're going to do is, click in here, and just go find where we want to save that. In this case, I'm going to save it right over here. As you see, I have a pod USD. I can either save on top of that or just give it another name, say Pod version 02, save that. That replaces that in here, so I can save the stage. Now the stage is saved, and I can start bringing it in a different object or in a different scene.

So another way to bring in USD data is through Arnold as a stand-in. So you can click in here, go find your USD, and that will bring it in as a stand-in USD. So you have the ability to change view draw mode, viewport draw mode. You can see the different types. You see Shaded, that's going to give you the actual pod in there. Now you have the ability to work Arnold procedurals, so alembic settings, if you do have certain alembic settings. And you can also add different looks, create a new look, for example. You can see the contents of the file over here, and you can also add specific assignments like a new shader or add an operator like this. And if you show the graph, you will see everything in here.

So regardless of how you import your data as USD, or you how do you bring in your USD data into Maya, you can, as long as you use the Arnold render, Arnold plug-in configuration when exporting, you should be able to see every single texture from Arnold. So again, if you use this, it's going to support all of your textures and all your lighting information, and you will be able to see it. So just a quick render just to show how accurate that looks, and it doesn't matter if you bring it through Arnold or if you bring it through the Bifrost graph or Maya directly. You're always going to get your results and for every single variant, you'll be able to see your render.

[END PLAYBACK]

JEAN PAUL GIANCARLO: OK. So that was covering the export/import work with USD. Now we're going to see how to assemble an asset variance and define the model view per purpose, so proxy and render. These are also composition arcs, or variants, and we're going to get an asset ready so we can then later procedurally build an entire environment.

[VIDEO PLAYBACK]

- OK. So one of the advantages of USD is it gives you the ability to create and purpose an asset for specific uses. In this case, composition arcs, we're going to be using in order to create different variants of the model. For that, first thing I want to do is create a stage, Add to Stage. See, I created a proxy shape, shape 1. Now I'm going to create a new definition, this one I'm going to call it Pod Asset in this case, because it's going to have different types of variants, different type of things.

One thing you want to make sure, it's set to default, but this one's going to be the one we're going to be using for rendering. So by default, it's set to proxy, so you want to check both for the moment so you see everything, otherwise you won't see the result on screen. So now that we have done that, we're going to create a new variant set.

The variant set, we're going to call it Pod_Vars for variants, and then I'm going to select my Pod_Off which is the one by default. Now from here, I'm going to create a new children, and this children is going to be my very first pod. So Pod_Off is the name of this one, and I'm going to copy this information from the variant set back into the variant set name. Then again, the selection is just going back to here.

So now that you've seen this, basically that you set this up, you now can see in the outliner that it has a new icon as a V4 composition act variant. If I right-click, you see it appears right over here. Now it's a new variant set, so now what we need to do is start bringing in a reference, or a reference definition. Why we're using a reference definition? Because we want to be able to change without breaking anything, so non-destructively.

So now that we've done this, now we can pretty much make sure. Let's go find that asset. See, it's over here. So we see some textures, in this case. So now what we can do is just bring the rest of the variants in. You either can copy and paste this for your convenience, and then just change these, for example, to On.

Go find that On pod, and pipe it in. So now I have the ability, again, to see different variants, for on or off. You see, there was a slight change. Let's go back again. A slight change, a different object. Yeah, and finally I can bring one more variant, and you can store as many as you will, if you want. In this case, it's going to be an open version of the pod. And again, all I need to do is go into the layer.

I think I broke that, so let me fix it.

OK. And now, you will see, I have the three variants, Off, On, Open, with the open version, on and off. OK, so one last thing now. I'm going to bring in another children from the pod asset, in this case, for the USD prim, and I'm going to call this Pod_Proxy. In this case, instead of default, I'm going to set it to Proxy. No variant sets, no need for that, so I'm going to create another reference definition. It goes back to the top, another layer. Then I'm going to go find that proxy file. OK.

OK, now that I have done this, I can go back into the outliner, select this guy here. If I go to the Attribute Editor, you will see I have to uncheck that, and see just a proxy version of it. So now I have my full versions, which I can see in different versions, and I also have to be able to see just a proxy shape on the screen.

Now let's have a closer look at the anatomy of this particular asset set-up inside of Bifrost. So you can see, recapping, you have the pod asset that we created first. Then we could have variant sets, one children for proxy, and one children for renders. The proxy, we created another X form where we reference in the proxy. Then for this one, we created a Pod Open, the X form we referenced it in, so on and so forth. That, in order for us to see again or have the ability to see this version or that other version. So we could also access different types of variants on screen. We're doing this so we can optimize the way we work and make it more memory efficient as we start adding more and more instances of this specific module of a pod.

[END PLAYBACK]

JEAN PAUL GIANCARLO: OK, now that you've seen how to assemble an asset and add variants and define the purpose, I'm just going to start showing you how to build an entire environment procedurally with Bifrost. So how we're going to go now, we're going to build construct points using the USD Point Instancer, and we're going to set prototype definitions using an existing prim path.

[VIDEO PLAYBACK]

- So next step, I'm going to show you how to build a full tower or a building out of just this one particular module, procedurally using Bifrost techniques. So let's start by adding an add to stage node. As I want this to have to coexist, we leave in a separate structure over here. So I'm going to cut this, going to connect that to that, and then back to add to stage.

OK. So next step, I'm just going to add a new prim definition. This one here, I'm going to call Pod_Tower, which is what we are going to build right now. And from here, I'm going to create a new children, and the new children, I'm going to create a point instancer. A point instancer node. And for the points, I'm going to be using a compound that was already pre-built by a colleague of mine, [INAUDIBLE] and I'll explain how that works in a little bit, and I use that to connect to my points.

Now, next is to build the prototype definitions. So for that, I'm going to create another USD Prim, and this one I'm going to call it just Tower. Good. And from here, now I can start creating a reference definition. So this time around, what I want to do, instead of using the USD that I might have exported before, I want to use a specific prim path. So for the prim path, let me bring this up here. What I want to use is this pod asset, which is over here. So I'm going to just copy that, and I'm going to input that into the value over here. So now you have seen, now that I've done this, let me adjust my camera clip settings. Basically, I just created a full tower out of just that one particular module, which I can now just, say, pretty much a radial count, you can see here. Oh, the vertical count. You can add more or less, say, stories to that particular build.

Now, let's kind of dissect these construct points pretty quickly, how that works. So all this is doing here is finding out the distance between that first pod module and the origin, then it's putting them all into different arrays and adding a pi node in order to kind of figure out the circumference, and pretty much starting to multiply. So then, you can obviously start adding different amounts.

So let's say, if I set that to zero, I only have at the moment. Actually, let me do just one. So you see, and I'm going to set this to one. So now I have just 1 over here, so if I adjust a radial count to two, see it goes right on the other side, that's because it's finding out the distance between this and here and it's multiplying it, and it's just going in the other side. So, say, instead of three, six, and as you can see, it's just multiplied around that. So I know, by fact, that it's 27 sites, so I can complete a full tower over here.

So last step, I'm going to show you how now I can create a full city out of this one particular building. And it goes the same way, now, let's inspect the outliner. As you can see, I first created in pod asset, then I created a pod tower as a separate group, and now also another separate group I'm going to bring all the city. So it's pretty much the same thing, I'm going to do another add to stage, which is over here. I'm going to connect that to that, this to this.

OK, and let's, I guess, move it over here, just go up. And I'm going to create another prim definition, or actually, I can create just a point instancer, go directly to a point instancer now. So I create these instancers here, and so for the points, I'm going to create a scatter.

Scatter points, and then connect that to the points. And let's create, pretty quickly, a plane. So let's have a plane, as big as we can. Let's expand these again, it's a big city. OK. Let's pull the shape. Let's make that a little bit bigger. And now if I right-click and hold, bring that over here, now I can simply connect that as my geometry.

And for the prototype definitions, I'm going to create a new USD Prim. I'm going to do the same thing we did before. These, we're going to call it City. We're going to create a reference definition, and then we're going to specify a prim path. Before we do, I'm going to take that amount from 1,000, which is the default, to, say, just 10 just to start, so it's not too crazy over there.

And what we want to do now, is the same thing, just copy the pod tower that we created before, and just going to put it into the value here. Hit Enter, and you see now we have procedurally built kind of a city. So we pretty much just scatter across the plane, which then can now be modified, a change and whatever, because it's all built procedurally. We can change things like blue noise, we can add more buildings, we can have 20 building, or as many as we can. And because this is all built procedurally, we can still adjust. See the vertical Cam, if we want to have smaller ones or bigger ones, and we could even bigger, add taller buildings.

And yeah, I think you get the idea how you can build a full city procedurally with USD, Bifrost, Maya, and Arnold. OK, so now that we're finished assembling the scene procedurally with Bifrost, last step is just to save these. It's a whole package. So there's a node called Save USD. Add stage. All we need to do is connect that to there and these to there, and now all I need to do is just, first, go save where I'm going to go, and place it or find a place where I'm going to save this. This one's going to be called Matrix City EMV, for environment, save that. And all I need to do now is save here, save in Maya file, and we are good to go.

Now, as you might see, it's being saved as a relative path. I can uncheck this now, and I'm good to load it in a separate scene. Now, let's just go ahead and do that. So I'm going to create a new scene, and save everything, and now we can simply go and create stage from file. City environment.

That is huge. Sorry. Ten. There you go, now we've successfully loaded all those buildings into a new scene. So next step is going to be then, maybe bring in new layers where we can create maybe some passes for animation, some passes for VFX, et cetera, so on and so forth.

[END PLAYBACK]

JEAN PAUL GIANCARLO: All right. So that completed the full use case. I know it's a little bit simple, but you get the idea. I know there's a lot that you can do with USD and Bifrost procedurally. So, yeah, thank you everybody for attending this Autodesk University talk. I hope you enjoyed it, and thanks again.