Get Ready for Real-Time Virtual Production in 60 Minutes

IGOR MACEDO: Hello, virtual prediction enthusiasts thank you for your time. And I'm very excited to present this class to you. It's going to be legendary. And I hope you get motivated and confident to start building your virtual production laboratory as soon as possible. The goal here is to get you ready for real-time virtual production in 60 minutes.

But you'll soon see that this subject is multidisciplinary and requires a lot of knowledge and practice in order to produce great content. This is not a sponsored class. So every manufacturer and software provider that appears in this presentation comes from our experience. And different ones can be chosen. So be prepared to receive an astounding amount of knowledge in a small amount of time. Now, let's get started.

So this class was planned and created as part of a training we give to everyone here the company that gets excited about and wants to work with real time virtual production. We divided the class into four main essential learning objectives that will be explained in detail during the class. Those learning objectives are in chronological order. And we'll use the concepts and knowledge to understand and learn how to apply them in a production environment.

We'll start by talking about the basics of real-time virtual production and how we're using it to transform our content production forever and for the better. Honestly, we're now able to do things we didn't expect to be possible before and with great ease. We'll then dive into how to choose the equipment, hardware, and software while explaining the importance of them in the VP workflow so that you'll be better prepared to understand our RTVP pipeline.

That being said, what would be sometimes talking about traditional studio equipment, we'll be focused on the aspects that are more important for virtual production. After understanding the importance of RTVP, its applications, and the equipment involved, we'll dive into our lean production workflow, our RTVP pipeline, and essential steps for what we'll show to you in the first learning objective. The last objective is aimed at wrapping up everything that was learned, talk about the construction process of RTVP lab from Revit to the physical world, and get you motivated and confident to invest your time and budget into the real-time virtual production world.

During this class, we'll follow the learning objectives and use them as checkboxes in order for you to follow along. All the materials and the knowledge that I'll share with you are focused on what works for us and what is possible with the technology we'll present. Ultimately, I want you to learn from our mistakes and also from our successes. Now, let's get started already, shall we?

Real-time virtual production is part of the spectrum of computer-aided production and media production techniques. And to give you a good start on this concept, real-time virtual production techniques enable content creators and filmmakers to interact with the production process in the same and sometimes better way that they're used to interacting with the live action process. Essentially, virtual production connects the physical and digital world together.

How? RTVP combines virtual and augmented reality with computer graphics imagery and graphics engine technologies to enable production crews to see their sets and stages as they are composed while capture everything on set. In the end, those techniques can be used as a means towards creative problem solving. When I first saw and understood the consequences of using the technology, of course, I was very excited to get on board.

Our company is specialized in XR software development. And we received a series A investment last year from Inter & Co. We're currently based at this amazing, modern building in the city of among other Inter colleagues. While we're specializing extended reality software development for business, we also gained the mission of enhancing and contributing to our existing Inter & Co. departments and initiatives using XR technologies.

The first department that could receive a great help from the XR world in the least amount of time was the marketing department. Since they have the all-around need for amazing content and media production, empowering them would be super beneficial for everyone involved. And also, it was a great opportunity to contribute to innovation and cost savings in the long run at Inter.

Real-time virtual production, here we go. So we had a few production things that were mapped that could receive contributions from the real-time technologies that we were so used to work with. But we also had a few business requirements and strategies to follow, such as quality improvements, help departments do more with less, reduce cost of production, while improving quality standards. Driving innovation as much as we could, rely on automated pipelines and frameworks to increase productivity and ultimately allow more creative freedom.

Believe it or not, real-time virtual production was the answer. And you'll understand why during this presentation. Let me tell you exactly what the production needs were.

So we wanted to pre-visualize as easily as possible for validating the complete production process. This helps creative teams to explore ideas, including staging camera composition, editing, and lighting, resulting in a custom-made trailer to demonstrate the project potential. We soon felt the need for virtually simulating the dynamics of each physical shot so that we could anticipate possible difficulties and solve them before the shooting day. We'll explain this in more detail later in this class.

Inter & Co. and several of our clients were in need of constantly producing live content-- events and presentations. You can already feel that by changing the production scenario from the set preparation work to a digital stage, preparation would allow creative teams to have fewer expenses with logistics, renting, and physical materials. And especially, recurrent productions would benefit from usable and easy to customize digital scenarios and virtual production frameworks for inserting videos, logos, 3D animations, and more.

The real live program-- we wanted to create live presentations for all kinds of things, such as product demonstrations and product launch events. Well, marketing materials and commercials could be shot internally and enhanced even more with small production efforts towards achieving high quality media production, and all that while having access to the raw footage and improving our growing library of visual effects, 3D models, and virtual production setups.

Another great example of recurring setups that can be reused are VP frameworks for podcast setups. We also wanted to create courses and educational materials, which again can rely on reusable code from repeat frameworks. We also wanted to explore the potential of virtual talents for our media production. We explored MetaHumans and traditional ways of creating avatars and virtual presenters using Maya in Unreal Engine.

Now, one of the amazing things that can be done with real-time virtual production is something called remote talent. Imagine that two directors of a company are presenting together side by side the launch of a new product to their audience, but one is in Brazil and the other is in the United States. We can basically stream the green screen from one location to the other and do the compositing as if the two or more talents were in the same space.

So basically, those were the things that we wanted to do. And all of them would impact name and satisfy our business needs, which are essentially are continuous quality improvements. Arguably, the biggest advantage of virtual production technologies is that it expands creative horizons. Stages can easily be customized using VP software. And design elements, such as props, colors, lighting, and much more can be edited and tweaked during the shooting stage of production. This empowers the filmmakers to translate their vision to reality without requiring limitless budget to do so.

Using a virtual environment, teams can transition from one scene to the next, contribute to the shooting performance, all of it happening at one location. Essentially, digital assets can be changed up quickly, allowing filmmakers to shoot several scenes with the same actors in a smaller time frame.

Virtual production has completely transformed filmmaking workflows into more efficient processes that save time and money. Visual effects are superposed in real time, meaning that less time is spent on the generally expensive post-production phase. By being able to shoot at virtually any place in the studio setting, producers can eliminate the need for travel. Additionally, since everything occurs in a controlled environment, the crew can save time by not having to account for changes in lighting, unpredictable weather, or any external factor that may impact a shoot.

Well, making creative decisions and revisions earlier in the process ensures that the right decisions can be made before and while actors are on set. This leads to cleaner takes and less rework. Technologies like motion capture, camera tracking, and the use of XR technology often lead to untested scenarios, where innovation can thrive in the form of finding better ways to solve creative challenges, which impacts directly on the innovation efforts of your company.

By leveraging reusable code, data sets, and digital frameworks, the opportunity to automate processes can get even more evident than traditional media production workflows. As it is said by so many experienced teams in real-time virtual production, the only constant is change. This is a big advantage, as changes in the post-production are very time-consuming. Therefore, this technology enables directors, crew members, producers, and as well as actors to make required changes, focusing more on creativity than other physical aspects of the shooting set.

And basically, this completes our efforts towards explaining what real-time virtual production is and how it impacts and transforms quantum production forever. Things will get even more clear throughout this class. Now, let's dive right into how to choose lighting, filming, audio, computing software, and the elements that allow those production objectives to become possible.

When we were planning to build our RTVP laboratory, one of the most difficult things to do was picking the right equipment, in order to avoid unnecessary expenses while having the best quality we could achieve in terms of media creation objectives. Here, we'll focus on how all the different equipment work and communicate with each other in a virtual production environment. And we'll basically start and first talk about equipment that might be already present in the marketing department and what they need to have for shooting with real-time engines.

Well, before even turning on your camera, you need to think about your lighting array setup, aiming for a chroma-key, which will require illumination strategy for all the background and the foreground of your studio or set. In the beginning, we didn't understand this. So this is how we started. And even then, we got some interesting results.

By far, successfully lighting a chroma-key project is the most important one for real-time compositing. Lighting the background properly can save you hours of needless frustration and problems. For example, when an actor stands in front of a large, intensely lit blue or green screen, two factors come into play and create a spill. One is [INAUDIBLE] the green or blue light that radiates off the screen onto nearby objects or people. And the second is reflection on the skin or other surfaces from the screen because it acts as a large reflecting card.

Both of these issues can be easily solved by using inverse square law, creating some distance between the subject and the green screen. However, reflections aren't as affected by distance. So other methods must be used.

A very simple technique is to light the background at a lower level than the foreground. Remember that the more lumens you put on the background, the more color it will reflect onto the subject's body, face, hair, and clothing. For classic chroma-key setups, the light in the background should be about half of what you use as key on the foreground.

And the theoretical ideal key will have a perfectly even color in the background. And the most common choice when it comes to lighting chroma-keys is to use something called space lights and lanterns that help you produce diffused and soft lighting. In our case, we used a mix of Aputure LS 600d Pro's and the LS C300d II to light up our final RTVP lab.

Now, the foreground must be lit carefully to preserve the even color, while interfering as little as possible with the lighting of the subject. Here, it is especially important that the lights are not pointed straight down or angled toward their camera from the [INAUDIBLE] position because we will create something called specular highlights on the floor that will appear in the compositing as [INAUDIBLE]

And then you can think about messing with some colored light effects in order to reproduce in the studio the light effects that will be present on your virtual scenario, such as police car lights in front of a talent, and ultimately match the color of a scene. Another example of this technique can be used when filming a scene with a virtual sunlit exterior. The talent can be made to look as if it's lit by natural sunlight by illuminating the subject with RGB lights that mimic the position and the color temperature of the virtual sun.

Now, when it comes to the camera, what are the most important features that a good camera must have for doing virtual production? First, you want a video output that is either HDMI or SDI, which stands for a serial digital interface. And it is a standard for high quality video and audio transmission over coaxial or fiber-optic cabling. The speeds currently range from 270 megabits per second up to 12 gigabits per second for SDI, which is the interface that you want in a professional setup.

The more bandwidth, the better it comes to outputting, for example, raw video signal at 4k. You'll need to receive the camera feed in the video card of the computer that will do the compositing in real-time. So the type of video interface will basically dictate the input of the capture card that you have to buy.

There are different levels of quality that are not related to resolution. So depending on the sensor size and the optics, you end up with different definitions for the same 4k image. We started with a Lumix GH5, did some testing, and ultimately selected the Blackmagic URSA Mini Pro 12K, which has all the features that we wanted, especially timecode and genlock support.

Clearly, an important tool for real-time virtual production is the camera use. We have experimented with everything from webcams to mirrorless hybrid cameras. But these are not ideal for the RTVP job. When you're selecting your camera tools for virtual production, an extremely significant feature to have is genlock. Anyone who's familiar with broadcast systems and multi-camera studio shots will be familiar with genlock. But those who are not in the know, genlock syncs up the frame capture of different devices.

Lastly, you'll want to couple your camera with optics that can output the best image quality as possible. But that's only the start. You want to capture and sync all the camera data as much as possible, especially the zoom and focus parameters. This can be done by attaching encoders that are sensors that will read rotational data of the lenses that they're attached to.

But what I truly recommend after using them is to invest in a full servo lens that has internal encoders that will output less data in a more precise and easier way because there will be no moving sensor parts attached to the lens. And that alone will minimize the need of doing lens calibration, which is super important for final compositing quality, especially in a tracked camera virtual production setup. That leads us to the tracking systems.

The use of tracking systems-- basically, they are simply amazing. And they take real-time virtual production to the next level. Camera tracking is a technique that allows us to make virtual cameras match the movements, positions, and other lens data, such as zoom and focus, from real world cameras to the virtual cameras within the real-time rendering engine. It works by having optical and inertial camera trackers that inform the graphics engine about the physical orientation and position of the camera in real time.

With this technology, the correct perspective of the production camera is rendered relative to the virtual environment, merging the virtual and the real prediction worlds. That alone will allow you to use camera cranes and do some crazy things, like making the illusion of having multiple cameras by switching from one cameras to the other, moving the first one, and then switching back to it.

And one of the astounding possibilities of virtual production is to put real people and objects into a digital environment. To better represent this effect, it's advised to track the 3D background so that it moves in sync with the real camera movement. This, of course, has latency coming from different devices. And genlock will help you sync that. The tracking system will also help us simulate lens distortion effects on different positions and zoom levels as well.

We initially tested camera tracking with a [INAUDIBLE] Sync and iOS Tracking app. But after seeing the benefits for ourselves, we decided to switch to one of the best available camera tracking technologies today, which is ncam Reality System. It has inputs for camera encoders and several positional tracking strategies using both reflective markers and natural features to use computer vision algorithms to do the tracking.

With lighting, camera, and tracking technology, we can then proceed to the audio equipment. And by the way, this was supposed to be straightforward, but it's not. There are some caveats and tips in order to get it right as well. One good tip here is to get a set of microphones that will work with the same RF signal. And that will save you some money right away.

We got a Sennheiser EW 500 and a [INAUDIBLE] so that we could choose from different microphones, depending on what we were doing in the virtual production. Lab for communicating between the command room and the stages, we got some SV200-W, which are good and almost inexpensive compared to the others.

For mixing, routing, and managing audio outputs and inputs, we use Behringer X32 consoles. They have all kinds of features, such as applying effects, filtering signals, digital output of audio, and a whopping 32 channels to work with. But for RTVP, the amazing part is that it can also get genlocked.

Or we can apply a specific delay [INAUDIBLE] to precisely match the latency of all the audio systems that go to the real-time processing. If you're using a digital output of a mixing tape, the latency correction will not be applied to it. And you have to tweak the audio latency with a virtual production software, such as Aximmetry or Pixel Tube.

Now, let's talk about computer systems for a real-time virtual production operation? So what are the priorities when building a virtual production workstation? Well, a content creator workstation deals with large amounts of workload-- compiling shaders, making lighting. It benefits for multithreaded CPUs for those use cases. But a workstation that will render things in real time gets more advanced advantage from higher clock speeds. And each process [INAUDIBLE] as fast as possible, running frequently single-threaded tasks. A good balance between the two are threadripper processors, such as Ryzen 5995X. But it should probably be OK with a desktop Ryzen 5950X for most cases.

As for the graphics card or the GPU, the more VRAM the graphics card has, the better it will be at handling high quality and bigger 3D scenes that must be rendered in real time, while not going over the memory pool. If you're using a green screen studio, you'll be able to get by with just an upper tier GeForce RTX, such as the RTX 3090 or the upcoming RTX 4090.

But when using LED walls, your GPU will have to have the ability to synchronize with the LED walls, with multiple cameras and ray-traced graphics, which will probably require an NVIDIA Quadro Sync II board as well as a Quadro RTX card. In this class, we'll focus on the green screen setups as they are more budget-friendly. And basically, they're what we're currently using.

Every camera will end up feeding its video signal, outputting it through the SDI or HDMI to the computer that will composite the keyed elements from the green screen to the real-time rendering engine. And for that, you need a video capture card that can receive video inputs and output the PGM, or program, which is the composited result of what the virtual production software will be computing.

While you can use multiple cameras with a single computer, that is not advised because you lose the possibilities of evasion maneuvers if a computer crashes, for example. And you also put the workstation under more computational stress, which can eat up GPU resources easily. Another amazing usage of the video capture card is for receiving inputs from external sources that are not cameras. This can be, for example, external PCs, like I'm doing right now, or even casting devices such as Apple TV or Google Cast.

This allows you to put those external video signals into the 3D environments of the real-time renderers. And they can also be calls from a meeting software. They can be live presentations. They can be virtually anything you might want to give control to a talent that will be presenting.

One of the amazing things that we do all the time is to put a slide presentation on the laptop that will feed this video to the real-time virtual production workstations while giving the user a presentation controller that has a USB attached to the laptop. This generates a safer IT environment, as you'll be isolating the VP workstation from external files, while giving the presenters full control over their content.

And when dealing with multiple 3D scenes, loading things up in real time, buffering content, [INAUDIBLE] on the VP software recording something, depending on the available workstations, you most definitely want an ultra high speed storage device with plenty of storage. Remember, when loading anything on a computer, you're dealing with reading and writing operations that will take data from your storage hardware to your RAM, video RAM, or possibly both. So go for a PCIe 4.0 NVMe with 7 gigabytes per second of speed with at least two terabytes of storage. You won't regret it.

Now, for control EVP software and creating macros for common operations, we use a MIDI pad and an Elgato Stream Deck XL. This leads to easier operations and the ability to create profiles for each production scenario. Remember, we want to create profiles and basically map the most used features as templates for productions that can leverage reusable code, 3D scenes, and much more.

Now, beyond the RTVP workstation, you want to have a directing workstation that will receive multiple RTVP video outputs and control what is being shown to the viewer. This computer doesn't have to be super powerful. But it will help you ease the burden of the multiple things a VP operator must deal with. The video director will switch between the different cameras, control overlays, and live transmission during the production and shooting.

In our pipeline, he takes care of all the video routing and preparation of inputs and outputs between devices in the stage. This computer will also record the PGM that can sometimes be a raw format. So be sure not to forget that NVMe. For overall storage, we bought two great NAS servers that allow us to work with high speed 10 gigabits per second and backup our files on multiple cloud storage at the same time.

Talking about routing, there is a generally misunderstood piece of equipment that is a killer hardware for the RTVP laboratory. But before talking about it, first, there is an equipment that can receive multiple video signals and output them in little squares that's called the multiviewer. And you can also monitor audio signals that are coming from the SDI connectors into this device on the fly.

This is super cool for not only visually monitoring signals, but it helps you diagnose fast during production-- for example, by literally seeing if a certainly required video and audio signal is being transmitted appropriately and where it came from. You will want to monitor your external devices, such as the slides computers and casting devices as well, not only your camera's raw green screen video feed and composited outputs from the RTVP computers.

And that gets us to the SDI router. This piece of equipment works as the brain of the media input and outputs. What it can do is to route any input to any output and duplicate input signals on multiple outputs. And this basically allows you to create setups for different production scenarios on the fly without having to touch your installed cables ever again.

Basically, you'll want to plug in almost all your video signal outputs into the video hubs channels. And from that, you connect almost every equipment that will receive some kind of video input signal. And then from the video hub, you'll be able to duplicate, change what kind of video signal gets outputted on the fly, again without messing with the existing cables. And this can lead to a cleaner studio environment, as the cables won't have to be touched after a careful signal routing project. On both SDI router and Multiviewer, you name your I/O accordingly with easy to understand descriptions.

Now, with all your signals configurable directly, we can dive into switching and monitoring. This was something that I did not give much attention in the beginning, until we faced some production issues. Having your monitoring ready, you'll want to have a Switcher that matches your audiovisual signal interface so that you can control the PGM or the program output that goes into your recorder that, in our case, is the directing workstation and the Atomos Shogun seven inches.

If you're starting out with RTVP, you'll see that a lot of softwares have the same kind of video switching overlay capabilities. So why would we want a hardware switcher? First, you have other capabilities with this hardware. For example, it will have internal encoders that can stream without using your workstation computing resources. But here are a few of my favorites and what I consider to be the most important things that you able to do with this equipment.

When a virtual production workstation that receives input from the camera and other devices crash, you're doomed if you don't have a spare camera to switch to. And let's consider that all the workstation computers crash. And you're only left with the slides that were coming from the external laptop with the slide presenter. It's easy to see now that depending on how many inputs you have, you have the same amount of evasion maneuvers to make. And this can give you enough time to restart things during a live presentation and get back on track in the case of an expected event.

Using a dedicated switcher, you can switch back and forth between two cameras, for example, while changing one or more camera positions. This allows your viewers to feel like they're watching a video with many more cameras than you actually have. And that will straight away save you a lot of money. This is literally super cool.

One thing is to capture a great image. But another is to compress it and stream it, delivering the best quality versus performance ratio to your online live audience. So to add on top of evasion maneuvers and also ease the computer requirements of the streaming process, you might want to get a dedicated encoder, such as the Blackmagic Web Presenter 4K. Adding to that, you can attach a backup internet to this device and be safer in terms of, let's say, the energy goes out and you have no break for your equipment, but internet goes away. That is simply amazing.

Well, your external laptop, responsible for those lights or the casting devices that you want to have available, most likely has an HDMI DisplayPort or a Thunderbolt port. And well, of course, none of them are SDI. So for that, you'll want these versatile devices that are bi-directional SDI to HDMI converters that can receive and send signals from one interface to the other.

We also got reference monitors for optimal color accuracy and reference monitors, like Blackmagic Video Assist to be added on the cameras so that the camera operator can see both green screen and the composited image from the virtual production software. So for recording, like I said before, we used both the directing workstation and the two Atomos Shogun reference monitors, one for each camera.

We mentioned before that synchronization of video frames and signals is crucial when working with multiple systems that might introduce some sort of latency. When you install all of the sensors in the device that you use, you have to sync everything up. Why is this important? Well, in the case of virtual production where you have a camera, a motion capture device, and software generating a virtual backdrop such as Unreal Engine and maybe an LED volume, all of those must be frame-synced to avoid problems.

Well, the timecode feature is a useful tool for synchronizing footage and audio because it helps you manage your meeting match clips to the sound files, this information is embedded and recorded raw files as metadata. You can set up your camera to generate timecode on its own or sync to an external source. Timecode is most accurate and effective when coupled with genlock.

Now, genlock, or generation locking, is a broadcast technique used to synchronize multiple audio and video signals with more accuracy. This enables precisely frequency-locked video signals, where every device knows when a frame starts and when a frame ends. More specifically, genlock uses a reference signal as the timing plane for the entire broadcasting system.

And just as a reminder, it is generally a good idea to opt for a genlock-compatible camera. A sync generator device will help you with synchronize all-- synchronizing all connected devices to its internal clock, delivering flicker-free, tier-free, and high quality mix output.

And finally, let's talk about the software packages that we can rely on. I'm going to be honest with you, the entire software stack of a virtual production framework can be gigantic. Here, I'll focus on what is more important when it comes to getting started. In the GitHub repository, software and equipment lists will be available so that you can check it out, serve yourself, and contribute to this community that we want to build.

For altering 3D assets and scenes, depending on the workforce you have available or your skills, you might want to start with an architectural project software. The reason is simple. In a camera tracking setup, the scale of the 3D scene should be accurately 1 to 1. So our AC team generally starts a 3D scenario in a very similar way to a traditional construction project.

If you're working, aiming for 4K real-time ray-traced rendering, creating an amazing 3D scene is not going to be enough. It must be optimized with proper UVs, caring about material complexity, draw calls [? count, ?] and memory consumption. Or you end up with limited rendering resolution capability. Those will have a direct impact on the performance and realism that the real-time rendering engine will produce.

Unreal Engine is a great development tool. And it is the core rendering system of this framework. If you haven't heard of light baking, check it out, as dynamic illumination will impact rendering performance from both [INAUDIBLE] and ray-traced workflows. By the way, a great tip here is to use and abuse the NVIDIA DLSS, which stands for deep learning super sampling. It will dramatically improve your rendering performance with minimal graphics quality impact. Oh, and it's free.

To get started with real-time virtual production fast, reliably, and with a lot of learning materials available on the internet, honestly, just download Aximmetry. It has a Community Edition that is free with watermark. Even if you have, let's say, a webcam and a do it yourself green screen, you will most probably work out of the box with it. And you'll be playing with real-time virtual production in a day, for real.

Aximmetry will save you time and money when it comes to programming, using overlays, controlling inputs and outputs, creating super useful control panels, and even making them available through web server. Added to that, you'll be able to connect your tracking system, stream, record. See for yourself. I got started in the world of real-time virtual production with Aximmetry powered by Unreal Engine. I fell in love. And we've been using it for almost a year now.

In a bigger and more complex or mature environment, you'll probably couple your tools and equipment with specialized broadcasting software. vMix is an industry standard. And it has a lot of features. One of the features that we use the most is putting remote talent and managing video calls within the virtual production framework.

A free alternative that can be used is OBS. But sometimes, you might face some limitations that are built right into vMix. In a virtual production stack, you'll want to use the best of every tool. Most of the time, you'll be working with softwares that have overlapping capabilities, such as streaming, that can be done by Aximmetry, vMix, OBS, your switcher, and ultimately your dedicated Blackmagic Web Presenter 4K encoder and streamer. The same applies to chroma-key. Unreal Engine, Aximmetry, OBS can do it. But you'll most likely get better results and free computing resources if you use a dedicated Blackmagic Ultimatte, for example.

When we started, we explored a combination of hardware and software that went from basic and budget-friendly tiers to more advanced and professional ways. Now, let's talk about what you can expect and achieve from the phases that we've gone through and also show you where you're going after as the plateau for our media production objectives. Just to remind you, feel free to choose, test, and experiment with multiple combinations of available software, hardware. I truly encourage you to do so, as you end up getting your grips of everything and confidently make your decisions after validating your needs.

Now that you have heard about equipment and the frameworks that we're using, let's talk about the money and some hardware combinations. We divided the evolution of our real-time virtual production endeavor into four tiers. And I'll show you how much you can expect to spend in each phase and the limitations and possibilities.

If you just want to get started experimenting with virtual production, this is the simplest setup you can make. And it can already be a lot of fun. An investment of a low cost tier 1 can dramatically increase the quality and possibilities of media production already. Here, if you have a camera with a video output, capture card, a lit green screen, and a computer, you'll be doing RTVP in no time. Just download a VP software like Aximmetry Community Edition, which is, again, free, and follow the tutorials. For this case, the investment can get up to $10,000. And you'll already achieve very nice results.

And on the tier 2 investments with $5,000, you can get a VIVE Mars that was launched this year. With that, you'll be able to track up to three cameras for a super competitive pricing. This wasn't available when we built our lab, so I couldn't forget to mention it here. And I would already recommend getting better equipment as well that has genlock and SDI outputs in order to improve your setup or you'll face issues when combining those devices. Add more computers, at least two cameras, and license for softwares, such as Aximmetry, and you'll be doing amazing stuff with an investment of up to $50,000. And that's basically it.

Now, when it comes to the tier 3, in this stage is where we are right now with all the equipment that I presented before. The complete list of devices will be available at the GitHub repository that will be shown at the end of this class. With this investment tier, you'll be able to shoot great media content for the entire marketing department and even provide some real-time virtual production services to all your clients.

You will definitely want help from more experienced professionals that will validate your infrastructure. This knowledge cost will actually help you save money by not spending on unnecessary or wrong devices. You'll be able to have outstanding professional grade equipment to power your RTVP lab with an investment up to $400,000. Depending on how much your company spends hiring external production teams, building sets for events, and other production scenarios, this investment will actually save you a lot of money in the medium to long run. This is exactly our case.

Now, with the technology and media production, the sky is the limit. And it really depends on the complexity of your production. Sometimes, you might want to rent equipment instead of buying them. But the biggest difference here with the LED volume setup is that you'll be capable of doing something called in-camera VFX, which is basically capturing the visual effects right from the camera recording.

On top of that, the LED volume will be able to provide accurate and realistic lighting and reflections. So your overall workflow will have a lot of changes. And you'll need help from other kinds of professionals on the production team to have-- to help you out with all the added systems.

This concludes our second and largest part of this class. Now, let's dive right into the ready-to-follow workflow and pipeline that covers the real-time virtual production side of things with a use case that needs external inputs, such as lights that I'm using right now. The way that the pipeline works is by minimizing or mitigating post-production work while increasing pre-production work. That allows us to better handle and fill the RTVP lab agenda with ready-to-shoot work and ultimately help multiple departments at Inter & Co. in the most efficient way.

Our goal was to have business and production requirements to be fulfilled at the same time, which we not only believed in. But after year of building and developing, I can confidently tell you that the dream came true. The roles and responsibilities that will be presented will most probably have to adapt to your reality. And you'll see that for several production scenarios, a bigger team will be needed.

In our case, we wanted to have a lean team at the VP lab that will empower external marketing teams of real-time virtual production technology and train them on its possibilities. In this part of the class, I will talk first about the roles involved, and then the pipeline, and how those roles got involved in the processes.

Our XR background with creating virtual reality in real-time engine software gave us the necessary workforce for dealing with 3D content creation and optimization for real-time rendering. We already had a complete organizational structure for software development. But in this class, I'll focus on what roles and responsibilities you must take into account for getting started and having a lean workforce that will enable you to do the things we talked about before.

Believe it or not, there can even be a one man army that will take care of all the responsibilities described, which is what I did a long time ago. But eventually, you'll need help and more talented people in order to improve delivery time and overall content quality. Now, let's get to the real time virtual production part.

For a lean RTVP laboratory team, we plan to have five people. Yes, five people. Of course, depending on the complexity, you will, again, want to hire more people to divide responsibilities. But think about everything that can be prepared and left in a ready-to-shoot state. Think about the reusable scenarios in real-time virtual production setups. You will soon realize that it is possible to achieve great things with a lean team.

The first one is the creative production supervisor that, in our case, acts directly in understanding, managing creative elements, validating the artistic needs from the departments at our company and external clients, of course. And that will come and ask for any kind of media production.

The second role is a technical production supervisor that, in our case, acts directly in understanding, managing hardware elements, and validating the technical needs for the media that is going to be produced. The third role is essentially a virtual production engineer that will take care of the programming, setting up control panels, broadcasting elements, data ingestion, and all the framework preparation in order to operate the real-time virtual production effort.

The fourth role is essential and will be taking care of the engineering operation of audio elements for the RTVP needs. The fifth role will help everyone stay on track on the pipeline, measure time, expenses, studio reservations, equipment borrowing, and create amazing reports in order to enhance transparency for everyone. And these lead up to the pipeline.

So learning from multiple sources and testing things out, we divided the workflow into traditional three stages-- pre-production, pre-light, and production. And then we subdivided them into multiple parts of the workflow. Any change always comes back to the scope assessment. And then the subsequent parts are reanalyzed because each step has a few dependencies on each other. With small changes, our middle stage, like the real-time virtual, production allows it to happen.

The pre-production part is of extreme importance for us. It is easy to see that there are more steps in it. And when it comes to filling the RTVP lab agenda availability, most of the work can be done outside of the studio. This will essentially maximize media production quantity and contribute to the return of investment of the lab. In the next part of this class, on every step explanation, the team involved will be on the bottom right part of the presentation.

Now, let's dive right into the steps. Scope assessment is the process of creating a document that displays all the project information to provide clarity among all team members and stakeholders. When a demand comes in, the creative and technical production supervisors will understand what is trying to be achieved. And the production analyst will collect the requirements while creating tasks around them in the management platforms such as Jira or ClickUp. And it will be present in a shared final report, trackable in the version of control system change log.

Next, we look to get and make available all data necessary for the development and preparation of the project. Things such as scripts, storyboards, and any media listed before in the scope assessment must be collected from the department or the client. In a marketplace live event, this could be images of products, logos from the manufacturers, to the image overlays, video animations that will be playing in the background and so on.

Now, in the content production, here, the VFX, the virtual art department, and the CAD department will prepare all the virtual content required for the project, such as 3D scenarios with their optimized assets. If you've worked with real-time rendering before, you'll be familiar with 3D asset optimization techniques, allowing you to develop large scenes that run smoothly even on mobile hardware.

Throughout the project's development, it is advice to use a virtual control system that will manage and track changes to test recordings files, data sets, or even documents that are pushed to repositories. Every commit that professionals do will effectively store each update as a revision. Each subsequent change can be restored and compared in the file level, being something much more powerful than a simple Control-Z.

On top of that, it enables all members of a team to collaborate from the most recently updated version or separately in different branches that hold different project configurations that can be later merged together if required. I literally cannot explain the amazing potential and flexibility that using a version control system will give to any production environment. But be sure to pursue the knowledge behind it.