Reference Architectures for Success with AI

MARK HIRST: Hello, welcome to Autodesk University 2024. This is The Design and Make Conference. This particular session that you're tuning into is Reference Architectures for Success.

Yeah, I know what you're thinking. It's a really clever play on words. But we will dive into that in just a little bit more detail.

So my solution, my job role is I am a solutions manager. And I have a particular focus on remote and hybrid work. I am part of Lenovo's workstation and client AI business unit. And, again, I'll talk into that in a little bit more detail.

But over the last few years, there's been a dramatic shift. There's been lots of diverse use cases across AEC, visualization, traditional 3D, CAD, high performance computing, and now AI as well. So having this kind of one size fits all approach doesn't really work.

So what we do as part of our team is we look at these different workflows. And we look at our customers, try to understand customer pain points, customer challenges, and try and figure out how we can point our customers in the right direction, whether it's towards a workstation or it's a different direction. So in this session, we're going to have a look at some different architectures, some different ideas, some creative ways to perhaps address some of the challenges that you're facing.

So we do try to look at things through a different lens. And my team is made up of some, well, to be honest, it's a really interesting cast of characters from a diverse range of backgrounds each with their own particular focus, whether it's AI, whether it's spatial, whether it's immersive, or my role is kind of remote and hybrid. But I think the common denominator against all of our solutions team is we do really have a customer focus.

We want to look at things through that customer lens. We really want to understand what your pain points are so that we can guide you in the right direction. Because from a Lenovo perspective, it's really important for us to be able to-- for our customers to be able to have a great experience with our systems.

We've got some of the most powerful, honestly, the best looking, robust workstations out there on the market. But unless you have a good experience with those, whether you're connecting remotely to them or you're virtualizing or you're doing something else a little bit more creative, we want to make sure that's an excellent experience for you, making sure that you've got the right tool for the right job at the right time.

So digital transformation. This is one of those phrases that's been bandied around for several years. And some of it is really transformative. You'll see these four different pillars here. And we're seeing this rapid growth in digital transformation over the next five to six years. We're seeing double digit growth. So digital twins are $142 billion by 2030.

Collaboration. Collaboration is one of those things that can be lots of things to different people, whether it's working on the same data set, it's a Teams call, a Zoom call, whether it's real time remote graphics. Collaboration can mean a number of different things.

So you've got challenges that you face on a daily basis whether it's, how do you bring your teams together? How do you connect to your data? So that collaboration aspect is something that's important and needs to be addressed.

Spatial and immersive. And this has been around forever, right? Virtual reality has been around for a long, long time. I was working in virtual reality. I don't want to tell you how long ago it was, but I've got lots of gray hair, I guess, to kind of prove it.

But working with large virtual reality caves and spatial immersive systems, that's obviously grown in recent years, especially with some of the consumerization of virtual reality and augmented reality. The metaverse has really driven this forward. And then everybody's new friend at the party, GenAI, right? GenAI is everywhere.

AI is everywhere, whether it's actually AI or it's not really AI, they're just-- everyone's jumping on the AI bandwagon. So GenAI is something that's here. It's something that's here to stay. You are likely impacted by it. In fact, the abstract for this presentation-- a confession to make here. The abstract for this presentation was written by ChatGPT, so just full disclosure.

So we're seeing this massive growth. And the underpinning, the foundation, of all of this is data. So how do you get the best from your data? Where's your data located? How do you work with your data? How do you manage and keep your data secure? So that's the underpinning, the foundation, of all of these different pillars.

So we're sitting in the middle of this technology boom, whether it's 3D CAD, BIM, simulation, 3D Viz, HPC, rendering, GenAI, as we've talked about. So it's a really exciting time to be at the forefront of this. And whether it's a fortunate thing or an unfortunate thing, in AEC, you touch on all of these different aspects.

And so as we're looking at solutions, as we're trying to ensure that we can design and build the right systems for our customers, we need to make sure that we're looking at services and software to ensure that it's a smooth experience. It's the best user experience. And you're getting the most from your investment.

So this is just a brief recap of where Lenovo has gone over the last 12 months. There's been a lot of technology advances. Time certainly hasn't stood still. We are now able to hit 6 gigahertz on a CPU clock speed. So for CAD and BIM applications, that is so critical.

They are largely single-threaded, so being able to hit that kind of turbo clock speed on a workstation is a game changer, especially when you start to contrast a workstation, whether it's at the desk or it's a mobile or it's in the data center to a virtualized environment where you can't access that turbo. You'll start to see build times that are 50% better than what you would potentially see on a virtualized server in the cloud or an on-prem data center.

Mobile workstations. Mobile workstations are now shipping with 24 cores. I mean, it's pretty crazy, really, when you think about it. We've got the full range of mobile workstations, whether it's extremely small, thin, and light to really powerful mobile workstations, such as P16.

We're seeing 96 gig of memory support even on the most entry level of workstations, such as our P3 Tiny. We're seeing 4 terabytes of memory support on the ultra high-end P10. This thing is a beast of performance. And with 4 terabytes of memory, we're starting to see some lines blurred between what was previously only seen in the realms of the data center and a server to a workstation class system.

We're making things lighter. So our P1 Gen 7 ThinkPad mobile workstation is coming in at 1.8 kilograms. I should have probably converted that to pounds as well. And I can't do that off the top of my head. But let's just-- we'll stick for the Europeans at the moment and just say, hey, this is 1.8 kilograms. But it's extremely light.

And our engineers have done a fantastic job of not only fitting all the components in there, but we're actually increasing the power on these systems as well. So the performance that you're getting for something that's so thin and light and extremely good looking, which is an added bonus, is something that's really impressive.

You're probably seeing the term TOPS, this new measurement that's thrown around, especially with the advent of AI, the increase in demand for AI. So Microsoft recently announced their Copilot with a 40 TOPS requirement. We're seeing 37 times more than that from the GPU, so over 1,400 TOPS on a GPU.

And then one of the big changes we've seen over the last few years is we're starting to see more and more people take that workstation away from the desk, move it to the data center, and put it next to the data. So in order to be able to do that, you've got to pick the best remote protocols. So we're starting to see things like remote protocols that support 4K displays, multiple 4K displays at 60 hertz, an ultra high definition with full fidelity supporting things like Wacom tablets, a 3D mouse, all these different things. There are lots of remote protocols out there on the market. And we'll discuss those in a little bit more detail.

So ISVs, I've listed everybody's ISV in here. Obviously, we're here at Autodesk University. So many of the CAD and BIM workflows are still heavily driven by the CPU. So gigahertz, because of their single-threaded nature, is extremely important for that CPU clock.

Applications are more demanding, more CPU cores, more memory, both capacity and bandwidth. Memory is not just about capacity, it's also about the speed and the bandwidth. So 64 gig of memory and dual channel is a completely different kettle of fish to 16 channels.

Think of it as like a highway. You're driving in traffic down a two lane highway. If you're on a 16-lane highway, obviously you're going to be able to-- you're not going to have the same level of congestion. You're going to be able to fit a lot more down it. So ISVs are really pushing the envelope. And I think, for us, it's about ensuring that we're able to make the right recommendations, the right tools for whatever your workflow, whatever your application.

So we spend a lot of time-- we've got dedicated teams doing ISV certifications, so all of your favorite Autodesk applications. We've got recommended hardware configurations. We're continually expanding the list of applications that we provide these configurations for.

And we put it through extensive testing just to make sure that all of our systems work with any of your Autodesk applications, so lots of benchmarking. We do performance optimization to get that little bit extra from the systems. So we're continually updating this list. So we can provide that information to you if you are interested.

And one of the things that's often overlooked is one of these business as usual performance requirements. So your desktop applications-- and this is just a snapshot of-- actually it's my boss's snapshot from his laptop that he was using. But PowerPoint, Teams, Outlook, Zoom, we've seen a massive increase in the compute requirements just to run these applications.

In fact, over the last 12 months, there's been a 57% increase in compute resources just to run Teams and Zoom. So if you then think, in addition to this, you're running your BIM, your CAD applications, you're running Revit. You need to ensure that you do have the capacity to be able to not only run your Revit 2024 or Revit 2023, you've got multiple projects open. But you've also got to be able to run your Teams and your Zoom and other things as well.

What also comes into play in this is the remote protocol. If you select a remote protocol, some can impact the performance of your system. So, again, it's about making sure you've got the right tools for the right job and you can support your users to ensure they're productive.

So productivity is key. I'm struggling saying that word today. Productivity is key. And this is for everyone. It includes all of your users. Obviously, from a workstation perspective, we focus-- our typical audience is the power user. So the power user can be anything from a data scientist, a researcher, an engineer, an architect.

This is a user group that requires high compute, high graphics capabilities. And they're also driving the profitability of the business as well. These are often the paid higher than everybody else in the business. So these are critical workers that we're trying to ensure that we maximize productivity of.

And those type of users do face different challenges when it comes to different types of workflows. Remote is a great example of that. Supporting remote power users is very different from supporting just a remote desktop user just using standard office applications, so making sure we got the maximum CPU gigahertz. We've already noted that even on our ultra small form factor chassis, we can hit 6 gigahertz on the turbo clock speed, which is just massive.

NVIDIA RTX GPUs. We'll have a look at some of the frameworks later, whereby we're taking some of our high-end workstations and we're putting multi GPUs in there. So even four times RTX A6000, 128 gigs of memory, that's kind of becoming the standard for Revit nowadays.

A long time it was 64 gigs. We'll just keep pushing that envelope up. And some of that is to do with Revit itself. Some is to do with those kind of business as usual applications.

Having a low cost per user. Obviously, that's very important to the CFO. It's important to the business. But we're looking at solutions in which we can provide a better cost performance ratio for our users, for our customers to be able to support their users.

We need to be lightweight and accessible. And part of this for us is we can develop the greatest, smallest, thinnest, sexiest, lightweight, little ThinkPads But it needs to fit into your workflow, right? We can provide a mobile workstation with more horsepower, with more performance. But if that's not portable, then it's going to not go down well with the user groups.

Everything needs to be secure. It needs to be reliable. Adaptable for GenAI is everywhere now. It impacts you on your everyday basis. We've got to be scalable, future proof. And the job of our ISV team is making sure everything is certified for you within the AEC industry.

So this one's kind of a Lenovo perspective. But I kind of like this slide because it talks about some of the competing challenges. And it really is a balancing act. When we're designing solutions, when we're looking at ways in which we can support you and address your challenges, there's always these competing resources within an organization.

So you've got IT decision makers. Now, they want to maximize their budgets. They want to lower their IT spend. They don't want to over complicate their environment. They want things to be simple. They need to be manageable.

You've got the users, especially when it comes to the power user. They want to be productive. They don't want to be sitting around waiting for data sync over the network, waiting 30 minutes in the morning, having a cup of coffee, walking the dog while they wait for their data to sync every time they check in. That lost time is just a loss of productivity, which is cost to the business.

So we need to make sure that we are recommending solutions that can ensure we keep folks working and not waiting around. You've got the architects, engineers. As I said, they're the power users. The managers and execs, they're kind of selling the dream almost.

You've got somebody read an article in Forbes magazine. And now they're an AI expert, so pushing some of these initiatives or maybe even pushing an initiative, hey, we think the cloud is the right way to go for our business. Let's take things towards the cloud. And then, of course, you've got the guy in the middle who's the CFO, the one that holds the purse strings and making sure that everything is compliant, kind of reducing complexity, and managing the overall profit and loss. So they're kind of stuck in the middle.

So how do you find that balance? How do you keep everyone happy? How do you have your cake and eat it? These are all things that we're all facing on a regular basis.

So if we want to change this, we've got to be able to make sure that we deliver the right tools at the right place at the right time and at the right cost. It's obviously very simple to say that. It can be a little bit more complex. But we're going to share with you some of the ideas, some of the different things that our architects that work on it together as a team really designed for the AEC market that may be able to guide you or help you along as you look at your own environment, your own workflows.

So one thing we'd like to encourage you to do is when you think about a workstation, it's about more than just the desktop, so a desktop, a mobile workstation that you may have at your home office, and especially during the pandemic what we saw is we had to all of a sudden support remote users. We had to now support this remote and hybrid workflow.

So we started to see this trend of workstations rather than just being that 1,500 watt box underneath your desk warming your toes. It's a fantastic tool. It's always been the tool of choice for the power user because you don't have latency. You've got the power of that system at your fingertips.

But we're now starting to see a lot more use cases, especially over the last few years, whereby we're moving the workstation. And we're putting it into the data center. So it's adjacent to data. So you've got your PDM in the data center. Rather than syncing data back and forth across the network, you're putting your compute adjacent to data, minimizing latency and vastly improving that experience. So we're trying to give that power back to the power user.

So when we talk about workstations, I would encourage you to think of a workstation as not just a desktop, not just a mobile workstation, but also a data center system as well. So if we can provide workstation performance, which is what everyone over the years has been the traditional model and everyone has learned to love, but we can also give this cloud-like flexibility. So that's the manageability and the flexibility that you get from the cloud or you get from an on-prem VDI kind of environment.

But if we can pair that with performance, then it's going to be a really good solution. So we've started to look at different ways in which we can add value to the workstation by putting it in the data center and trying some different things. So the different ways in which you can slice this pizza-- one to one, this solution here is actually taking our P3 ultra, ultra small form factor workstation, we are putting seven of these in a 5U rack shelf.

So it's accessible. It goes in the rack. You can slide it out. It's got power integrated in there, cooling with fans. It's got active cooling. But these are one to one workstations.

So if you can hit 6 gigahertz on a turbo clock speed for the CPU, which is obviously very important, as we said, for all of those single-threaded applications you use every day, you can have up to 128 gig of RAM in this little workstation. You can have an RTX 4000 Ada GPU, so 20 gigs of VRAM in that little workstation, and 8 terabytes of NVMe SSD as well. So it's a really powerful system packed in with low density.

What we also do is we then make recommendations on things like best in class remote protocols. What we don't believe in doing is saying, we want you to take this system and this protocol, and this is your bundle. What we want to do, and I think the smartest way for us all to look at this solution, is every remote protocol works in a slightly different way.

They all have different accelerations and optimizations in order to give you the best, most responsive experience. Because at the end of the day, if you click your mouse 1,000 times a day and there's a little bit of lag, a little bit of delay, you're going to be frustrated 1,000 times a day. So we don't want that for our users.

So it's a matter of us providing the recommendations, providing reference architectures. We do all the testing on every protocol on the market to make sure that you've got the right one for your particular workflow, whether you want high resolution displays, whether you want Wacom tablets and 3D mouse, whether you want collaboration, so multiple users connecting to the same session. So that's one to one. And that's kind of your ultimate performance. It could be an ultra small form factor or it could be one of the high-end workstations that are now all rack optimized as well.

Next, we have what I call partial virtualization. So partial virtualization is we're using a hypervisor. Often we're just using an open source hypervisor, something like Proxmox, for example, or open KVM. And we're virtualizing that system.

What we are not doing is virtualizing the GPU. So for example, a high-end flagship workstation, the PX, can fit four double-wide GPUs, so four 6000 Ada GPUs, or eight single-wide GPUs, so 4000 GPUs. We dedicate a GPU to each individual user. So you can have a virtual machine with the dedicated GPU. So it gives better performance for the GPU. But it also helps with reducing costs, licensing costs, for not virtualizing the GPU as well.

If you want to go full virtualization, that's taking one of those systems and you're using NVIDIA's virtual workstations, virtual workstation licensing, to virtualize the GPU as well. So it enables you to have more users on a system. You're not going to hit the performance levels, especially for CAD and BIM.

But obviously, if you've got a workstation like that and you're doing some simulation or rendering, you could potentially-- you're potentially doing AI, right? You're doing some stable diffusion, some GenAI, AI development, whatever it may be. You can have a system that's truly flexible. So during the day it could be allocating it to a particular user as a CAD machine. At night time, you could have that full power of that workstation with all those GPUs, all those cores, up to 120 cores on a P10.

And you could use that system for AI, for a render job, or whatever it may be. So it does give you a lot of flexibility. And you can start to change how you use that workstation.

It's not just that big workstation under your desk dedicated to a single user anymore. You can use connection brokers and gateways. You can share out these resources across your estate, wherever they're located, whether it's at the desk, in the data center, or even virtual systems in the cloud as well.

So this is the solution, I guess, coined the Magnificent Seven. So we've got seven P3 Ultras in a rack. So, again, it's 5U. So it gives you some excellent density. So 3D CAD, BIM, simulation, 3D Viz, rendering, it's a great little box just for that kind of system. So rather than that just being a small desktop, how else can we use something like that?

So with virtualization, this is the P10. So you'll see the side of the system here. I'm not going to go into the technical details of what's what within this system. But it does give us a lot of adaptability and a lot of flexibility. This is NVIDIA certified. So we are using NVIDIA GPUs.

You can have up to eight single-wide GPUs in there and a remote management card. So we do have an add in card for BMC. So that's an out-of-band remote management card. So any kind of remote environment, you've also got to manage it right.

The last thing you want to do if you need to power cycle a system is send someone down to the data center to push a button. So being able to manage these systems, being able to update the bios, the firmware, have all the monitoring and things is extremely important as well.

So at Lenovo, we're really good at giving things acronyms. So we're calling this kind of a workstation a compute solution. And what we've done is we've created this base layer of a system, which is a virtualized system.

Again, we're using open source solutions, hypervisors. So we're using things like Proxmox. You could use one of your enterprise tools, maybe you're a VMware user. And you've got VMware experience within your business. So you could use VMware as a hypervisor layer or Citrix or whatever it may be.

We're using Proxmox because it does have enterprise support, and it's open source, so just to reduce cost basically. So you've got a virtualized system, each with a dedicated GPU. We are supporting your favorite AEC tools.

It's adjacent to your data, which is extremely important. You've got the management GUI in there as well. And then you're just sending pixels over the network. So it's remote pixels, secure.

We're not transferring data. And you can go to-- you can support up to four users in this particular example. As I said, we're doing it for eight users. We're doing this for four users, so lots of different options.

Once you've got that foundation of, hey, this is my flexible system that I've got, you can create and carve that up in as many different ways as you want. So they don't all have to be the same. So you can configure things on the fly.

Providing you don't oversubscribe the hardware to the users, it's really easy to set up for four GPUs, two GPUs, eight GPUs, whatever it may be, and with a ton of memory. So the possibilities are truly endless. It gives you an awful lot of flexibility when you start doing this sort of workstation, whether it's in the data center or it's at the desk.

So the best in class remote protocols, I've just listed a few on here. There are lots, TGX. There's a high performance, high resolution, remote desktop, supports collaboration, things like Wacom tablets, 3D mouse. It's very responsive. So it's one that we really like.

Parsec is a great protocol. It came from a gaming background. But they've really worked hard and developed some awesome enterprise level support. And it's a great tool. AnyWare is owned by HP now. And AnyWare was rebranded to Teradici. So HP acquired Teradici PCoIP. And that has now become part of the AnyWare platform.

There's good old trusty RVP. I'm sure many of you use RVP. RVP is a good tool. Obviously, there's no cost to it. So that helps with a lot of customers. It doesn't have all the features that some of the others do. But for a basic remote desktop tool, it does work extremely well.

Splashtop probably has the best integration for Wacom tablets. They came from more of an IT support but, again, did awesome work over the last few years looking at their high-end remote graphics platform for the enterprise as well.

So they've come a long way in that side of their business. And part of that, especially when supporting media and entertainment workflows, is working directly with people like Wacom to integrate with their drivers to reduce latency and just make everything more responsive.

So there are lots of choices when it comes to the protocol. But we do recommend doing your due diligence, testing some of these things out. We are happy to share our insights if you're interested in which ones we think are the best for which particular workflow. But do your due diligence and make sure that you are picking the right protocol for the right user.

You can have a mix and match in your environment. And you can also roll in things like connection brokers and the like, which can be a really useful addition of how you manage your users and how they connect to their systems, when they connect to their systems.

You can create pools of systems for users. You can have dedicated systems. You can manage hybrid environments. So connection brokers can be a really useful addition to the environment as well.

So we got Workstation Computing Solution, WCS, Workstation Computing Solution. It's quite a mouthful I think. I think we probably need to be a bit more creative in our naming somehow.

But you can also spot things like VR. So we do have an example whereby we've got a reference architecture for multi-user collaborative VR sessions. So you can have multiple headsets connecting directly to a virtual machine.

So whether that's wired, it's a tethered virtual reality headset, or it's wireless, as in this case, with either Lenovo products, like our ThinkReality VRX, we're using some tools from folks like our partners at NVIDIA with CloudXR. And obviously, laptops and ThinkPads and everything else are all part of that. But having multiple people interact in virtual reality and especially at a price now that's not completely cost prohibitive like it used to be lots of years ago, but now using almost an enterprise grade headsets, but not much more expensive than the consumer level headsets, we're able to do some really great stuff.

So that Workstation Computing Solution can be flipped. You can turn it into a multi-user collaborative VR environment as well.

And I think that really wraps it up. I think that's about time on the session. If you do have questions, you can reach out to us through the chat if you've got any interest in any of these type of solutions or you're looking for more information on protocols, you're looking for connection brokers, you're looking at how best you virtualize a workstation. How do you put your workstation into the data center?

Anything that we can help you with and support you in your day to day business to help make your power users more productive, to help make these systems easier to manage, and to help you do your jobs and grow your business, then that's why we're here. So please feel free to reach out. Thank you.