The New and Expanded Forge Design Automation API

ALBERT SZILVASY: I think I'm going to start. Thank you for coming. My name is Albert Szilvasy. And this is Scott, and that's Andrew, and we're from three different teams at Autodesk. I'm from the AutoCAD team. Scott is from the Revit team, and Andrew is from the Inventor team. And today we're going to talk about the design automation API.

How many of you have used design automation for AutoCAD as it is now in Forge? A couple of you, great. How many of you are have any add-ins or have programming experience with any of these three products? So most of you, great. And so how many are AutoCAD developers? OK, how many are Revit? And how many are Inventor?

OK, so there are people who raised their hands for all three, so that's good. Good mix. So that means, probably, most of you are .NET, C# programmers, is that-- who does not .NET? OK. Who is on C++? OK, a few. OK, Lisp? Great. This was good.

OK, so this was just for me so that we can orient the presentation to use the right language. OK, so disclaimer-- everybody has to read, and there will be a test at the end. This is just because we talk about stuff that is future-looking, and if somebody reads this presentation online, we don't want them to get the wrong idea. But yeah, hopefully all of this will come to pass, but I can't promise.

OK, so the agenda today is just I'd like to introduce the v3 version of this design automation API, talk about design automation in general as it is today and how it evolved into v3, and then Scott and Andy will talk about Revit and Inventor, and hopefully show some demos as well. OK, so the first question is, since most of you have never used design automation, is why would you want to use it to begin with?

I mean, what the heck is this? So essentially, it's the ability to run these products remotely. So as it is today, you can run AutoCAD remotely on an Autodesk server-- it's called the cloud-- and do almost everything that you can do on your desktop, as long as it doesn't take interactive input. So you can do data extraction. You can generate new drawings. You can query drawings.

So existing AutoCAD customers, AutoCAD Design Automation customers use it, for example, to generate permitting drawing for solar installations, stuff like that. So the possibilities are endless as long as, again, it doesn't take interactive user input. It's all batch processing.

And what we've tried to do is since we're running, essentially, the desktop code base-- a modified desktop code base-- on our servers, your investment in plugins or Lisp code or whatever you did in the past to customize these products pays off, in a sense, that you can reuse that stuff. As long as, again, it doesn't try to show a modal dialog or something that clearly, on the server, nobody's there to click on it. So it has to be without user interaction. But we've tried to make sure that you can carry forward your existing investment.

There are some seats, if you'd like to sit down. No? OK. That's fine. Yeah, you're sitting all day, so sometimes you do need to stand. OK, let's see if we can progress. OK, so the v2 API, as it is today in production that you can use, is basically the simple DWG processing service. And it's simple because all it is is just it takes some input-- URLs, links on internet-- and processes those, and then uploads that result to somewhere else-- some links again.

And the nice thing about it is it's a service, so it's consumption-based. You pay as you go. There's no upfront licensing. There's no hardware to maintain. There's no extra cost related to having that software on your machines.

And for AutoCAD, they've been in production since November 2014. And it's a pretty big volume service. We've processed about 60,000 work items a day, and it adds up to millions in a year if you do the math. So this is the infrastructure that we're trying to replicate, now, for Revit and Inventor.

So with that, this is the announcement that we've made yesterday, actually, at the Forge DevCon, and I'm just repeating it-- that we are now going to do this very same thing that we've been doing for AutoCAD for a number of years. We're now going to allow this same batch processing capability for Revit, Inventor, and Max. And for these new products, we are in a private invitation beta, so you can talk to us after this presentation if you'd like to sign up. And we have PMs for the products in the back, and these engineers here, so as long as you talk to one of us, you'll be fine.

All right, so I'd like to-- very few of you actually use this stuff before. I'd like to give you a framing of what the design automation API is. So there are three players in this-- well, four, really, but since you are a developer, three players here. You are the developer on the far left. Then there is the data in the middle, and then there is the Forge service-- the design automation service-- on the far right.

And then there's that cloud thing-- whatever that means. The point is that it's not om your machines. And so the way this works is that the first step that you have to do is make your data available in the cloud. What does that mean?

That basically just means that you have to be able to produce an URL-- an HTTP or HTTPS URL for your data. So clearly, if your file is on a file server or behind a firewall, then it will take some durations to generate that URL. And you will either have to upload it to BIM 360 or Fusion 360, or put it in Dropbox or G Drive, or use some sort of service bus to generate that URL-- whatever it takes. You've got to create an URL, because it has to be available to be our HTTP. But it's up to you how you make it happen. We are agnostic about it.

And so then you send us that link. You can generate an [? URL, ?] so you can send us that link. And of course, that link can contain all sorts of permissions. And we're actually not-- this is just the link. What you send us, what we call is a pair-- or a triple-- of how you-- what's the URL? What headers we should use to access that URL-- an authorization header, or that sort of stuff. And what HTTP verb we should use to access that URL-- so get, put, post, that sort of stuff.

And, OK, maybe I should have asked, at the very beginning, how many of you are familiar with the HTTP terminology of URLs, verbs, and headers? OK, that's good. OK, so I'm not talking completely in the vacuum here.

All right, so once you pass us these links-- so again, URLs and headers and a verb-- we use that to download the files or input files to the processing VM that runs the product-- Revit or Inventor or AutoCAD-- and run through the script or the plugin that you requested us to run through this input data. And finally, we upload the results. So it's a very simple processing model. Just take some inputs from some URL, and then upload that output to some URL. And we can do this for all three processing engines.

And so now the rest of the presentation is just going to be about-- at least my part-- is just how do you describe the inputs? How do you describe the processing instructions? And then so on. So this is the important slide that you've got to remember. This is the mental model that you have to keep in mind.

OK, so the API itself has three or four endpoints-- HTTP endpoints. So this is an HTTP API. So this is not C# APIs and a Lisp API. You're working with HTTP requests. So you use a browser or some command line utility, or whatever you want to make these HTTP requests. And so these are the four endpoints to do your business.

And to describe what these are, I'm going to try to use parallels that you may have encountered before. So if you are a programmer-- like a C++ programmer or C# programmer-- you can relate to work items as function calls. So in the service of work items, essentially like a function call. Or if you are more familiar with the products, you can imagine that essentially, each work item will result in a session of that product. We run it, and that's all. So it's an execution of that product.

And if you're familiar with v2-- the two of you that, I think, used the v2 API-- well, essentially, the endpoints are very much the same. We just changed the casing a little bit, and some other things about that, but the concepts haven't changed. So if you're familiar with v2, the v3 API's going to look very familiar.

So work items are function calls. What we call activities are essentially function definitions. And then there are apps, which you can think of as shared libraries-- like plugins, in case of a product. And then we call engines as, basically-- in product terms, that's Revit and Inventor and AutoCAD and Max. So hopefully that's going to orient you.

And so now, once we have this mental model of a function that sort of work item is-- so as you can see, that f function-- it takes some URLs. Those are some links-- those HTTPS, x, y, z, and whatever-- and produces some result. So that's really the programming model of the design automation API-- that you essentially, when you submit a work item, invoke some function, f-- and we'll talk about how you define that function f-- and then we just run through some processing steps that f defines, and we produce some output. That's all.

And what is important here is that we don't retain your input. We didn't retain output. This is purely a transactional process. There's nothing retained on our servers. So if you don't provide storage for some output, all that output will be lost, OK? So we don't do any storage in this service.

Of course, you can use other Forge APIs to store your stuff. So you can use the OSS API, you can use the Data Management API, or you can use other storage services. So we are, again, very agnostic about it.

OK, and so how do you define this f-- this f function? Remember, the work item is the function call. The function definition is what we call an activity. So I've generalized the f function, because in our case, the whole thing can have multiple inputs-- the p1, p2, p3-- and it can have multiple outputs-- r1, r2, r3-- and then there are these instructions. And so we're now going to define-- I'm trying to define, in our API, how do you define this function?

So in case of the input parameters of the activity, or of the function, you can define them the same way as in case of a regular C++ or C# function. You have to give it a name, and that's it. And there is no need to specify a type, because everything is an URL. So all inputs and all outputs are also-- there's no other type of input.

Similarly, the outputs-- they are essentially, in our case, are just named files, because the outputs are generated on a virtual machine that is processing your request, and then those named files, then, can be uploaded to some output URLs. And then the instructions are some commands.

In case of AutoCAD, you can actually specify a script file-- an AutoCAD script. And those are the processing instructions in case of Inventor or Revit. The processing instructions are embedded in a plugin. They don't have the concept of a script. And Scott and Andy will talk about it a little more.

And so the activity itself is just a sequence of instructions in case if AutoCAD is-- like I said, it's a script. Then there are two other things that come into play when you are defining this f function.

There is the engine. You have to choose which product you're going to use. So every time you create an activity, you will say, OK, this activity's for AutoCAD. This activity's for Revit. This activity's for Inventor, or Max. And I'll show you. Or there is another concept, which we call the apps, which are basically plugins, like I said before.

So each activity can reference one or more plugins, and then you can use the functionality inside of those plugins to do your script in AutoCAD, or to do what you want in Revit and Inventor. So that's where you can reuse your legacy code. As long as your legacy code doesn't do GUI, doesn't do anything that interacts with the user, you'll be able to reuse it-- or you have to [? refactorate ?] to be able to reuse it most of the time. But you get the point-- that it's not a completely new development that you have to carry out here.

And just one more note on how you should think of activities. So like I said, activities are functions. So you create it once, and we store it in the system. So you create this function once, and then you can submit work items that reference this activity again and again.

So usually, the way we see it used is that some developer creates this activity, and then their production service just goes and submits work items against it. So they use the activity APIs and the app's API very rarely, because they only have to do it once as they set up that. It's almost like when you're developing a plugin, you develop the plugin once, but the customer will run it a million times. And it's the same thing here.

Apps are your plugins, and they can also contain any sort of content if that is what you would like to do. So for example, your block library or something like that, activity wants to reuse again and again in multiple work items. So that's actually a pattern that we've seen with some customers.

And again, you only do it once-- or rarely-- as you update your code. The most frequently used API is going to be to submit a work item. And of course, you don't create engines. We do that, and we update new versions of it, and you reference it.

OK, so I referred to this a few times already-- of how this works behind the scenes. It's very, very simple-- and how we go and, for each work item, it goes into a queue. When you submit a work [? item, ?] it goes into a queue.

We go get this item in what we call a job object from that queue. We have a ton of processing of VMs. They get it from the queue-- one of them-- and goes, downloads the inputs that are specified in that job, then runs the engine-- whichever it is, Revit or Inventor or Max or AutoCAD-- and then produces some outputs, and then we upload the outputs. So it's as simple as it can get. Nothing complicated happens here.

OK, so I'd like to talk about the new features that are relative to v2. So like I said, we have v2 API in production today. That only works for AutoCAD. And what we're doing is that we're updating that API.

Get rid of some of the AutoCadisms that are currently there, because we want to use it for Revit and Inventor and Max as well, and build in some of the learnings that we have had over the years. So these are the new features that-- and probably there are more fewer or smaller ones, but these are the highlights.

So we simplify the JSON payload. So this is the v2 payload. I don't know if it can be-- can you read this from the back? OK, so the activity ID. This is the work item.

So I'm submitting the work item, and so I have to specify which activity I'm calling-- which function, the f. It's called plot to PDF. That's the activity ID. And then you specify input and output arguments. In this case, it's some DWG file that we're going to plot, and then the result is some S3 presigned URL. And I didn't want to put it there because those are so long that it would have scrolled off the screen.

But anyways, that's not the point. The point is that in v3, we have simplified this a little bit so that the activity ID is now this three-part thing that I'm going to talk about in a moment. We call it an alias. And the arguments-- instead of specifying, oh, it's input argument or output argument, we inferred that from the verb.

So if there's no verb-- the first host DWG argument has no verb, so that implies that it's a get. So that will get input argument. And the second one, called result, has a verb called put. That's an output URL, because then we will do a put request on that URL that we specify.

Then in v2, it was quirky of how you could pass variables around in the activity. So we made it more explicit. Is this still readable? So basically, you can use this syntax of a dollar sign, and then object.property to refer to various things within-- mostly in the command line.

So when you're specifying an activity, the most important thing is, hey, what's the command line that I'm going to run on the server? So what are the parameters of that command line, and what [INAUDIBLE] am I running to begin with? So the command line-- so this is an activity definition for the plot PDF activity that we just saw that call to on the previous slide.

So this is the definition of that function-- the plot to PDF function. So I told you that you have to specify which engine this is going to run against. Well, there's the AutoCAD plus 22. Well, I'll tell you what that means.

It's clearly something to do with AutoCAD. We don't know what that 22 is, but I'll tell you. Then there is a script at the bottom of the screen there that's-- [INAUDIBLE] well, we're doing a [INAUDIBLE] whatever. But the important part is that command line on the very top.

So what we do is that there are these variables-- engine.path and [? args ?] [? hostjoin.path ?] and all that stuff. So those reference various pieces of information within this activity. So for example, the engine-- so when we process this job-- so when the activity sits on our server, but you submit a work item against it, and then all this goes into your queue-- we put the two together, and we put it into the queue.

And then it arrives to your processing VM, then the processing VM looks at it and is like, OK, what command should we run? That's the command line-- and expends those variables. So the engine.path will expand to wherever the Autodesk, that AutoCAD, plus 22 engineers on that VM-- so something like [? C ?] AutoCAD something. They just have an installation of that AutoCAD on that machine.

And then the [? args ?] [? hostdwg.path-- ?] well, that expands to the local path of the argument that was downloaded to that VM. So that's going to be the second input to that Core Console. I mean, how many of you are familiar with Core Console? That's an AutoCAD thing.

OK, so it's an AutoCAD console application that ships with AutoCAD and has these various command line switches. And [? eye ?] [? switch ?] specifies the input file. And so then the third variable reference references the settings thing within the activity, and the one that is called script, and then it just specifies that it's the path property. That means that we will save the stacks-- [? style mode ?] 0 space export, blah, blah, blah-- into a file and give the Core Console the path to that file.

So makes it more-- so this was all happening undercover in v2 as well, but it's made more explicit, now, with these variable expansions and makes it more flexible, as well, for being able to specify different command lines for Inventor and Revit. As you can imagine, that's why we did this.

OK, so I talked about that AutoCAD.PlotToPdf+Prod, and then all these other long names that we use. They are called aliases, OK? So the problem that we're trying to solve here is that as you evolve your activity or app-- let's say you created the first version of your app. You upload it to the service, and somebody starts submitting work items that use that app.

Now, if that work item uses an explicit version, then every time you update your code, they need to be notified to use the new version. And that's not very convenient, so we added an extra level of interaction-- this recall aliases. If you think of it in source control systems, this is like a label, where you can move the label from one version to another.

That's the same thing that we're doing here. Instead of referencing a specific version-- so AutoCAD, we have three version of the PlotToPdf activity currently in the system-- one version 1, 2, and 3. And instead of specifying, in your work item, the production work item that I'm referencing the first version, you're going to be referencing a name for it called Prod. And then you can keep on testing, let the production users use the Prod version while you start using the beta version and can release that to a set of customers who will be beta testing that next version of your app.

And then when you feel like, OK, we are ready, then you can go and switch the alias, and now it points to-- the prod points to version 2, and the work items that the production users are submitting-- they are just referencing this prod, this indirect name, and now they're using the new version. So this makes it easier to update your test code. And you can roll it back if it turns out that oh, it didn't work out. You can switch the alias back to point to the first version. So you can go back and forth.

OK, and we also support sharing. In v2, we also supported public thing. Now we try to expand that with explicitly sharing it to another user. So in Forge, you have these Forge client IDs. Every time you make a request, you're making it on behalf-- your identity is that client ID.

And so now, when you create an activity or app, you can share it with everybody who uses Forge Design Automation, or you can specify a client ID and share it with that user. Currently, we only allow one. Definitely see the need to expand that, but we'd like to get some feedback on that.

So when you specify the receiver in red, there, that's the client ID. You copy that from the Forge portal. How many of you have used the Forge portal apps there? So there are a few of you.

Anyways, you go there. There is a list of your apps. You click on one of your apps. Then you can see the client ID of that app and the client secret for that. But this is the client ID.

It's very ugly and long. And for that reason, in our case, since we use these client IDs for sharing and as the owner of activities and aliases-- and so when you share something, you would like to be able to share it with a nice name, because that's the first part of the alias-- is the name of the owner. So what you saw before is that I was specifying AutoCAD.PlotToPdf.

Well, hold on, where does that AutoCAD come from? Well, that's a nickname. So we create these nicknames where you can specify hey, my client ID inside the design automation system should be called AutoCAD. And then any activity or any app that you create will have that namespace of it, or the owner of it will be AutoCAD.

And so for you, maybe your company name or something like that would be an appropriate thing. And so what you have to remember is that you can only do this before you have any resources, because once you have resources, you cannot change your client ID, because then the owner changes, and we don't allow that.

OK, then there's also canceling and progress, [INAUDIBLE], or built-in support for progress in v3. So you can now cancel a work item. In case of AutoCAD, has never been a big deal, because AutoCAD work items usually run, at most, a few minutes. But it turns out that Revit and Inventor, or Max-- they can run for many minutes-- like tens of minutes-- or hours. So OK, so you need to have cancel.

So we have two ways-- allow you to cancel two ways. You can go and explicitly-- here's the work item ID, workitems/(id), and now you'd make a delete request on that, and I will cancel the work item. It's not immediate, but within a minute, it'll be canceled. And then there's another way, where you can-- in the work item that you specify, you can use two built-in names.

So there is two built-in arguments. One is called onProgress. The other one is called onComplete. You can use one or the other, or both. So if you want to, let's say, submit a PlotToPdf work item and you add this extra parameter to it, or the extra argument to it called onProgress, and you specify an URL there, then as we start plotting, we will call this URL every minute to see if we should proceed.

If you return 2 or 5, we will cancel the request. This is very similar to what-- I don't know if any of you were in the keynote yesterday-- the Forge keynote yesterday-- but they were talking about [INAUDIBLE] [? hooks. ?] These are the same concept. It's exactly the same thing. [INAUDIBLE] callback URLs.

And then similarly, we also have an onComplete callback, which is the end of the work item, with post-JSON payload to you that tells you how long the work item took, whether it succeeded, where the detailed report is, and then so on. This is much better than polling, obviously, because then you don't have to dedicate a thread to find out, OK, has the work item completed? We just call you back. But you have to have a web service to implement that URL. So that's the generic overview part of the presentation, and now I think Scott will carry on with some Revit-specific comments.

And I'll-- [AUDIO OUT]

SCOTT: I'll be doing-- [AUDIO OUT]

ANDREW: So can you guys hear me? OK, great. So what does that look like for Inventor? It's going to look really similar to what Scott was showing for Revit. That's kind of on purpose.

The API is the same. The workflows, actually, are pretty similar, too. So as Scott mentioned, they had three for Revit. We have three primary workflows that we're targeting for Inventor as well.

So we want to be able to provide configuration so that you can generate some output. We want to provide a way to do export. So what you might think of is a custom translation. And then be able to generate so you can actually interact with the model and programmatically do things via the API in the cloud.

So with those three primary workflows, what does that look like? So with configure, that's typically either an engineering automation case, where you're generating output for manufacturing, or a sales automation tool where you're generating some kind of output for whether it's pre- or post-sales. You want to be able to generate the artifacts, or the costing for that.

For export, you want to export, possibly, to other CAD formats, export drawings, export or modify the BOM, and then any kind of custom interop, as well, where you need to generate some kind of output to feed into a system.

And then on the generate side, you can generate design alternatives. You can generate product families or bucket of parts. You can do some model validation. So similar to like what Scott was mentioning with the overnight processing, you could also run a process to actually validate your design in the cloud. And then things that you might have done with Task Scheduler in the past, you can actually do bulk operations in the cloud as well. So those are just the three we're targeting.

Similar to Revit, we provide access to the Inventor Server API, which is basically no UI, but full Inventor API. So that API's available in the cloud. So we'd actually like some feedback specifically on these three workflows, but if there are other workflows as well, we'd really love to hear those.

So our fine print is really similar to the Revit fine print. So we are going to have the ability to run iLogic scripts as well. It's on our backlog today. So any of you using v2, that's something we're looking at for v3.

Albert sort of lied a little bit. We do take scripts. We do allow you to run iLogic. But typically, we'll also allow you to run your plugin as well. And then just the other thing, here, is full Excel access. We do provide read, we just don't provide write at this time. So that's another little thing you might need to look at if you have Excel really embedded in your process.

So we got really creative-- Inventor on Forge. So we don't have a site like Scott and [? Sasha ?] set up, but feel free to reach out. We're in private beta right now as well, and we are looking to expand our current private beta depending on the workflows, and to really gear us towards that public beta.

So we worked with coolOrange, who had an existing Inventor add-in to take-- Inventor has cosmetic threads. But if you want to actually have an output file-- let's say you want to print your threads-- you actually need those threads modeled. So coolOrange had an add-in that would take an Inventor model and actually model the threads, and then provide an output. So what we did is we took that add-in, and we used Forge and made a cloud application to upload a model and output that model with actual threads, and not just the cosmetic threads.

So see, here you're in Inventor. You have the cosmetic threads. There's no feature. So what we're going to do is, on the website here, you're going to pick your part file. We're going to upload that part file. And you'll see here, since the current Large Model Viewer doesn't support decals for Inventor, you'll actually see here that you won't see any threads at all. If it's for decals, it would just be the cosmetic thread here.

And then we're processing that job in the cloud, running that add-in, and generating threads on that model. So these are actually modeled threads, all done in the cloud without needing Inventor on your desktop. And then you can actually download that file as well, and we'll bring that back just to show that we actually do have, now, featured threads in Inventor. And that was all done taking an existing add-in, turning it into a plugin, and using design automation to drive that workflow.

So where do we go for more info? So certainly, start at developer.autodesk.com. Yeah?

AUDIENCE: You need the source code in order to create that pattern?

ANDREW: Do you need the source code for the add-in?

AUDIENCE: From the add-in in order to create the plugin [INAUDIBLE].

ANDREW: No, you don't. If you have some questions, I'd love to go over that with you a little more. You actually just upload the binary for the DLL, basically, for that add-in. And then there are some related DevCon and AU classes as well. Revit specifically-- the one that Scott mentioned-- 8:00 AM tomorrow. So that's a great reason to get up early. And I think, with that, we're at our Q&A time. So yeah?

AUDIENCE: Question for Scott. For data extraction from Revit, how is it different from a Model Derivative API [INAUDIBLE]?

SCOTT: So eventually, whatever the Model Derivative API-- whatever it supports today, we will be able to access through the Design Automation API. But today, with the Model Derivative API, you basically are just getting parameters for all of the elements, and you're getting the graphics. So you're not getting geometry. You don't have any access to adding additional logic to whatever you want to go ahead and extract.

So if you wanted to create your own extraction and you wanted something that's currently missing from the extraction, like if you want to figure out space layout or something like [INAUDIBLE], you can use the Design Automation API to get that because you'll have full access to the [? Revit ?] API when you do the extraction. So that's something that we need to look at in terms of actually allowing you to embellish a real extraction that the Forge viewer takes a look at. So that's something, down the road, that we'll have access to. But we're going to open this up as much as we can for you guys.

ANDREW: And just to follow up on that a little bit, one of the big differentiators between model derivative and design automation is you get your own chunk of code before you actually do the translation. So there's lots of workflows that we could support with that.

ALBERT SZILVASY: Thank you, everybody. I think we're out of time, and everybody probably wants to get to the keynote, but we're around. Come find us if you have any questions, and use those email aliases and fill out the survey. Thanks, guys.