Automating the Workflow of Data from Revit with Autodesk Forge Data Exchange

FRODE TORRESDAL: Hello and welcome to my presentation for Autodesk University. I'm here today to talk about automating the workflow of data from Revit and Autodesk Forge Data Exchange. Digital twins are taking industry by storm, and the amount of digital information that needs to be managed is growing exponentially. Finding the right information, managing it, and exchanging between application gets harder and more consuming by the day. 4G data exchange provides a great framework for addressing these issues, and I will show you a couple of cases where we have made proof-of-concept applications.

My name is Frode Torresdal. I'm head of development in information systems. I've been working with BIM for over 20 years, and mostly as a software developer. I'm also a sustainability manager in the company. And if you have any questions related to this presentation, feel free to contact me on my LinkedIn profile. The link is in the front of this presentation.

Norconsult Information System has a mother company, and that's called Norconsult. And consult is Norway's largest multidisciplinary consulting company. Solid expertise in building, transport, renewable energy, water, industry, environment, risk management, architecture, planning, and IT. We are at 20,000 projects each year and have 5,300 employees worldwide. We are definitely largest in Norway, but we are growing in Scandinavia. We have offices around the world.

Our purpose, every day we improve everyday life. It's our promise to everyone, and it defines us who work in our consults. We offer consultancy services for all phases of project, from development of ideas and concepts through the planning, engineering, design, operation, and follow up. We are dedicated professionals who adopt an interdisciplinary approach based on a unique combination of engineering, architectural, and digital expertise. Together with our customers, partners, and colleagues, we develop tomorrow's societies, seeking sustainable solutions in everything we do. We use all the creativity and knowledge that lies in our organization to improve everyday life for everyone.

And at the center of digital expertise within Norconsult, that's Norconsult Information Systems. And that is the daughter company that I work for. We are 250 employees and develop AEC technology for the entire lifecycle of buildings and infrastructure. We have both the internal and external customers. We do a lot of work for Norconsult, but we also sell our application for other companies. We have been an Autodesk Forge developer since the very beginning in 2015. We are also a certified Forge System Integrator.

And before we begin the topic of the class, I will tell a little bit about some of the Forge projects that we have been working on in the past and are working on at the moment as well. We are using Forge as a viewer in our [? WebAPI ?] application that needs a BIM viewer. And of course, we are using the Model Derivative API as well. This image shows easy project, a web-based project collaboration tool with an integrated BIM viewer that eliminates any need for printing. This project won the AEC Excellence Award in 2020.

So before I begin, I will try to give a short explanation of what the Data Exchange is. Now, other classes that goes into much more depth, and my class will focus on where we have used Data Exchange. But from the documentation, the simple explanation is Forge Data Exchange help you unlock your data and give you the flexibility to share the right bits with the right stakeholders at the right time, no matter the app or industry you work in.

And this is a slide that I got from Autodesk that expands it. And you've probably seen it before. So Autodesk created this collaboration tool that supports manage trackable data sharing workflow. As we can see here, part of the model in Revit moves into the cloud automation model. It can then be moved to Autodesk Inventor back into the cloud model and then into third-party apps. Like in this case, it's Rhino 3D.

The purpose is to share the right data with the right person in the right context instead of sharing massive files with minimum control over data noise and who has access to what. Shares protected intellectual property while downstream receivers selectively consume the data they care about in their app of choice. And all the while, it can adjust latency in the user's Cloud Workflow and reduce cloud storage costs. I will call this as an integration platform.

And in my head, a simple explanation is the combination of Google Docs, where you can share and work on the same document simultaneously, and git, which is a version control system. That's a little background on the API. We have been part of the private data Vanguard, which helped for the Forge Data Exchange team. Now this Forge Data Exchange API has moved into a public beta, and we have been working for this quite a while. You should note that this API is still in beta, so some things of what I show here will change.

As I said before, we have two cases we would like to show to you. Use case one, we have Revit for designing the model, and our application is Calcus, that is used for cost estimating, carbon footprint, and lifecycle costs. I will focus on carbon footprint in this class, but this will also work for other data.

The sustainability calculations and management is often done by person that do not use Revit. Also, the models can come from different companies. So the workflow is the modeling is done in Revit. You export on IFC file, import that into the third-party application-- in this case it's Calcus-- and there's generated a report, could be a PDF file, could be an Excel file. And it's sometimes manually plotted back into Revit. or you can have an add-in that loads some XL file into Revit, updates the parameters, and undergo an iterative process of doing this a lot of times. But it's manual workflow, and it's large files, and you have to send files back and forth.

So this is what we have done as a proof of concept using the data for Data Exchange. And I will show you step by step how we have solved this. First we have, this is not part of this test, but we have an add-in Revit that automatically adds classification automatically to all the models based on materials. And the Calcus software is typically used in the early phase of the project, and there's not enough lot of information in the model to get a precise CO2 equivalent.

So here you see that the classification has been placed on this wall. The next step is to create a Data Exchange. This is the Data Exchange add-in that is provided by Autodesk that makes it possible to take a part of the model or the entire model and make an exchange of it. This exchange can also be created from ACC or using the API as well. I have used the Data Exchange add-in to do it in this case. But if I log into Autodesk Construction Cloud, I will still see the model here if I select the right project, and I can see the exchange.

The test app that I have made is a dot NET Core backend app. I'm a originally an add-in developer, so I know C-sharp, so that was what was easiest for me. And I have angular front end. And of course, we use the Autodesk Forge SDKs and APIs. There are also NuGet packages for C-sharp to use for some of the Forge SDKs.

So here I'm going to give you a little demonstration of the application that we have made. First we will log in, then we will select the exchange. Then we will review the material takeoff from the model in this web application, and then we will use the application to set the CO2 equivalence and also view the model in the Forge viewer.

So first we log in. I have set this application to all the permissions. That is not needed, but I just did that. You have to select the hub and then the project that is in the hub. Select the folder that your exchanges are in, and then finally, select the exchange. And then if you load the assets from this exchange and show them in the material takeoff.

And here we see the classification, the quantity, and the type name. And here you see this is a part of the application, a database list application where we can select and specify the classification and get the CO2 equivalent. And you see down here, it's, again, an easy project that's integrated into the application with the model.

And now it's time for some code. So the first thing we do is to authenticate, and this is a standard Forge functionality. You create the app in the Forge portal, you get the client ID, you get the client secret, and you use the standard functionality to get an access token. This is well-documented on the Forge website, so I will not go into very much detail there.

And then when you have the access token, it's very easy to get all the hubs for the logged-in person. It's just following this request there with hubs, and we get a list of all the hubs that are available. When you select the hub, you get the hub ID, and using that to get the projects. And when you have the projects, you can use the hub ID and the project ID to get the folders. In this case, I have just used the top folders to get the flat list. But to make this on a real application, you would implement a tree structure.

And when you have the folder ID, you use the project ID together with the folder ID to get the old exchanges in that folder. And then we would like to have this list here, the material takeoff. What you see in green here is collected from the model, and what is in blue is what we set with our application. So you can manually set the CO2 equivalents, or you can use a selection tool here based on a classification and a database in this application to get the correct CO2 equivalent. And what you see here is different concrete classes, qualities of concrete.

So how do you get an exchange? Well, I've shown you, when you select the exchange, you get the JSON response. And from that, you can retrieve the URN of the exchange file. And then you use this function here to get the exchange response. You can get more than one, but in this case, I just get the exchange from one URN.

And the response we get back looks like this. And the important things here are the exchange ID-- that is the idea of the exchange container-- and the collection ID. That's the ID of the collection where the exchange data is stored. We use this in the next call to download assets.

Downloading any assets, as we see here, it's the collection ID and the exchange ID. And this can be a huge amount of data. As a default, it will download 100 at a time. So you have to iterate through it to get all the assets. I will come back to a better way of doing this at a later time.

And the response we get back is, as I said, very large. This is a very small snip of it, but it gets all the connections between the assets, the same as families, family types, and instances in Revit. And it also gets all the attributes on types and instance assets. So as we can see here, the first with red marked out, it's the volume parameter of a wall, a selected wall. And the one at the bottom that's highlighted in red, that is my shared parameter. That's called classification. We see that both of them are not very human-friendly, readable name, but my shared parameter, it's not possible to know all of this.

So to help with that, there is a property schema that you can call. So here, if I call this requester to get the host volume computed, I will get this response here. And as you can see, here I get the name of this parameter. And the same will work for my custom parameter.

And also, as we see in the material takeoff, I had the type names, and I added all the volumes for the type. That is also something that you have to do an extra call for. And in this case, it is the relationship you get, and you will get an asset from ID and an asset to ID and will use that to find out what's the type of the instance assets.

So the goal of the test app is to set the CO2 equivalent for each type. And as we see, I can get the type, quantity, and the classification using the Forge Data Exchange API. And based on the classification, I can use the database of the app to select a specification from a list and then set the CO2 equivalent. Or, as I've shown you before, the CO2 equivalent can be set manually. We could very well add much more data here, like the price, but as I said, this app is intentionally made simple.

So writing the data back, the property data back into the model is not yet ready in the Forge Data Exchange. API. So I had to write an add-in that gets the data from my test app's database and then adds that back. So it's a very simple demonstration of how it is. It's an add-in with a button.

So here we see that this classification, the CO2 [INAUDIBLE] parameter is empty. I update, click on my add in, and it updates this for all the elements in the model. And then you can go back, you can publish to do more work here. Publish it back into the exchange, use it in easy calculus, and import it back here.

So the round trip is-- and the iteration is made much more easier without file exchanges. So this is the case that we had. IFC export, IFC import, manual making reports, manually adding data. And with this proof of concept, we have this case here where the data goes back and forth through the cloud information model using Forge data exchanges.

Case number two, it's called-- I call it structural analysis. It's a little bit more complex because there are three applications there. You are doing the modeling in Revit. The structural analysis can also be done in Revit using robots, but very often applications-- other applications are used, and also the engineers like to calculate the different software to be sure about the vessels.

So the other softwares in this case is [INAUDIBLE] design and [? ISY ?] design. And in this case, very often, the model is redesigned. It's not-- sometimes it's exported from Revit with IFC and imported back into the third party application. But very often, it is modeled from scratch.

And also there are applications or add-ins to Revit to export XML files, support them. But still, it's files or manual work. The proof of concept app in this case is not the same web app. This is a rather simple C-sharp minor line that I have made to communicate between Revit using the Forge state exchange API under two other applications.

So here we use Revit and a Revit analytical model. The analytical model is a simpler model than the geometry model. Walls are represented by planes. My columns are represented by just lines. So it's a much more simple model. But we can still make our data exchange from this model. There are some things that lacks in the Forge data exchange API because the lines are not there today, but they hopefully will come very soon.

And then we can use this SDK here to download the geometry as a STEP file. And this SDK is available as a new package, so it's very simple to install that into your application if it's a C-sharp application. And it's just one line with code to call it.

If you don't have the exchange file URL that I showed you earlier, you have to get that in the same way. Managers download geometry has step. And then you get a STEP file that you can get the geometry from.

So a little bit about the applications here. It's beyond the scope to go into the details about it. But FEM design, that's used as the method to calculate forces and force loads in the model. And as I said, very often the engineer would redraw the entire model here instead of importing it.

The other one is one of our applications. It's called ISY design. It's used for code check, statics, and the design of structures according to your code or some other codes. And it calculates the size of foundations and creates structural reinforcement. Also we had a possibility to use the geotechnical analysis.

And if the results from these two applications that we would like to use to create foundations in Revit underneath the walls, like we see here. So the C-sharp-- the command line application that I made, it's nothing to show really. You just run it. And then you do the calculations in the two other applications and the results are saved back into the database of the C-sharp command line application.

And then we have an add-in here as well, to write the data back in to Revit. And this is, as you can see, the walls in the basement here. When I run the add-in, the foundation of this walls are thematically created. And this is just as a proof of concept.

So in the future when we implement this in our application, we would add structural reinforcement as well. So all this is done automatically based on the analysis and other applications.

So to recap this, again, we are Revit. You might do some analysis in robot, but you do redraw or you import IFC files or do something with other files like XML into FEM design and/or ISY design. We had proof of concept. We have everything through the cloud information model using the Forge data exchange.

One thing I haven't tested that is quite-- that will be very useful, the amount of data can be enormous. So to use something called GraphQL, that is something that is supported in [? Forge ?] [? Data ?] [? Exchange. ?] So GraphQL is query language for APIs and a runtime for fulfilling those queries with your existing data.

GraphQL provides a completely understandable description of the data and your API. Gives clients the power to ask for exactly what they need and nothing more. And to expand it a little bit more, you have these three issues here that you would like to solve. It's under fetching, and at times, we might find ourselves having to fetch from multiple endpoints to retrieve all the data needed for one view.

With rest, you would have to make multiple requests as we see that I did. And then handle and use the retrieved data as needed. With GraphQL, we can make multiple trips to the server with one request, in which we have all the data we need in one query. And also we have the problem of overfetching.

At times, we might find ourselves needing only a small part of the data returned by request. Fetching a full chunk of data to only use a tiny bit of it it's just inefficient and can slow down the application, especially cost issues for people with bandwidth issues. With GraphQL, we can include only the part we want to query and avoid retrieving un-needed data.

And the last one is slow front end development. It's common that given certain issues, we might make a new endpoint on the back end to simplify attached request specific view and avoid under or over fetching, but that requires some work. With GraphQL, the client has the power to fetch the data needed and nothing more.

Nothing slows down and no needed endpoint is needed. It's very simple to add a new field to the query. So this is certainly something that I am going to investigate, at least on the large model, the data you get from the data exchange API, it is huge. So using this type of query language is a huge asset.

And finally, I would like to end with our little wish list of what we would like to wish for the future development of Forge data exchange. In the start, it was very frustrating using the data API, at least in the very early phase, and many things did not work as we thought it would. It's not getting much better, but still, there are some things I still wish that we could have in this API.

One is to write parametric values back to the model, like I shown for the carbon footprint to get the CO2 equivalent back. It would be a huge thing for us. And also get all the geometry, like we saw that in the analytical model. You could get the walls, but we could not get the lines that represent the columns.

And also the possibility to generate a change geometry in the model. I know that this probably is very difficult to do, but that is something that we would like in the future. We could then not need an Adam, but could to get to make the foundations. But we could make them directly from-- through the Forge data exchange.

Also to create my own assets, like the results from the structural analysis application. If I could have that as my own asset, that would be of high value to us, and also more high level functions, like I show from-- for when we get the schemas for the parameters. That could be a much more simple than calling things back and forth.

But it has been a very nice journey investigating this API, and I am very thrilled to take it even further. But that is all I have today. So thank you for listening. Goodbye.