Elevating Digital Twins: From Reality Capture to Intelligent Operations

LUIS CLEMENTE: Hi, everyone, and welcome to this session for Autodesk University, where we will be covering digital twins and how to use them for intelligent operations. My name is Luis Clemente. I'm Manager for Innovation and Emerging Technologies. And with me is going to be Francisco Tabanera, Director of Digital Assets at Modelical. So let's start.

For today's session, we divided the content in four parts. We will do a very short introduction about intelligence operations and digital twins. And then we will go through all the implementation roadmap, focusing on two very important steps, the two last steps of data integration and how to use dashboards and views for facility management.

So why we need intelligent operations? You will find several studies that will show similar results, that operators are struggling with poor and fragmented data when you do the handover from the construction to the operations. A lot of energy is being wasted in operations due to ineffective asset operations. The space is underutilized, maintenance, inefficiency. So I don't know you, but I see a lot of [INAUDIBLE] here to save some money. So it doesn't matter what is your area, what is your use case. I'm sure that you can find a way to get benefit from a digital twin.

So just a very brief intro about digital twins-- in this context, for us, a digital twin is a digital representation of an existing physical asset. And more focused, we are talking about buildings. It could be even a hotel, office building, residential building. And the bigger the building, probably the bigger the impact that we can have from a digital twin, properly connected.

So there are several levels between digital twins and the level of implementation, how digitized is the application. And here, the important thing is that, the more you're connected, the more value you are going to get from your model and your digital twin. In this session, we are going to move from a descriptive model, which is basically the 3D model of the building, to an informative model, which is basically connecting some sensors and getting some live data from them.

So let's go into the implementation roadmap. We will focus, in this section on existing building. And we will be covering all the steps that we are somehow giving a guidance here for you. So you can do it yourself, and you will see it is not that hard.

I want to mention also that these two paths, they will converge somehow in this as-built model. If you are already for a new building, working from the very beginning in the design level, using some BIM model or any 3D model. So Francho, please.

FRANCISCO TABANERA: Well, we are going to focus on our Modelical headquarters here in Madrid. And we are going to develop several use cases on top of this building, on our offices. First of all, I want to tell to you how we have model the existing conditions model for the [INAUDIBLE] model for our facilities, that it comes with several technologies. Go to the next one, please, Luis.

OK, first of all, we need to capture the reality. We can go with a tape measure to take all the distances, but we have a lot of technologies that can help us to develop the model that we want. It goes from a minor precision to a plus precision to a major precision. We can have 360 cameras. We can use drones with photogrammetry if we are working in exteriors. Or in our case, we have been working with mobile scanning and laser scanners. Next one, please.

Well, this is just a corner of our offices. And I will show you how we have developed the whole building so you can have an idea. All these icons that you have here in orange and blue are the 360 pictures that we have taken. In orange, we have the ones that we have taken at the beginning of the refurbishment project. And in blue, we have the ones that we have taken after the refurbishment. Go to the next one.

So I'm going to focus in these two pictures that are just looking to that corner. And we have developed our own tool in Modelical because we have plenty of existing conditions modeling in the office. So we have developed our own tool that is called Deja Vu, like the Denzel Washington movie, that allows us to open the 360 picture of the right time where we can review the details that we cannot see maybe in a point cloud. Next one.

So we have this corner with the-- next one, please. So we have been working with FARO and Navis to develop the point cloud of this-- of the offices. Next one. And one of the main things that we do with these point clouds is that we use them to check what are the formations of the project. In this case, we can see the slab deformation where we can see some places on the left hand side that are a bit upper in-- they are not flat. And in the middle of the offices, we have some lower depressions on the slab that maybe are important in the future.

After the point cloud, we have model-- go to the next one. We have modeled the existing conditions in Revit. But we have made a slightly refurbishment on the offices. We have removed these-- a couple of walls. We have developed some new partitions. This is the final situation of the refurbishment-- if you go to the next one, please, Luis-- where we can-- we have checked through landscaping render what it looks like before the refurbishment. Next one, please.

And this is the final situation once the refurbishment was fully completed. Next one. Once we have the model, and all the work has been done, what we can do is to move all this information, the Revit model and all the information, to Tandem to keep working with it, with all the information that we need to manage. Next one.

OK, so-- and as build model should be useful for several things, not only for operation and maintenance, but also facility management. And if we plan a really good 3D model that we are going to be doing, we can have a lot of information and a lot of usage from it, like in marketing, doing some AR/VR, virtual tours, and many other things that we can check in the-- a few slides afterwards.

LUIS CLEMENTE: So continuing in our roadmap. The next step is checking the data, right? So what are we going to measure from our digital twin So there are a few steps that we can follow, and we can stop in each of them. One is choosing the right use case, choosing the sensor that we want, what communications are we going to need. And finally, evaluate if you need any system integration. So let's go in detail, each of them.

So for the use case, we are looking on what gives you value. The idea is that every single department, every single area within any company, they will benefit from a digital twin one way or another. So I'm just quickly showing here a very long list of use cases, just as an example. And you can check these in the handout if needed. But the important thing is that you will find many ways to take advantage of a digital twin.

So when it comes to choosing the sensors, in the market nowadays, you will find virtually anything, everything just to measure whatever you want. So I'm providing here a sort of a decision tool for you to understand what kind of device or sensor you are going to need to apply to your use case. And I want to highlight just a trend for these IoT devices and communications on sensors that are battery-powered, with wireless communication, and low cost-- not meaning cheap, just low cost-- and also with low maintenance cost.

So when it comes to communication, in terms of the wireless communication, there are plenty of technologies available for choosing the right one for your use case, you have to consider a couple of things. In first place is the range because we are moving from the very just contact as an NFC up to several kilometers for 5G communications. And you will find everything in between.

And more importantly is the data rate that you will need for your use case because if you are thinking of a small IoT devices that are going to send very-- just bits of information with just a piece of temperature, humidity, or CO2 level, anything that is a very small packet of information, that's how you can use the Zigbee, Wirepas, LoRa, Sigfox solutions that are very convenient for these type of sensors that we mentioned before, the battery-powered.

And of course, there is a trade off in terms of that rate. If you need to send, say, livestream video, you will have, then, much more power consumption. So you will need to define very well the kind of communications that you want.

So finally, in terms of data, you should consider integrating to any existing systems. Nowadays, building management systems are also evolving, kind of in parallel to digital twins. And they are somehow overlapping nowadays. So it's a good practice, or it's advisable, when you are working on a digital twin, also to try to integrate with any existing system or any already connected devices that you have.

So let's go now try to connect this data, so some hands-on about connectivity. And we will be moving a little fast, so please bear with me because a lot of information to cover. So when we want to connect a digital twin, the data is moving in-- the flow is moving from left to right here from the device. It usually, depending on the technology, but this one we're using, you will need a gateway in between, then a broker, and finally, the platform.

For our pilot, we are using this kind of very small and affordable devices. They are quite practical. This is again battery-powered, but the battery can last five to 10 years. So it's a very low-maintenance and wireless communication.

In this case, we use Zigbee communication, which, together with the low rate of data that you need to send, they become very efficient. And that's how the battery lasts a lot. For the gateway, we are using Raspberry Pi, which is like a small PC, with a Zigbee dongle, which is the one that is going to read the communication from the devices.

In the next step for communication, we are using MQTT protocol, which is also a very convenient protocol for your IoT devices. In terms of for MQTT, you will need a broker that we are going to use, an Autodesk Tandem connect for that broker and later communicating, with the platform that you are choosing as Autodesk Tandem for our digital twin.

So in order to implement, everything is not the same order. We have to start deploying the gateway and the devices. So this-- again, using the Raspberry Pi 4, and you can install the Home Assistant operating system there, connect the USB dongle, and pair the device. This is a very straightforward process. We are not going to stop here, because it's not the purpose of this session. But if you need any help, there is a lot of online support, and I will give some links in the handout.

So next step, we should go to Autodesk Tandem Connect to deploy our MQTT broker, which is the one that is going to be managing the data flow from the sensors and the gateway up to the platform. And here, when you enter in the platform in Autodesk Tandem connect, you will just have to create a new broker, give a name, short description, choose the kind of protocol, if it's MQTT or MQTTS.

It will be the port there, 1AA3, in the case of MQTT. And you will have to define the username and password that you will choose. This is something that you define. At this moment, Tandem Connect is in beta, so you will need to request access. And I'd recommend that so you can start using these kind of solutions.

So once you completed all this data, you will get, in return, a URL which is the address for your broker. So that's the server that you are going to be publishing your information from your sensors. If you want some help, also, MQTT Explorer can help you just to check if everything is working, if messages are being published or the subscription is working. And you could use if you have any trouble with this broker also HiveMQ is a very good option for a broker.

So let's go back to Home Assistant. And please, don't be fooled with the name, Home Assistant, because this can be used on very-- plenty of use cases on building level. It's not only just for homes. So in order to connect to the broker, we will bring in this screen, and you will follow this path to add one integration of MQTT within Home Assistant and then configure, completing this information.

So what kind of information we are completing? The broker on the left, right, the first one, is the information that-- the URL that you got from the previous step. You will assign the port, username, and password that you already defined, again, in the previous step.

And just be careful, since we are using MQTT, so we know we don't need any broker certificate validation. So we keep it off. And please be sure, at least at this moment, the broker in Tandem Connect is only accepting MQTT protocol 3.1.1. So keep this in mind because it's important.

So if you click Next, you will know that everything is working already because if not, you will get an error message, and you will have to double-check all the information. On everything else, you can remain as default.

So the second step to connect to the broker, you will have to install the Zigbee2MQTT add-on also in Home Assistant. And then go back there in that path to do the configuration. So that configuration is quite straightforward. You will find a lot of by default information.

And you will have to focus one thing. And let me open one parenthesis here to talk about the topic structure, OK? You will need to define in your own case the right topic structure that will help you to develop your pilot or your use case. For this pilot, we use this structure of Zigbee2MQTT as the base topic, which is because it's from the system.

The building name, the gateway area, and then, finally, the device name, which is divided in two parts is the room name and the sensor type. We will use this later on in the process. So keep it in mind. And one just warning on this process-- please, be careful on having the right word, user, in this step, and not username. So you can save one week of trying to find out what was the error, like it happened to me. [LAUGHS]

So we are going back to Autodesk Tandem Connect. And we create the pipeline. It's a very simple pipeline. It's just, like, three boxes, and we will go in detail in each one of them. So the first one is MQTT Stream. This is the one that's going to listen to the information from the MQTT broker based on the topic that we defined before.

And this is important. You will see there the Zigbee Modelical HQ Floor 3. This is, let's say, the topic that is aiming at or targeting to the a gateway that we develop in that floor. And we will check later about this.

Again, you have there the host. That is the URL that you had defined before. Just be careful here that also using a data type. Double-check, because you have the option to make it raw or JSON. And this could be, depending on the broker you're using, you can have one or the other. So please be careful on that part.

And finally, quick tips-- quick tips is be sure to use application data. And the ID is coming from-- the resource is coming from the data and the device. This is just-- keep it like that. We are not going to go into much detail. And this is all by default. You can keep it like this.

The next box is the converter because we are receiving the information in a format, and we are going to send it in a different format to Tandem. So for this, you will need to create a JavaScript code. Believe me, I'm not a coder. I don't know how to do JavaScript code. But my dear friend, ChatGPT, was able to do it.

But you have to understand very well the instructions that you have to give to the ChatGPT or any other LLM of your preference, what kind of instructions, what things you are getting, and what you want as a result. So for this code, the first thing is creating a filter or whitelist. This is important.

The red part of the topic structure we have there will be used for many, many messages coming from the devices and the gateway, which are not really useful. It's not really a data that we want to measure. So for that, I'm doing some kind of filtering whitelist on only considering those that has the device type or the measure type that we're using to read and transform.

Then you provide the input, how you are getting the information, the format or the template of the output, how you want to send it. Define what is for you, the device ID. And you will give this instruction there. And finally, adding the timestamp, and I will comment this later. And when giving all these instructions, you will have this JavaScript code quite easily. It took me to two times just to have the right one.

So the final box is Autodesk Tandem Connector, which is sending, actually, the information to Tandem, to the platform, to our digital twin. So we are having here just choosing publish stream times. Choose that one.

You will need one client ID and one client secret. So again, let me open one quick parenthesis here. For this client ID and client secret, you will need to go to Autodesk Platform Services, and you will be able to get that one. Just you need click the button, Create Application and choose the traditional web app. And immediately, once you create that one, you will get client credentials, this client ID and client secret. And those values are the ones that you are going to put in your Autodesk Tandem Connector box in these two pieces.

Additionally, you will need the facility ID, which is, if you go to Tandem Connect, to the platform of Tandem-- sorry-- you will see, in the URL, there is a small part of it starting with URN and ending with the next slash. That part is your facility ID. This is the one that you need to copy and paste. You will find much more information on the handout, please. So we will keep going very fast for this.

So in this same menu, the next part, you will have to choose a few things. Most of them are by default. But here, please consider these two things. You need to create the stream. This will create a stream in Tandem every time a new device is added, OK? So it's going to do it automatically. We will see later how it looks.

And at this point, the Add Stream Timestamp Data-- initially, I had that one activated. But the problem is that the format going there, it wasn't in the right format. So that's why I decided to keep it here unchecked. And I added that one in the JavaScript code so that the system, every time the data comes, it will add the timestamp.

So OK, now we need to set up a Tandem to start receiving that data. So let's go to Tandem. We will need to create a parameter for every type of data. We will see one quick video about that. We will need to assign to a given classification-- it could be existing or any specific one for IoT-- those parameters that we want to match to that classification.

And finally, you will need to create some elements to host the connection. Or you can even put the actual sensor, the object model, within your 3D model. And it's going to be the host for your connection where the data is coming.

So let's-- quick video here just to see the process. So the first step, we are creating one new parameter. This one is about lighting, if there is any light coming. It's a light detector. Be sure that you have the category Streams. This is a very important.

So then we already have, in a facility template, one-- a classification called IoT, and there is one subclassification for every type of sensor. So now we are assigning to that classification the parameter that we already defined for this type of sensor of lighting. So that's it. So everything-- now, every time we choose this classification for some object, it will have assigned the parameter of lighting or some other examples, as we will see later.

So at this point, many things are done. [LAUGHS] This is kind of a TikTok mode. It's why we're going very fast. But it takes some time. If everything works, you will find a new connection in Tandem. That's-- with that new connection, we will see one video now, how it works.

You will need to-- that new connection to assign to a proper classification, connect to the host or assign the host, and then map the parameters, or pass the parameters if the names don't match. So let's quickly check here. For this purpose of demo, I used kind of a specific tool within Autodesk Tandem Connect just to send a test stream or a piece of information.

So that's how you will see in Tandem in your Connections tab. There is a new connection available. But you will have to classify it in the Property panels. So this is what we're going to do in this video. Choose the new connection. Assign the right classification. So this is the one that we use. In this case, we are assigning for temperature sensor, which by the way, brings also the humidity. It's going to be a double one.

So now we have to assign to the host. And this is what I said before. We create kind of some fake objects to be the host of every connection that we are giving in those rooms. So that's the host assigned. And last part, you have to configure the connection doing the match between the parameters that are coming, temperature and humidity. This is from the test on the left. And you are matching with the parameters that that classification is waiting for, which are humidity and temperature.

And that's it. That's it, pretty much. This is how you connect a stream of information, a connection into Tandem. And you will have to repeat this process every time for every sensor that you activate or you connect.

So finally, this is-- our model is already connected. And now we will have to move forward into the facility management. We need to start using all this information at some point.

So there is a very interesting thing with Tandem, with this platform for digital twins that it makes much easier for users that are not familiar with BIM models to get value and get information from this digital twin because you don't really need to be an expert on Revit or Navisworks or nothing like that, because this is a cloud platform that you can use very easily with just a little training and understanding where the things are.

And two very useful things that we have are the views, which is the kind of bookmark on the left, and the dashboard, which is the pie chart there. So let's see how we can use them to take advantage of a digital twin for facility management. So as I said, we have probably a full model, which is kind of complex.

And the idea is that we, as a programmer of doing our digital twin, we want to make it easy to the final user to have access to the right information. So in this example, we will create a view that, with only one click, for example, the facility management, the office manager can check what kind-- what the temperature is in all the floors, in this case. So let's start activating some filters. And we are doing this so the user is not-- doesn't have to do it.

So we want just a clear view of our third floor. So all these steps, our filtering and everything, is done only once. Now we are going to add the heat map for the temperature parameters. That is going to take all the values that we have assigned previously. So there you are. And we have this kind of heat map.

And we see these because this is the one that we want to look at. We save that view. It's going to be temperature for the third floor-- not second. It's third floor. And that's it. We have it done. And every time that you want to see exactly this view, you only, with one click on the Saved List, you will be able to reach this view. So you don't really need to do any filtering, anything at all but this view.

So one more example of, OK, let's try to create one dashboard. To create one dashboard, we go to the menu on the right top, create a new one. And we will have to choose a view that we have already created. So we have to do that previously, create a specific view for the dashboard that you are going to create.

So by default, you will get a lot of graph and information based on the parameters of classification, a percentage of parameters completed because this is initially focused on kind of the handover between the construction and the facility management. So here it is. And this is how you create a dashboard. We will select a few real applications of this, of course, and this information.

So one more use case is for the IT management, for example, in the data room temperature. You know that, for a data room you need the AC working very well all the time because otherwise, there could be problems with the servers in the data room. So for that, we can assign thresholds with warning level and alert level. So let's check a quick video here, how we do it.

So we choose the sensor in the data room, right? Which is it? We click the Threshold button, create a new threshold choose what parameter we are measuring. In this case, it's temperature. Give it a name to that threshold-- Data Room Temperature, in this case.

And give a warning level and alert level for 22 degrees and 23 Celsius degrees. Then you have to assign what connection are you going to check with the threshold. And that's it. So every time the temperature crosses that threshold, it will change this color. You see the yellow color.

And that yellow button there is giving you the initial warning that the first level of the threshold was crossed. And it will turn red if the alert threshold is crossed. At this point, we can see this live only, OK? We'll see this alert live. So one pending process that is not going to be part of this presentation of this session-- but I invite you to try to do it also, if you reach up to this point-- is now we need to send an alert to a responsible person or whoever is going to do something if this threshold is activated.

And you can go, also, historically, in the past, and see how was the behavior of that sensor and the stream of data. And you can see also with the colors every time the threshold was crossed. So this is the kind of things that you will be able to see in Tandem. So Francho, please, take for the use case.

FRANCISCO TABANERA: Another use case that we wanted to check in Modelical was if the meeting rooms are being used in a right way or not. What I mean is that we have made a refurbishment in the office. And we want to check if the amount of meeting rooms that we have are enough or not for us.

For this thing, we have-- we want to check two different things. One is the actual occupancy, which is-- if you go to the next one, Luis-- is what Luis has been showing to you. We have a sensor in the meeting rooms that is following all these processes that Luis have been describing to you and will give you the amount of times-- the amount of time that someone is inside the room.

And on the other hand, we have the parameter occupancy, which is we are working with Google. So we have the Google Calendar. We have access to the meeting room calendar. We are pushing it to Google Sheets. And with a webhook, we are mimicking or-- yeah, mimicking a sensor, let's say, for the meeting room. Could you go to the next one, please.

So that's how we have been doing this, is that we have taken the ID of the meeting room, and we are pushing all the information of the users for that meeting room to Google Sheets. As you can see, on the top left-- top right corner, each line is every time we have a meeting on the calendar for this meeting room.

So for instance, the last one, Pelican Brief meeting room-- oops, go to the previous one, sorry-- it has been used on July 16 from 4:00 to 5:00 PM by four invitees for an hour. That's the actual-- the things that have been put on the calendar. On the other side-- well, sorry, on the bottom right table, we have the amount of times that every room has been used. So all of this is data information that we can action in order to see if we need more meeting rooms or less meeting rooms. Go to the next one, please.

This is just the code as it will be on the handout on how to send data not from a sensor, but from Google Sheets to Tandem. So we just need to copy the URL for the virtual sensor that we have in our Tandem model and just send the information from Google Sheets to that virtual sensor. Go to the next one, please.

What is the information that we have? Here, we can see the seven meeting rooms that we have. And if you go to the next one, please, we have two different information, two different data, that is coming for each room. The occupancy, which comes from the Google Calendar, the amount of time that a meeting room is booked. And the occupied is the amount of time that someone is going in that meeting room.

If you go to the next one, one thing that is a brand new characteristic that we have in Tandem is that we can over-post two different information in a single graph. For instance, we can put together the occupancy and the occupied information. If you'll go to the next one, please.

And with this over-post, we can see in red, which is the amount of times that the meeting room has been booked, and in blue, the amount of time that someone has entered to the meeting room, maybe to attend a phone call or to-- whatever the reason. So we can see, if the meeting room has been-- actually been used or not. if you'll go to the next one.

And another use case that we are developing right now in this particular moment is for the IT team, that they wanted to have all the information about the port and the switch for the internet connection that we have in the office. They want to have all that information in the model in order to check if the port number 37 is-- has any kind of trouble. It's easy for them to know which table, to which a user is affecting this malfunctioning.

If you go to the next one, please, so we have created a view for the IT guys, for the IT team. And in this view, we can only see the tables for the team. And you can see, in color, the ones that are already tagged, which means that the information about the port is already in Tandem for that one, and in white, the ones that hasn't d added the information yet. So you can see that only 62% in the bottom left pie chart or donut chart is the amount of tables that has-- that we have the information already in there. If you go to the next one, please.

Now, this is a small video, and we can see that, real-time, we have 62% of the tables tagged. But if we add information from for another table, just [INAUDIBLE]. We are doing this video. Automatically, the information is updated on the left-hand side, which is the dashboard that we are managing.

So we have real-time information, which is really cool. And also, clicking on the donut chart, we can see the table that we have the information for them and the ones that we don't have the information. So it's easy, also, to talk with the IT guys and check what is the status of all of this work.

LUIS CLEMENTE: Cool. So let's wrap up what have seen-- what we saw today. Initially, we talked about the roadmap. We focused on the existing buildings, talking about reality capture, what options are available, the process from scan to BIM, that Francisco was explaining, how we reach up to the point of the as-built model. Then we talked about data, what use case that are available and giving you the idea of choosing your right use case, the one that gives you value, and how you are-- what kind of sensor you're going to use.

And then the biggest focus was, OK, let's create a proper digital twin. I mean, we have the 3D model. We have the data. Let's connect that data into our model into Tandem so we can see that data live and we can explore historical data, also.

And finally, for facility management, we just saw a few use cases. So in terms of data integration, yeah, we talk about how the data flows from the device. We used for this pilot Zigbee. In the gateway, we use the Home Assistant OS in a Raspberry Pi. We also use the MQTT protocol for IoT communications. The broker and the pipeline to send information was created in Autodesk Tandem Connect. And finally, as a platform for the digital twin, we use the Autodesk Tandem.

Last part, yeah, we saw the views and dashboards of the use cases that we can explain, is we can use these kind of heatmaps just to see the information in a more visual way. That helps a lot with just a quick view, understand the general situation in a full building or floor. And also, the kind of information that we can see using the views and the dashboards.

So what's next? Let's see. For us, we will keep exploring on how Tandem keeps growing because we can see that, every now and then, they keep adding more features. So we will keep an eye on that. And also on Tandem Connect, which is-- even though it is in beta, we are able to use it and get some value from there.

We need to complete those steps of sending the data out, the alerts. So it's not only about live data that you can see on a viewer or a dashboard, but also getting activated, actionate that data so we can get even more value. And explore the integration with other systems, like VMS systems.

And for you, this is an invitation. This is there to do yourself. I mean, you try-- you-- really, it's not that complicated. You can do a pilot in your own building. I mean, if you are here in this session watching this, I'm pretty sure that you are able to do it. All the tools are there. All the details are there. You can check more details in the handout.

And finally, just connect and share. I mean, this is how we all will learn more and do more things on the digital twin world. So this is all for this session. Thank you for watching.