Augmented Reality and BIM-Based Progress Tracking on the Jobsite for General Contractors

JASON WADDELL: Hello, everyone. Thank you for joining our class. This is on Augmented Reality, BIM-based Progress Tracking for Jobsite for General Contractors. My name is Jason Waddell. I'm director of construction technology at Batson-Cook Construction.

CANER DOLAS: And my name is Caner Colas. I'm the cofounder and CEO of GAMMA AR. And very happy to talk with you about progress tracking.

JASON WADDELL: This is the Autodesk safe harbor statement. I'm sure many of you have seen this in the past. So our agenda today is going to be about progress tracking and how we capture the direction of the construction and what's going on as far as schedule goes. The introduction-- we'll introduce ourselves. We'll go through progress tracking, talk about the hospital campus that we worked on on this project. We're going to talk about setting up progress tracking in Autodesk as well as in GAMMA AR. We're going to look at using GAMMA AR on-site, and then we're going to talk about the reports and finalize the progress.

So speakers-- I'm Jason Waddell, as I mentioned, director of construction technology at Batson-Cook. The middle picture there is a picture of my team or most of my team. And then my home team is-- I have three daughters. Two do cheerleading-- travel competitive cheerleading, and the other one does equestrian. I have two German shepherds, and when I'm not dealing with all of that, I like to go mountain biking. It's a big passion for me. And John gave me a hard time for not having a picture of me on here, so that's a picture of me.

CANER DOLAS: So then, perfect-- basically, I will say I'm a professional cheerleader as well, but more like in a company sense. So we have a company in basically augmented reality and for construction. And what basically-- shortly about me, I used to be an architect before. Now, basically, I have this construction technology startup. And just to give you a quick intro about myself, outside work, I love, obviously, traveling. That's one of my favorite things to do. On the left side, you can see me coming back from Southeast Asia after backpacking probably around 2,000 miles. It has been a crazy journey. That's the reason I look exhausted.

And I love taking pictures. That's what you see on the bottom part. And on the top right, you can see the UFC Performance Center, where we met with Kajima-- big MMA fan as well. And you can see in the picture also-- in a few seconds, Jason. There you go. He is also on this picture. So that's where we actually met.

JASON WADDELL: OK, great. So we're going to introduce Batson-Cook. Batson-Cook's a $600 million company. We have about 500 employees. We work mostly in the Southeast. We have seven offices. We've been around since 1915. And probably 50% of our business is health care. So we do a lot of hospitals, hospital expansions, renovations, things like that. In 2008, we were purchased by Kajima in Tokyo, Japan.

A little bit about Kajima-- they're operating in 28 countries, $16 billion company, nearly 20,000 employees. But the best part of it is we get to work with them. We get to work with a bunch of the companies that they own within the US and their R&D center. They have a $75 million building they built a couple of years ago just for R&D. They have 300 employees in R&D. Most of them are PhDs, so they do inventions on robotics and software to help the industry. It's really pretty exciting working with them.

CANER DOLAS: So then let's get to our company. So what you can see here in the next slide is basically our team at GAMMA AR. So we have, let's say, a lot of people from different migration backgrounds. We're here all in Luxembourg. So you can see, a very diverse team, super international. And we have also two dogs in the company. One of them is actually hidden by the other guy here. But generally, it's a very awesome team. And we work internationally as well. Next slide. Here we go.

And you can see here that we basically run in 33 countries. And generally, the customers that we have are half-half, basically, owners and contractors and with some percentage in architects and engineers. That's generally the distribution. And a lot, basically, of our customers are in Europe and North America. That's like the base of that, and then some also in Asia.

And what we do at GAMMA AR is actually helping construction companies and owners to cut errors and delays with augmented reality on the job site. And we do that in order for them to-- or for the people on site to understand design better before they execute it, which then helps them to avoid, basically, construction problems, which ensures a proper execution on-site, so reducing end rework.

And another thing that we do is basically helping with the QA/QC process on-site. Since we are overlaying the models directly at the correct position on-site, the superintends and format can directly see what had to be there, and you can just compare, and then see if there's mistakes. And another thing that we do is basically to help to track progress.

And the achievements of our customers are, at the end, time and cost savings, which basically typically range from 75% to 80% compared to current digital solutions that exist in terms of the [INAUDIBLE] of subcontractors, the comparison of the plans which [INAUDIBLE] and crash detections on site. And as you know, rework makes up, in the US, between 4% and 6% of all construction costs. And we have to mitigate that by basically fighting rework one by one. And this can actually massively reduce, also, the cost in that.

And what is great about GAMMA is it's super plug and play. So basically, what we do is bring the models to the construction site without any preparation that is required. So you can just take your Navisworks, Revit, or FC models and then basically place them on the jobsite, like on a specific corner. And with that, you're ready to go. So there's no preparation needed, and it works with our huge models.

And what's also great is we are very inclusive in how we work with, at the end, people. And as you know, not everybody on construction sites is very tech-savvy. And in order to get really everyone on board, you also have the possibility to store the positions on-site directly with the QR code waypoints that we have, which allows everybody, at the end, to basically get started.

JASON WADDELL: And one thing that's great about the QR codes is we've used previously-- tested out different solutions as far as augmented reality goes. And traditionally, you have to put the QR code in the model in a very specific location and then go out into the field measure and place that location exactly where it sits in the model.

With GAMMA, you don't have to do that. You print out the QR codes. You walk out, you select them or place them on whatever wall column you see fit, scan it with the software, and it will triangulate your position and position it for you in the model. And then you're just off and running at that point, which is a huge benefit to the field and getting these QR codes placed.

CANER DOLAS: Which brings us to the two final points, which is the QA/QC part. And then we're going to jump right into the progress tracking.

So, as you can see, with the overlay that you see here also in this video, it's quick and easy to identify any deviations on-site. And this deviations then can just be directly visualized also in the model. And one click again here-- we will just see in a second how, this, for example, looks awesomely in the ACC viewer. So each issue that is recorded is basically placed in the model from the site directly, kind of like an inception thing. And if you click on any issue, they will also go to the same basic camera position. So that's basically this, and right next back to progress tracking with Jason.

JASON WADDELL: Thanks. So yeah, we're going to get into progress tracking now. And the biggest question is why? Why are we going to do progress tracking? When it comes to complex buildings, or really any building, we traditionally are going to be doing progress tracking. We talk about what gets measured gets managed.

So, typically, our superintendents are going to be out in the field, potentially with paper plans, maybe an iPad, and they're going to be highlighting the plans on what's installed. They might even go to his subcontractor meeting and ask everybody, hey, where are you at? And they're pretty much going to tell you what they think you want to hear, which is not as good a solution for keeping track of where you are on the project.

20,000-foot view, kind of high level, you might have a project that's getting behind or a subcontractor that's getting behind, and you don't realize it because you're just looking at percentages, big picture, where are you on a floor-- 20%-30%. You're relying on the trade partner, the subcontractor to tell you where you are with this system we're proposing and we implemented on this project.

You're looking at very granular data, every single piece and part of the model and, on a daily basis, tracking where you are. So if a subcontractor gets behind, from day to day, you can tell them, instead of waiting weeks or maybe months and then saying, hey, you've got to bring in extra crews. You've got to work nights and weekends to catch up, that type of thing. You can catch it earlier. 85%, as an industry whole, projects are delayed. We pride ourselves on being in that 15% that's not delayed.

So this is the project-- big picture, complex project. It needs a lot of attention, needs a lot of people to track and maintain it. It's called Northside Hospital Gwinnett. It's in Lawrenceville, Georgia. That's where our headquarters is, so big project for us. This is the overall campus view. We built a 1,753-car parking space, medical office building as well, 160,000 square foot medical office building.

Those two buildings were turned over a year ago, which, as you can see in the picture, also turned over about 75% of our site. So we had to build this 600,000 square foot patient tower with limited space for lay down, on-time deliveries, that type of thing. Underneath the front there, you can kind of see there's a detention vault, a 3 million gallon detention vault, where there was a pond. So that was a complicated start to the project-- and then a 50,000 square foot two-story central energy plant to power the whole facility.

A lot of interior work-- we're connecting to the new building, so there's a lot of renovation work, other projects within this project. So it's a complex, big project going on.

CANER DOLAS: And this brings us to, at the end, what we want from progress tracking? So, for most cases, it's actually super important to get a better overview about what's currently happening on-site. And this can be divided into multiple basic questions at the end. And at the end, we all want to know, actually, where we are currently standing with the construction progress.

So, for example, for the superintendent to understand, OK, what is basically current stage? Where are we? What did we do? And this requires, at the end, a visualization of the progress, which is traditionally done by markings with highlighters on the plan. And we're going to come to that how we are doing now. But that's just basically one way.

And, at the same time, what we want to know is basically who's working on what and where? And this is also super important because you, at the end, are managing a lot of people to achieve a certain result. And, in this case, this result is basically the work of many people that come together to build something up.

And, with this, it's actually super important that if you see any delays, you want to be able to react to that. So manage, at the end, your workforce to put them into a specific place to ensure that they can build it up at that place also then faster. So this helps to-- this should be also one of the questions that we help actually to answer.

And, at the same time, we want to know, basically, what are the effects on schedules and what are the implications on costs? And for this, we need to be able to have much better data from the site than what we have currently because, currently, it's more like estimates that are happening.

JASON WADDELL: OK, great. So we're going to get into the actual kind of meat of the class. Now goals-- what we should expect to be able to learn is we're going to show progress in the model, how that happens. Many of you might already be doing this type of work with ACC Build. That's perfectly capable of doing that. We're going to show you how to extract the quantities and work in the field, which is maybe not what you're doing. And that's a little bit challenging with ACC in the Model Viewer. A more immersive experience is easier to work with. And then preparing the data, tying it to schedules and invoices, and then how we're going to visualize the data at the end.

So setting up BIM-based progress tracking, how we're going to do that within Autodesk and within GAMMA. So there's a lot of different ways we can actually track the data. There's counts, dimensions, properties. Obviously, it's kind of similar to how we would take it off as an estimator. So there's count objects, doors, windows, columns could be a count object, dimensional objects, walls, slabs, things like that, ceilings. They have to be measured by area length, volume, different types of dimensions that we can set up.

And then properties-- master format types, date and time level. There's a lot of different opportunities to combine all of this data with properties within the model and in assets. So you're going to have to create custom fields, custom categories, and organize the data within ACC. We'll go into some of that.

Some issues that we dealt with out there is unmodeled data. Sometimes, there's going to be elements that are not modeled, like formwork, for instance. And it's hard to track the progress of that if you're not modeling it. Multi-component objects like slabs, that might have rebar PT in it, or walls, that might have exterior, interior, drywall, brick, things like that, it's challenging to track the progress of that if it's all one object.

So missing properties might be a big issue as well. If you're trying to pull, say, assembly codes, for instance, which is masterformat, we might not have all of those in the family. So we want to make sure the data is there. Model hierarchy issues, we dealt with around parts, and some of our trade partner models had been using parts because they're fabbing from those. So the object data and the property data are different levels of the model hierarchy within the data.

So it's challenging to get that information. And then models not reflecting construction, it's hard to go out there, pull up GAMMA, look at it in an immersive experience on your tablet or your phone, and not see what's actually being built. So that can be a big challenge.

So the first thing we did on this project was we worked with cloud models. And that was really important. And it helped us later using those models in GAMMA and pulling the data that we needed out. You can use IFC, you can use Navisworks. We chose to use cloud models. We were already working in cloud with all of our subcontractors in our environment. They actually asked us to do that. So everything was live, helped with coordination immensely.

Our design models were not. So we had to download those, but we only had to do it one time, as did you see. Everybody else models were already linked. So we only had to do it once, made them cloud models. They all updated, and everybody was moving forward. So that was great. But just to go through a cloud model in Revit here, you're just going to go in, set up your sets, select your views, if you have a lot of 3D views, which I happen to have in this particular instance, then searching for it is a lot easier.

So I set up a GAMMA view. Just make sure that they're all checked, save that. It's going to ask you to save the model locally if you haven't already. So you're going to save the model, and then you can go into the cloud, select your folder, select the file name, abbreviate it if you need to, and then you got your model in the cloud. And once it's in the cloud, it shows a little cloud next to it. And you can see the versions. We're updating these constantly throughout the project, and through coordination.

So setting up assets in actual ACC. That's the big bulk of it. How do you want to organize the data? You want to talk to your team, make sure you're understanding what they want to track, and how they want to track it. So that's really important, because later on down the road, when we get into analyzing the data, we want to be able to sort, and filter, and organize the data how the team is going to want to see it moving forward. So we got categories, master formats, and trade.

There's a lot more than that. We used all of these. The models were set up by trade, obviously. Each trade modeled their own model and then had multiple versions potentially if they were doing multiple trades. But each one was a separate model, which was great. And then we used master format and even Revit categories in some instances. So as far as setting that up, you're going to go into Assets. You're already going to have categories and custom fields set up ahead of time, because you've already talked about them and gone through that.

So you're going to go into your model, make sure you select the right model. In this case, I'm going to be selecting our drywall model. So we had a lot of people modeling. In this case, drywall was one of them. And we even had curtain wall modeling as well for embeds. It was great to track that and be able to position that in the field as well.

But we already set up our categories for framing well. So we got by fire rating, and then ceilings, and non rated. So you're going to come in, and basically set up a search set essentially, like you would in Revit, or in Navisworks. So not a big deal. I'm doing it by fire rating equals, you can see one hour, two hour, two hour again, separated twice. So in the two-hour scenario, I have to do an OR statement and select both of those as well as non rated, because some of them were blank, which represented by a zero. And then some of them are actually called out as non rated.

So ceilings, we do by category. And then we're off and running, and it is pulling in all of those assets, and creating thousand of assets in this case on this model just for drywall. So it's been really beneficial. And then once you get done mapping all of the actual assets, and pulling them all in, then you're going to map the actual properties within each asset.

So you can see, we're going to go next here, and we're going to start mapping properties. And then you can define your property on the left, which is going to be an ACC and then what element that-- what property within the model that actually ties to. And what I like to do in this case is actually have the model on another screen opened up. So I can select the wall I'm interested in, and I can see what properties I want to map. Some of them have multiple links, multiple levels, different things like that.

So you want to make sure you're selecting the correct one for your mapping. So it's nice to be able to go back and forth between the model and the properties as you're mapping them. This goes really fast. Once you do the first one, you understand what the properties are. You're just turning them on, mapping the property, and moving on. It really works really, really well. And then once you get everything mapped out, you're going to hit, Next, and then it's going to import all of those properties into the assets, which can take a little bit of time to do. But it's really beneficial and pretty seamless as far as getting the information in.

And then they're linked to the model. So you want to make sure you're using that exact same model within GAMMA once you set that all up. So here, it is importing the information. We go back to our assets and we have all of our one-hour information with all of our asset information filled out, properties, links, areas, assembly codes, fire ratings, really good to go.

So now, how do we connect that to GAMMA within-- from Autodesk? So setting up GAMMA is really super simple. All we have to do is go from setting up your project, going to Add Integration to GAMMA, and then select Construction Cloud on the top. You can see there's multiple integrations you can use. We use the very first one, Autodesk Construction Cloud. Select your project, and then you're good to go. Once you-- that's all you have to do. Set it up. The integration is complete between the two projects.

And then I select the models, I have my whole folder tree on the left, which is awesome. I can browse through the entire folder tree, find the models I want to pull in, I import my models in, and then I set up automatic updating. And this is a huge benefit. So inside of GAMMA, it will automatically update any changes that are done inside of Autodesk Construction Cloud. So in the past, I've never wanted to give the subcontractors, the foreman, anybody. They've always asked me, but I don't give them the models. I don't want to give them the models until they're signed off.

There's going to be changes. We're moving stuff around during coordination, writing RFIs. Things are changing a lot during coordination. So I don't want my field guys using those models until we're done with them and they're signed off, they have shop drawings and everything, they're good to go. But with this new feature, and the added benefit of the tool itself out in the field, you can give them the model earlier. They have the ability to work in the model, in the software, in the immersive experience.

And I am confident that they're going to keep getting the updates so they can keep tracking how the progress is going for coordination out-- right out in the field, start making decisions earlier out in the field, which is awesome. Now, once you have all the models loaded in GAMMA, the next step is just setting up views. You don't have to do this. You can do this on the fly out in the field. I like to set it up ahead of time. I set up all my levels, maybe my areas too, and then attache the appropriate models.

In this case, we had a lot of models that were per level. We also had models that were the entire building. So GAMMA will-- once you select your level, it will crop the model down for you. So you're only loading up smaller portions of the model, not the entire thing. So you can see some of these models are pretty big, but you're not loading the entire model, which is great.

CANER DOLAS: So then let's get back to GAMMA on the job site. And now, we're going to discuss how we can actually basically do the progress tracking with GAMMA AR on the job site. It's pretty simple. We have seen how we align the models before. So that's basically how everything starts. You just go to the site, you align the model on a specific corner, or with the QR codes. It doesn't really matter. And then, basically, the job starts, which is you are walking the site and you are looking around yourself, identifying with the tablet overlay, what is actually already built there, what is installed.

And then the next thing you need to do is just to click at the end, Track. So just very simple, selection and tracking. And here, you can see in this video how this actually works. So you can see here how the people at Batson-Cook are overlaying the site. Then here with the concrete objects, they can just understand what is already finished. And then just with basically the swipe of their hands, get everything selected, and then mark them as complete.

And this allows them to directly identify what's finished. If there are some additional objects that need to be marked as well, you can do it all the time, obviously, at multiple objects to the selection, and continue. And other structured objects, obviously, you can do it with every object here, also with the MP objects, where Jason is selecting the data on site. So as you can see, it's pretty simple. It's just brushing over, and then selecting the objects, and then marking them as complete, which allows at the end, users on site, to get a much better control over what is finished.

There is different stages where this can be done, but as long as you are doing this on site regularly, it's not a lot of work also. It's just you're there, just finish it doing your walk downs, and you get the data collected. And the great part is it's all related to the BIM objects, which allows us at the end to basically reference this-- all these properties from the objects that we will use then later on for visualization. So that's basically it from here.

JASON WADDELL: And I can't stress enough how beneficial this is having the model in an immersive experience right out on the job site. We've done this in the past where we've tried to go through the model manually, section it down, walking through it, trying to find where I am so that I can select objects, and turn them on or off, depending on if they're complete or not. And this, you align it, you're good to go, you swipe over it, you multi-select, and boom, you turn them-- you turn the statuses to complete. It's super easy, and really anybody can be doing it. So--

CANER DOLAS: And which brings us at the end now to the visualization of the data, because what do we want to have at the end? We want to, first of all, be able to understand in the model, what's the status? Then and as a final product, we want our PMs at the end, the project managers, to be able to understand the data that is tracked from the site so they can take the right decisions. Right? And in order for us to go there, we will need to-- just a second.

Yeah, there it comes. So we'll need to basically get the data from GAMMA [INAUDIBLE] ACC, which is at the end done with the setting up of the integration, as Jason showed before. And here, you can see in a very easy way that all the data is overlaid. So all the objects that we marked in the last video are visible here, for example, the structural objects, which allows the superintendents on site to get a much better understanding of what is actually complete.

And at the same time, this will also help everybody on the site to get a better overview where, for example, delays are already, because you can just see and compare it with the schedule, which allows at the end, yeah, people to get a much better understanding.

And then here, you can see all the data that was already actually tracked and in ACC assets. And here, the overview actually helps then, first of all, to identify any things that should have been done and so on too. But you can see, especially here, first of all, the volume, length, and area, and assembly code setup that was done by Jason before, which will at the end be used later on.

So the spreadsheet view is a great overview, but we want to at the end have a interpreted and analyzed overview. And for this to happen, we need to normalize the data. So when we normalize the data, this can be done either in Power BI or in Excel. It doesn't really matter. There might be some properties that, for example, are duplicates. And some information is in one or the other, just slightly differently. There's always things like this happening because you are working with a very complex environment with different, let's say, stakeholders that may have different naming conventions as well.

And you just need to do it basically once. And after you have basically set this up, you can use the data that was collected in order-- and then to-- and then visualize it. And next slide, we can just see directly-- [INAUDIBLE] Perfect. Thank you. Next slide, you can see here directly in the Power BI dashboard that is used by Batson-Cook. All the specific data from the different traits, and also how they are organized. And you can get the completion of specific traits even, or the overall progress. Everything is basically live and streamed from ACC assets.

So in this case, the Power BI dashboard, that was built by them, which allows them to get a much better understanding live, again, from the site in combination with the overview and ACC assets. It's really helpful for PMs to understand where they are. But if you don't have Power BI, it's not an issue too, because you can just get the data into Excel and then get the same overview there. So you can directly understand here as well. Hey, what's the percentage of completion by level, or by trait, or for the overall progress?

And you can actually even get the data down to the smallest component that you can find. So you can basically get the equipment numbers in a pharmaceutical project that are then referenced in the schedule as well, which allows you to, at the end, get to a very, let's say, clean overview about what is there. And that's at the end, what we want to have. We want to have this transparency on the site that allows the superintendents, the foreman, the project managers to really, yeah, understand the site, and also then make the right decisions directly on site.

JASON WADDELL: So some takeaways from the project that we saw working through this pilot was it's simple. It's simple to use, it's simple to understand, easy to implement, very precise granular data pulling in information on a daily basis is really a benefit to the project. It's clear. The Power BI reports are really easy to understand, filter, organize. And then some challenges that we had. It's important to start early. That's one thing that we saw is if you're halfway done with the project, and you're going out there and trying to understand where you are, and start turning-- saying stuff is installed, it's going to take you a long time, it might not be accurate. It's going to be challenging to do that.

So starting out early, really right away, would be the benefit and easy to do if you're doing it on a daily basis. The Power BI was actually a challenge. None of the custom fields actually come out of the data connector. So we used the data connector for a lot of stuff. We have a bunch of Power BI reports that we use for different things like submittals, and RFIs, and issues, and meeting minutes, and all of those types of things. So we use the data connector pretty heavily, but the assets don't actually pull out custom fields in the data connector.

So you have to use the data connector to pull in some of the information into Power BI, but then you also still have to export Excel. We store that in SharePoint and then we link that into our Power BI report. So we have two different data sets. And then we can mix, and match, and combine those into Power BI. So it wasn't the end of the world. That actually worked pretty good, but you do have to do an export every time you want to see an update from-- into Excel.

We talked about the designer models. That was a big issue. You don't want to be using uncoordinated or models that don't match what's being installed in the field, because the supers are going to look at it and be like, well, this doesn't match, so I'm not going to use it. Revit models, when you have shared Revit models in the cloud, they are going to use shared coordinates. So if you're using project-based coordinates for coordination or anything like that, and you're not actually using the shared coordinates, it's not set up correctly, that's going to be a big problem making sure they're all going to line up and work appropriately.

Obviously, in Navisworks, or IFC, is you can export whatever format that you want. So that can fix that issue. And thank you for watching our presentation. We hope you learned something and can use it on your project moving forward. Our contact information is below. If you have any questions, please reach out to us.