The Rise of the AI: Impact of AI and Machine Learning in Construction

ANAND RAJAGOPAL: Welcome, everyone. Thanks for making it up here today morning. After last night's party and given that the keynote is going on, we're really thankful for having a crowd-- any crowd-- here today. So welcome, good morning.

We actually have a really packed deck today, so I'm going to try and go as quick as I can. But after the talk, feel free to stop any of us, ask us questions. We'll try and save some time for Q&A in the end. But we'll be around right after the talk to answer anything you might have.

So let me start off introducing my co-speakers today. Christina is a regional quality manager at Swinerton Builders. Josh Lannen is a QA/QC manager at Bond Brothers. They both have years of experience on the job site. And they've been strong partners to drive AI-driven tools in their workflows. Josh Kanner is the founder and CEO of Smartvid.io and previously Vela Systems. So he's as familiar as anyone in this space about building technology solutions for construction.

I work as a data scientist at Autodesk. We work as part of a small but focused team with a goal of building smarter tools for construction. We work on AI and a machine learning-driven product called BIM 360 Insights, also known as Project IQ. I hope all of you have heard of that. The experience of building it has been, at least personally, some of the inspiration behind me compiling this talk.

So we have three main sections to the talk. I'm just going to do an introduction to AI, some myth busting, talk about a few examples outside construction and use that to drive applications in construction. After that, Christina is going the presenter her experience of using BIM 360 Insight on a project for Swinerton. And finally, Josh Kanner and Josh Lannen are doing a section together, where they're talking about how they use AI-driven tools-- Smartvid.io and BIM 360 Insight in this case-- for a project in Bond Brothers.

So what do we mean by AI? We hear a lot of AI these days. And with all the hype and sci-fi movies, it can be really hard to piece together what it actually does. Google isn't helping. This is the image search for AI. I didn't even have to edit it. And all you see are these androids.

The public perception of AI, or Artificial Intelligence, usually ranges between having the extremes of it ruling the world or being dismissed as a fantasy with no real place for a serious conversation. But in reality, it's somewhere in the middle. It can and is definitely being used to make smarter products. We either learn how to use it, or we really miss a big opportunity.

The confusion is, however, understandable. We haven't really been able to define intelligence, [INAUDIBLE] artificial intelligence. So let me rephrase this question. How should we perceive AI, at least for the context of today's talk? So Artificial Intelligence, or AI, refers to a broad field of science. It's like computer science, psychology, linguistics, a lot of other things. It's really involved with getting computers or machines to do tasks that would normally require human level intelligence. It's like a set of tools that try and simulate intelligent processes, like learning, assimilating knowledge, reasoning, conversations-- to a certain extent, it's getting there.

And just one more explanation of technology here. So we just spoke about AI. Let's see what is machine learning and deep learning. The phrases get thrown about so much in the media, and the names can be a little misleading.

So while artificial intelligence is a broad field of science, machine learning is really a small subset of that. It's a set of techniques where you can give a set of data to a computer and allow it to learn things from the data without explicitly telling it what to look for. It helps the computer understand processes in the real world. We're going to get into a lot of examples. And that'll help clarify it.

Deep learning, it's really the rage today. Everyone's talking about it. It's like a more recently matured part of machine learning over the last decade, more or less. And it's the same purpose-- getting more from data, just a slightly different way of doing it. And it's yielded a lot of better results in almost all the breakthroughs recently.

Two years ago, what changed? Two years ago, at a [INAUDIBLE] session when [? Pat ?] and I were there, we polled the audience about-- it was mostly a construction audience-- about how much people cared about data-driven insights. Around 10% raised their hands. There was definitely no one talking about machine learning there.

This year, even the CEO keynote was really just about this. So in a nutshell, what changed? Computers, computing power technically. It just got so much faster, but it also got cheaper. And we have data, like a lot of data. According to an IBM report, which came out a few years ago, 90% of all the data was created in the last two years, so stretch that, and today, it's probably 95% of all of the data was created in the last two years.

Every one of us has a mobile phone here. And we are pretty much adding to this massive pool of digital data every day. Most importantly, everything got more accessible. It's online. It's in the cloud. And suddenly, computers everywhere can start making use of this.

So my next two sections really kind of play off each other. I want to talk about two kinds of problems that AI is really good at solving and then spend some time looking at examples in construction where it's having an application for this reason. So text analysis, email-- all of us spend way too much time on it. Let's just pause and think about how much it's changed.

Five years ago, you received at least one email every day saying you want a million dollars. You won a lottery. A prince in Nairobi want to give you money. I wish one of them were true, right? But spam detection is one of the older and most well-known applications of machine learning.

By looking at thousands of emails, computer programs have really understood what a spam email is. We don't have filters telling look for these words, move to another folder. There isn't really a set of steps it does. It just learned what a spam email is with the context of your email. And every time you've actually marked an email as spam, the programs got smarter. It didn't focus on memorizing words. It really understood what those words meant.

So that's text. But another area of development is images, like image recognition has just been there. And well, who doesn't like cats? If you take the problems in this space, you can divide them into two broad classes. One is what does the image represent, and where is an object within an image. For example, given this photo, is there a cat in this photo? And where exactly is the cat in this photo?

And then people got on to slightly harder challenges. Are they the same cat? And all of this is on the cloud. What you learn is already out there. So what you learn in one problem, it can really transfer easily into solving another problem.

Almost all the newer iPhones in your pocket, they come with face detection base locking functionality. The algorithms can identify the finer details of a photo and do it quick enough that it can be used for real-time applications of the sort. And just one more generic example, let's look and see how these things connect together.

Self-driving cars-- now these aren't built on if statements and then statements. They really build on insights. They're able to sense their surrounding and use that to navigate the environment. These don't run on formulae or functions. Machine learning really focus on the principle of driving.

And let's dive in a little here. This is the image you would see of what a self-driving car would see when it navigates. The system is able to distinguish between the people, the cars, the stationary objects. It can look at the red light, understand what it means, read a one way sign. It's able to navigate, understand, and estimate things. The longer it's driving on the road, the smarter and better it's getting. And think about this-- when one car learns something about the environment, all the other cars are learning simultaneously.

So that was a lot to take in, but this was just really context. Now let's just look at the real thing. How does this apply to construction? I'm just going to touch on a few examples. Going to look at a little bit of pre-construction, an example there, a little bit of the construction execution phases. I'm just going to briefly touch on project management in the end.

So we saw self-driving cars. You can think of generative design as self-driving design. As a designer, you can input specific design objectives. You can talk about the functional requirements, the material, manufacturing method, maybe the cost. You give a system these requirements, and it searches a space of all the designs and gives you a set of designs that actually meet these requirements. That's really what we've come to here.

Let me tell you a story to explain this. We have an office in Toronto. And our team moved into a new building, and they're using this as a living experiment. It's an experiment to discover how much we could maximize productivity and create a great environment for the people working there.

We use generative design. We fed it the forces influencing human productivity. We basically did that by surveying our employees, capturing their preferences, work habits. So the system evaluated all the survey data against a set of hard constraints, like the boundaries of the building, the location of the kitchen, and so on.

And it then generated thousands of alternate floor plans. It started to minimize distractions, maximize the outside view, personal relationship, team dynamics, and find a solution for everyone's needs. It had that information. And we weren't stuck with the first floor plan that worked. You had many to choose from. And the people-- the architects and the designers-- could have this tool to aid them in that process.

The generative design tools were anticipating our needs. And this really helped find a good design, probably the best design, possible. I found out earlier this week that the AU exhibition booth area was designed using generative design. So next time you walk around, think about how the layout is. So that's a little bit of an example of how design is changing.

Let me move a little more downstream. So risk mitigation-- all of us in construction know construction risk happens every day. You've got hundreds of subcontractors. You've got thousands of open issues. And everything is really changing on a daily basis, or it better be changing.

And let's just talk about the first one-- issue management. It's really a big task on the job sites. You've got projects of hundreds of open issues. It's changing. You've got several stakeholders. And when we went and spoke with the superintendents on the field, they spoke about how much of the huge volume-- it was just flooded by issues. And just also the inconsistency and the chaos of it. You really have the important issues which often get missed in this thing, kind of lose the big picture.

So this is where Autodesk's BIM 360 Insight really addresses this problem. This is also really personal, because this is what we really started this project with. This is the first challenge we took on. It comes out with a prioritized set of issues. You can think of it a lot like your modern inbox. It's like a lot of noise reduction. Based on what is already entered into the system, the algorithm automatically assigns issue priority and helps bubble up the important issues.

Now important really varies on the context, right? Look at several things. It looks at the costs of the potential rework, would it impact the schedule. Maybe something's been open for a very long time, and it's not been acted on. It uses all this. And one example of a high priority issue which the system is really good at identifying are water infiltration or penetration issues. It can automatically identify if an issue can have a water damage, and it can get your attention for it.

But that's issues. Now general contractors also manage many subcontractors of different trades. Construction projects really succeeds or fails based on the performance of the subcontractors. And this is where, again, BIM 360 Insight uses several different models and identifies the subcontractors who are at risk on any given day, those who could benefit with the Supers time and attention on that day. It also shows you why it thinks so. It makes it very actionable. You know what are the reasons it thinks someone is high risk and gives you something to look on and think about.

That was quality and issue management. But let's look on the same theme, and let's talk about safety. Construction safety is the number one priority across all job sites. Based on the text for provided in the descriptions, BIM 360 Insight has a breakdown of the issues by risk and hazard. So in 2015, there were about 900 fatalities in construction in the US. About 2/3 of that was really due to four categories. You had the fall, struck by. You had electrocution and caught in between. This got infamously termed the fatal four. It's almost like a Tarantino movie here.

But what if we can automatically categorize behavior and patterns by these high risk areas? And by surfacing this risk every day, when it happens, could we help avoid the incidents before they occur? Addressing one issue fixes an existing problem. But analyzing a trend of this sort can prevent you from having these problems.

So like I said, this is based on the text which was already entered by safety managers on a job site. But they also take photos. It's a really fast growing data source within construction. Every construction worker has a camera. They see something, they click a photo of it. And it's almost standard practice now to take a photo with every issue you create. Leave that, you have drones, security feeds, Go-Pros, all kinds of variables which are being used.

Traditionally, however, photos have not been the best utilized resource. That's an area which AI really has had a lot of breakthroughs. So let's just talk about Smartvid.io. Smartvid.io is a technology startup addressing exactly this problem. They provide a platform that integrates with several different technology vendors and brings all your images into one place. But they go further than that. They use AI to automatically tag what is there inside that image.

Take this example. The system can identify the different objects in that image and sometime the object that should be in it, but it isn't. For example, he should be wearing his high vis gear. There's a hammer and a ladder. He's on the roof, probably not in the right place on the roof he should be. This information is really powerful. For example, it is now being used automatically to detect missing pp on job sites. And it can do that automatically. You have security feeds. It can do it in the background. And it's really all about prevention.

But there's another use to it. It makes a better retrieval of the photos. Because AI can understand what it sees in an image. For example, you want to see the duct work, and you want to see all the photos taken of it. You can just really ask for it. You can just ask for everything about ceiling, and it shows you all of that. There isn't the entire search.

And this is really out there. So, like, all your phones can do it on Google Photos. And it's the same functionality. And all you need to do it is you have the existing data. And you just need to start applying this on it.

So let me just talk about one concept for you. We've seen a couple of examples in safety. But what if we're able to bring this all together? Same metaphor, imagine if there's a self-driving safety walk. So traditionally, if you're on a job site, and you want to take an issue, you have to pull out your smartphone. You need to type in something. And it's really hard if you are wearing think gloves.

So what if things change to become more photo first? You see something you don't like. You flip out your phone. You change to your camera. You just point to something, take a photo, and it automatically detects what's wrong in the photo and creates an issue for you. Now you've got insight to what is there. You have the understanding and knowledge to identify how those things come together to identify a photo. Maybe you can even find out who it needs to be assigned to. This is a concept, but it's not really that far off.

All right, so we spoke about a few things which is already out there. I want to talk about another area where it's really going to have a big change and probably an impact across all of construction. So let's talk about how this is impacting design, and how is AI pushing all of this knowledge upstream.

We have data across different phases. You know your project outcome data. It's always stored. Now we can measure quality and safety risk. You've got the RFIs and change orders being captured. What if all of this could come up back together to the design and model coordination stages? As you're doing something in the model, every change you make there, what if you can see the impact? And I feel like that's going to be the real future where things can be fixed so much higher in this process that the whole process becomes simplified downstream.

So many examples. They're going to talk about how they use it on a project. But I just want to talk about one last thing here. What is really common about all these examples? They all require a lot of data. These algorithms are only as good as the data you give it. And that's really the limiting factor. A saying when [INAUDIBLE] is garbage in, garbage out. Give it bad data, what you get is not very useful either.

And one of the problems we face, especially in construction, is we have huge data silos. We've got so many providers that all your data is usually fragmented across all of a number of them. And unfortunately, the real power of machine learning is when you can connect these different sources together.

So Autodesk has been trying to work on this. And one of the things we've started off with is we've been partnering with a lot of companies in this space to try and attack this problem. Smartvid.io is one of them which we announced earlier, but there have been a slew of others which got announced this [? AU. ?]

We've been also re-architecting BIM 360 to try and address this challenge by having a central platform where customers can connect all the data across several phases. What is common is one common data platform, one common AI and machine learning layer. An Autodesk is helping you work towards customers prepared to thrive in this new technology.

All right, that was a little quick. Just trying to save time for the real conversation here. And we can jump into questions about any of this and all of this at the end of it or after the session. Let me just get Christina to talk about how they used BIM 360 Insight on a project for Swinerton.

CHRISTINA TETRICK: Hi. So you have a context of my piece of the discussion. And I'll tell you a little bit about myself and my company. My name's Christina Tetrick. I am one of two regional quality directors for Swinerton Builders. I began a little over 20 years ago-- we don't count that exact number anymore-- as a bridge decking carpenter.

My first job with the GC was field engineering, so I surveyed. I ran a few crews for a couple of years. Then I went into the office as an office engineer, project engineer, worked my way. I was project manager for about six years. Three of those years was in a special projects department. That's important to me because our company does many very large projects. But we also do many more smaller projects. And when you're working in quality, and you're trying to build a culture, it's important that you understand both sides of that coin.

I transitioned into quality. And at the beginning for a couple of years, I was the divisional quality manager for Colorado. And last year, Swinerton decided to revamp some of the structure in our quality department, and they put two of us at the corporate level to support all of our divisions. My co-leader is Dennis McCown, who is also here today.

And then just to kind of show you a little bit about Swinerton, those are our division offices. Dennis covers half of our work, which is all in California. So half of all of our work is in California. Dennis supports that, and I get to support all of the outlying areas. We both fly the same, but my flights are a little longer.

Swinerton's been around since 1888. We have an insanely low contractor license number in San Francisco that we're pretty proud of. I think that's 92, if I remember off the top of my head, but almost 130 years. Our employee number, we have, I think, half of those as craft employees. So we're very proud of that also.

My favorite thing about our company is that we're 100% employee-owned. And for being in quality, I think that's really important. Because you're trying to train a culture to do the right thing for the company as a whole. Everyone wants their job to be successful.

But when your employee owns the money that the company raises, or the money that the company ends up with at the end the day is actually in your pocket, you want the job to be very successful. You want your division to be very successful. And you want your overall company to be very successful. So that helps a little bit with what I do.

Another thing I would like to mention is we have an enterprise agreement with Autodesk. I think we're three years into that, and it's been very successful. We enjoy working with them. But we do use a number of products. This is the first relationship that's really caught on fire the last couple of years. We truly believe that it's the best quality tracking system, and it's very ingrained in a lot of what we do.

This is my analogy for 360 Field because I have four bandies at home, which means marching band children. And I'm a band mom. If you think of this as the database that holds all of your 360 Field quality tracking issues-- by the way, one of our projects that I've pulled a lot of my information from today is Country Club Towers. We were one of the pilots for IQ. That's why I'm going to be using that information. So a project size of 170 million, you're going to have between 30,000 and 50,000 issues.

I heard someone say earlier that means you have 30,000 to 50,000 problems. That's not true. Hopefully, your issues also include a lot of documentation issues, a lot of progress issues, maybe safety kudos. There are a large number of positive things in those issues.

Anyway, if you want to take a look at some of the things out of all those issues, you want to look at how many deficiencies you are. It's very easy to filter and sort in the system. You can pull that whole line of tubers out, and those would be your deficiencies that you want to focus on. Or if you wanted to look at where are our punches right now, you could pull out all of the group of trombones there.

But it's still a really large number to look at. And so we were very excited to hear that there was a program out there that let the risk bubble up to the top. I love that term. It wasn't mine. I've heard it a few thousand times in this week. But it's a great term. I'm quality, so I'm part of the risk department, just like safety is, just like our legal. And that's really where we want to focus if you're in a position like mine.

So here's all of our data. Now we're going to find the risk in that big data. And we started using IQ to do that. Sorry, I'm going to refer to it as Insight just so you'll start learning the newest name that you're going to hear.

My big point is when we started using Insight, we didn't realize how many roles in a project or in the company that it really benefits. This is a high level. This is called the executive view. So if you're a quality director, or in risk, or an executive at the very high level of the company, or even a division manager, or even a general superintendent, you can go into the system.

And this is your first look. And over on the left, you have a list of the projects. And it bubbles the higher risk projects up to the top. And then on the map, usually, you see a large number of flags. So it'll show you where the red ones are in the country. So if you're spread out all over the country, you can see very quickly where your risk is lying. And then you can drill down accordingly.

I blacked out a lot of stuff to protect the innocent, or the guilty, whatever that is. If you're a project manager, or a PX, or a superintendent that's kind of high level for that project, there's a lot in it for your level also. These are some of the great pulls of information that it lets us see as you drill into a certain project.

That top one, it's every subcontractor on the job. And the green is low risk. Yellow is medium risk. And the red is high risk. So you can see as a timeline through the project how the sub is performing, when they might be in a riskier area. We've got one sub there that's almost all red the whole way through. Can anybody guess what kind of a scope that is? Just guessing.

AUDIENCE: [INAUDIBLE]

CHRISTINA TETRICK: What's that?

AUDIENCE: [INAUDIBLE]

CHRISTINA TETRICK: It's our current wall. So water intrusion is your highest cost, your highest risk. And so almost every one of the issues associated with that sub the system sees as a high risk, which it is. And then other moments are, there was one that has a real good red spot in there. I forgot to look at what it is. But I would look at that, and I would say, OK, they've hit a rough patch. But look, they've gotten that taken care of. We're back in the green now.

As a project manager, I would love to look at this monthly, when you're getting ready for billing, and you want to hold their feet to the fire and say, look, we have an issue here. We need to address it. This is a quick way.

My general superintendent loves to look at the bottom one. Those are our active issues compared to closed issues. Because you can get a quick look at how your team is performing. Are we documenting the issues? That's great. But it's almost more important to make sure we're addressing them and make sure we're closing them out of the system. That's just a quick look. There's a number of different very helpful graphs of information you can flip through there.

My project engineers and assistant supers-- the people that are really putting the issues in and addressing them, making sure they are addressed, and then closing them out-- when they come on to the project, this is the first thing they see. Because they're only tied to that project. And it's a list of subcontractors-- how many high ones and how many medium ones. Now you might think that's a really large number of high. This is out of 64 subcontractors on this project.

And so when you come to this page, they'll click on that first sub. Sorry, half of the page is one. And half of the page is the second one. So when you click on one of those high risk subs, you'll see what number of risk they are. And then you can drill down, OK, why are they high risk? You can see that this is a number one. And they have water risk pretty high there. They have outstanding issues that are way past their due date et cetera.

And then you also have a list over there on the bottom right. OK, what are those high risk items? So when you are that guy standing in the middle of that field of marching band, and you can't see where all of this structure is, and you really just don't know where to even start your day, this is great assistance. That's about it on that page.

So the next slide I'm going to show you, if you click on this high risk issue, this is my favorite thing about artificial intelligence. It's machine learning for a reason. It's learning for us. We're the ones putting additional information in there to teach it. So if you click on one of those high risk issues that it's showing you over there, you get this page. And you can look deeper into it.

I had to look a long time before I could find one that made sense to change the risk level. Because I wanted to talk about if you do that. We train very hard on the projects that are using this. We're still a little bit early getting it out now. But it's so important to tell people a, when to change that risk, because you don't want to teach it incorrectly. That works against you. And b, if you are changing it, make sure you put in there why.

So this item is water pressure issue in the bathroom. That's not a high water intrusion risk item. They just need to go in and tweak the pressure. And so I put that down to low. And I made sure to say, water pressure is contained, and the adjustment is the only thing needed. So logical adjustments to the AI.

What some of the people that are out there putting in many issues and kind of worried about how their project looks, they may have the thought to go in and say, oh, that's not high risk. That's not high risk. That's not high risk. They're teaching it incorrectly.

So we need to be logical in teaching them, wait, if it's-- the example I heard this morning was concrete. If the concrete break is 56 days out, then we want to change the due date in the issue to be 58 days or whatnot. And then it won't bubble up as a high risk overdue item.

If the item has just been open for three months because oh, it's not really that important. They'll get there eventually. Don't go change it as low risk. Go into the system, and make sure you're closing it out accordingly. So that's very important when you get your teams going on it.

These are a couple of quotes from that project. I hate reading to people, so I'll let you read them. Tyler over here on the right, he's one of the two people that used the pilot program every day. And he very much appreciates that he had some good stories of some water items that they had taken. And it seemed really small, and they put them in the system and then kind of forgot about them. But then when they bubbled up a couple of weeks later, it's very important to hit those.

I think we're going to do questions. So I'll take a few questions if you have some specific to what I was talking about now. We saved a couple of minutes in there.

ANAND RAJAGOPAL: I think we have microphones to go around if someone has a question for Christina. We have a few minutes. And then we'll just transfer to the next section.

CHRISTINA TETRICK: I can repeat what you have too. Go ahead.

AUDIENCE: Yeah, I just wanted to clarify that this Insight is not the same or a subset of [INAUDIBLE].

ANAND RAJAGOPAL: No, this is a separate applications. This is part of the BIM 360 ecosystem, you can say. It's called BIM 360 Insight. It's a different thing from the energy modeling application you're talking about.

AUDIENCE: And it's available now? The BIM 360 Insight is available [INAUDIBLE].

ANAND RAJAGOPAL: Yes, it's right now. You need have a pilot agreement with us specifically. And then we can talk about this.

CHRISTINA TETRICK: Yes?

AUDIENCE: Yeah, I'm wondering about the people in your organization. Did they feel like there's a certain amount of value to this, but it's a pain in the neck to input all the data? What are the opinions about that-- how much work it is to generate the data and then have to power [INAUDIBLE]?

CHRISTINA TETRICK: So this is the cultural training that I go through.

ANAND RAJAGOPAL: Can you just repeat the question?

CHRISTINA TETRICK: Oh, sorry. To repeat the question, do we run into a number of people that think it's a lot of work to put data into the system? That's a really broad question. But the answer is yes.

It's like when I first met my general superintendent. He was using his laptop as a paper holder on his desk, if it was on top of his desk. This is the same general superintendent that told me that was his favorite graph. And he is in the system all the time, and he has learned now. That's been a number of years that we've grown that way.

And somebody made a great point earlier. We have all these really young, ambitious PEs that love technology, and they're all about it. And then we have a large group that is not used to doing it this way. One of the reasons I like to explain what I did in the past is because, especially when I'm training my peers or teams that I support, I want them to understand that I did it the old way.

I did punch lists by Excel spreadsheets. And we sorted them and distributed them-- whoever was doing the distribution. I remember a project not that long ago. We had a person out there full-time. And that is all that person did.

The way that technology allows you to put information in the system now is unbelievable. And it's faster than writing it down. And if you have run crews, this is a good one that supers love to hear-- this example. And it comes from superintendents. They walk the project all day. They're interrupted 30 times. They have 50 fires that someone hands them. And then they're hoping to remember all that by the time they get back to the trailer.

It's always a little bit of a tough challenge at the beginning when you're trying to train superintendents on how to use it. Sorry, I don't mean to call that out. But it might be true. Every superintendent I've ever really walked with, and taught him one thing at a time, and then they gradually start appreciating the system, they've fallen in love with it. And I'm not even exaggerating a little bit.

I had one super that they were allowed to have it on one project. I spent all this time with him to really get him into it. We had a good relationship already, so he trusted me. And then the next project-- because this was when we were first starting with Autodesk-- the PM didn't let him have it, and he blew a gasket. It's like taking away your cell phone. Hope that answered the-- Yes.

AUDIENCE: Can you talk more about how can the top down [INAUDIBLE] program isn't used in isolation. There's a lot of other things that [INAUDIBLE].

ANAND RAJAGOPAL: Shall we actually get to that after this. Let them get on, and then we can come back to this question.

CHRISTINA TETRICK: Absolutely.

ANAND RAJAGOPAL: We'll definitely take that first step after that.

CHRISTINA TETRICK: Yep, we'll remember that one.

ANAND RAJAGOPAL: All right, Josh, take it off.

JOSH LANNEN: All right, so I'm Josh Lannen. I'm a quality manager for Bond Brothers. We are about a 100-year-old family-owned company. Actually, we're over 100-year-old family-owned company. The fourth generation is still there. Fifth generation is being groomed to start taking over the company.

We are primarily in the Northeast, pretty much everywhere on her map that wasn't filled up, we fill up the other half. We serve different market sectors. We have five different divisions.

And a little bit about me-- I have a degree in civil engineering. I started out as a field engineer, assistant superintendent, superintendent, project manager, and then I took this role over as company's quality control manager when they decided to expand the program. We started seeing a trend in issues that were recurring over and over again. And that's where machine learning and AI, for me, is really powerful.

We had a question just a little bit ago. How hard is it to push this on people? You're not really pushing it on them. You're asking them to do the same job that one, they're required to do and that they're already doing it. The power of artificial intelligence and machine learning is to take that information, skim it off the top somewhere else, and then deliver it to people that are in positions to make decisions and correct it.

So what this happened with Josh is about a year ago, Josh's company came to us. Actually, I reached out to them, because one of our VDC managers said, Josh, have you seen this new company out there? I said I haven't and reached out. I was like, wow, this looks really cool.

So Josh and his company, they reached out to me. We set up a meeting. And he started showing me this app. And wow, this is cool. We're going to take all these pictures, and we're going to tag, and it is really effortless. And then the first thing I asked is, well, how do the pictures get into your system? And he said, well, you have to take them on our app. And I said, OK, that's great. I can't even get my superintendents and even myself to take anything in my app. That's not going to work.

But BIM 360 Field, which we've been using ourselves for 10 years since Josh was at Vela. We were one of the first customers on Vela too. We have hundreds of thousands of photos in our database. Can you get that information out? Can you get that? And Josh said, I think so. So I opened up our platform. I gave him access, and I like to believe that, I think, we were project zero for the Smartvid integration with BIM 360.

So Josh and I are actually going to talk about the project that we have access to, which we now have two years worth of data that we've extracted. And that's what we're going to present to you here. So I'm going to hand it off to Josh.

JOSH KANNER: Thank you, Josh. I'll take that.

JOSH LANNEN: Try not to get confused with the Josh and Josh thing.

JOSH KANNER: Yeah, maybe I'll be JK, but not. Just kidding. So thanks Autodesk for the opportunity to present on this panel. Hello, everybody. It's the last day of AU. Hopefully everybody is still awake out there or excited for the exhibit hall, and then the party here, and then we all get to go home. It's been a crazy week, a lot of discussion around machine learning, which has been great for us. Because that's really what we focus on.

So as Josh Lannen mentioned, I've been doing construction technology since the Vela Systems days. So Vela Systems is now called BIM 360 Field. The applications that generate the issues and that gather a lot of the photo data for quality management, safety management, commissioning, and more, those are applications that I was a part of helping to build.

What we saw at Vela is that there's a bunch of field processes that gather that data and that benefited from mobility in the cloud. And after Vela, we wondered what's next. What's the next thing that could, just like mobility in the cloud, unlock value for quality control and commissioning and safety? What new technologies, whether it was mobile, or drones, or machine learning, how could they impact the construction industry?

What we saw is that there's great potential there. And what Josh Lannen and I are going to show you are some real examples of that from bringing data in so that field personnel, field supers, projects engineers, they don't have to change what they do. They're still using Field. We're mining the data on the back end. It's no more work. But now you're benefiting, just like you can with BIM 360 Insight analyzing what's in your issues and thinking of it as reading it and doing that analysis.

What we're doing with Smartvid.io is were unlocking the value of the photos and videos that are associated with those issues. So it gets back to what we were talking about before. What Anand had showed at the beginning-- image recognition, speech, and text. We're doing image recognition and speech recognition.

And the goal here is, just as we did with Vela, with mobile, and the cloud is to address really specific, very important parts of construction delivery and oversight-- quality, productivity, and-- as we're going to talk about now-- safety. Let's see if the clicker works. I did. Oh, wait, hold on. Got it. I was hitting the laser pointer. It's the last day of AU.

All right, so Josh has already made this point, so I'll quickly gloss over it. A lot of folks don't realize that there's a ton of photo data that you're already collecting today. So photos and videos are a critical part of all those processes that I mentioned before. It adds up to gigabytes of data that you have as an asset, but right now, it's really only used for, let's say, documenting in-wall or taking a picture of, let's say, there's a crack.

So what we realized early on is that there's not just a lot of data, but there's also people in that data. So it's a little bit of a deeper level there. What else is there? It's not just that you have in-wall progress photos. You actually have photos that have people in them too. And around 50% of the photos that you're taking on your jobs actually have people in them. And you can analyze that if you have the right kinds of machine learning that can look for stuff.

So we saw pictures of kittens before. And what we said is, well, what if instead of finding cats, you could find other things? What if you could find instead of cats, you could find cracks? Or if instead of cats, you could find and do on-the-fly PPE compliance inspections based on what people were wearing?

So we built this product that allows you to use our app or-- thank you for the feedback, Josh-- if you say, hey, wait a second. Our field guys, we just trained them on how to use BIM 360 Field, or we just rolled out this other thing. We don't require you to change that field process. We actually allow you to integrate. Seems to be I like pushing the different buttons on this thing. They like to integrate the data.