Using Reality Capture to Help with Hurricane Harvey Relief Efforts

JAVIER: All right. Welcome to a reality capture class. So why reality capture, and why photogrammetry? Because we believe that almost everybody can do it. In fact, we're going to run our first experiment of the day. And I want to ask you to take out your phones and take a picture of this. Try to include myself or us in the picture.

Nothing spectacular is going to happen, but

DAT LIEN: I post these on Twitter.

JAVIER: I just want to--

DAT LIEN: It's the best class ever.

[LAUGHTER]

JAVIER: Five stars, yes. So if you look at the phone and you can infer somehow the height of the screen based on that height and my height, you're already performing a sort of photogrammetric calculation right there. Because you're inferring geometry based on images. OK?

In fact, a very similar event happens inside our eyes. If we close one eye and then we close the other one, we notice a shift in our vision. That's called parallax effect, or disparity.

So, in fact, let's run our second experiment of the day. We're going to make sure that the parallax-- we're going to learn how that parallax effect helps us. So we need a volunteer.

DAT LIEN: We need one volunteer.

JAVIER: And that's going to perform the experiment.

DAT LIEN: OK, great. Stand up. All right. And listen to Javier's instructions very carefully.

JAVIER: So please cover one of your eyes and try to catch the ball with one of your hands.

[LAUGHTER]

That was rough.

DAT LIEN: That wasn't fair. My throw-- OK, let's try this again.

JAVIER: The standard for experiment might not be perfect, but, yeah. So now just with one hand, try to catch the ball.

AUDIENCE: [INAUDIBLE]

JAVIER: Just one. Yeah, without covering your eyes.

DAT LIEN: OK, good.

JAVIER: OK. Was it easier? Was it easier?

AUDIENCE: Well, it was-- I should've covered my eye with my left hand, too, so I can have my--

JAVIER: I understand. The experiment wasn't perfect, but you get idea. [LAUGHING]

AUDIENCE: [? No, ?] [? it ?] [? was ?] [? a ?] [? bit ?] harder.

JAVIER: OK. So it was harder because of what is called, what we know it's called depth perception. OK? Another important piece of information that we can get out of this is that pirates don't do well doing photogrammetry.

[LAUGHTER]

Right? So throughout the presentation, we going to cover things like that. But more about applications, pros and cons. How to capture quality data. And then how to process it. And then how to share the results and how to analyze them.

But first, some introductions.

DAT LIEN: Yes. So let's go out, talking about natural disasters about eight years ago. Haiti probably had one of its worst earthquakes ever. Probably the worst-- a really bad earthquake. Javier, want to go to the next slide?

And out of disaster a lot of time comes--

[VIDEO PLAYBACK]

- --greatness. The president of the Pest Control Society was traveling back and forth between Houston and Haiti. His name's Raleigh Jenkins. And he felt like there was more that needed to be done. He was putting up screens at hospitals, just making sure the sanitary conditions there were a little bit better.

[MUSIC PLAYING]

And so he saw all these kids running around without parents and without supervision, right? And that's when he asked, where are these kids going to sleep tonight? And they said, well, they'll probably find somewhere to sleep.

And then that's when he found out that Haiti had over 20,000 orphans. And still today, there's thousands of them. So Javier and I we're lucky enough to visit Haiti and to see the construction of one of the first A Child's Hope orphanages, or homes for children.

And this is not just any orphanage. It's a very Christian-based home that these kids will grow up in. They're going to get all the training they need to grow up to be leaders of their own of their own country.

And the reason why I bring up this story is because it's the heart of what [? Axoscape ?] stands for. We're all about solving problems and we're all about helping the community out wherever we can. Like I mentioned before, what is technology if you're not able to use it for good, right?

My name is Dat Lien. I'm with an organization called Axoscape. And for six years, we've had the pleasure of helping architects, engineers, and contractors and owners get on with technology, BIM, and things like that.

What we do is we hire college grads. We don't hire a whole lot of experience folks, but we hire college grads. And we train them to become the BIM specialists of tomorrow. We give them the experience that they otherwise wouldn't get or very difficult to get. So we're very proud of our team members.

Speaking of one of our team members, Javier is from Ecuador. And he's got a very interesting destruction story of his own doing.

[END PLAYBACK]

JAVIER: Well, I'm guilty, but it wasn't my fault that my dad [? stapled ?] this two-year room we were in. I was just a kid, and he just walked away. And this small, red, shiny icon was on his screen.

And after a couple of clicks, I was able to convert those polylines, block, hatches, in the smallest and simplest forms. That was my first interaction with the explode command in AutoCAD. Just being a little kid.

So fortunately, I was very pleased, but my dad wasn't so much, of course. After a couple of more years, some of that curiosity took me to college and I pursued a civil engineering degree.

And then I found myself in Houston as a grad student looking for more insight in this thing called BIM. I don't know if you've ever heard about it. I met Dat at a Revit user group. And yeah, that's how the story begins.

DAT LIEN: Yeah. So another member of the user group, his name was Christian [? Tenwall. ?] His name is Christian [? Tenwall, ?] if I can say so. And he saw some of the presentations that we were doing about photogrammetry and things like that. And he said, hey, guys. I want you to go scan my house.

And we were super excited. We got there. We grabbed our GoPros and we just went nuts. And we captured over 800 pictures. Two videos. And we hurried back to the office, started processing all the images, and we realized it didn't turn out as great as we thought it would be.

So a little more on that in a little bit and the reasons why it didn't do so great. But I am pleased to announce that Christian is actually able to join us here at AutoDesk University today. So he's right here in the front. Christian, why don't you grab a microphone?

Now the reason why Christian had us scan his house was because it was destroyed during Hurricane Harvey. And Christian, why don't you tell us the stories of how your house got to this state?

CHRISTIAN: So we got about 49 inches-- or not about, but we got 49 inches of water in our house. And it was there for 16 days because the Army Corps of Engineers kept the gates to the reservoir open.

And so we couldn't get back to the house until they slowed the release rate so the water would drop. So I haven't seen this in while. It's kind of shocking. It looks a little different today.

We're not back in yet, but we hope to be back in by the end of the year. Fingers crossed.

DAT LIEN: So tell us what happened as water started to come in, and how you guys evacuated, and things like that

CHRISTIAN: Yeah. So the rain, it just wasn't stopping. It just kept raining. It felt like it had been raining for weeks. And we tried to move some of the stuff. It was just me and my wife. Kids were asleep. We were trying to save whatever we could.

And so our nice, heavy furniture-- we couldn't move that upstairs. So we had the cheaper IKEA stuff and went upstairs. But first, when the water came in the house, I made the decision-- there were some people already in the neighborhood that had canoes out.

And so I asked one of them to take my family out, and my dogs got in the canoe, too. And then I stayed behind because I wanted to kill the electricity and grab the go bag. My wife's super organized, and so we have a go bag with all our important paperwork in there and our passports and all that stuff.

DAT LIEN: And just describe the water really quick, and how clean it was.

CHRISTIAN: Oh. Well, you know, initially the water coming in, it didn't look so bad. But there was a treatment, a sewage treatment facility adjacent to the reservoirs that was flooded pretty much immediately.

[MUSIC PLAYING]

And so there was a toxic soup. There you can see the Addicks and Barker reservoirs. And so I live just east of them kind of in the corner over there. So we got some good toxic water into the house.

DAT LIEN: So your house was along the Buffalo Bayou over here around this area, right? When they released the dams to the Addicks and Barker reservoirs?

CHRISTIAN: Right.

DAT LIEN: That basically flooded thousands of homes down the Buffalo Bayou, which flows through Houston and now the ship channel.

CHRISTIAN: Right. That's correct.

DAT LIEN: Yeah, that's crazy. So when did you guys actually start cleaning things up?

CHRISTIAN: Well, I guess on that 16th day, we couldn't actually drive in yet. But we boated in. I've got a friend who's a landscape architect and he offered up his crew. So he and eight or nine guys came in and they took off all the drywall and they took out all the stuff from downstairs because it was all pretty much ruined.

So the water went up over the counter tops. So when we initially were putting things up high, we thought, oh, we'll just move it up a shelf. And so we learned we should have moved everything upstairs no matter how small or how big.

DAT LIEN: Had you ever thought in 1,000 years that your house would flood like that?

CHRISTIAN: No. The house has been there since 1965 and it's just never flooded. So this is an extraordinary circumstance. So yeah.

DAT LIEN: Yeah. That's crazy. So that's when you decided it was a good time for us to do some laser scanning or photogrammetry at your house?

CHRISTIAN: You know, going to the BIM user group meeting and then talking to you, I was like, oh, gosh. It'd be super helpful to get some layout of my house so I can get back to-- or I can use that to create some drawings and hand it off to somebody to go with it and give me give me a number how much it's going to cost to build back.

DAT LIEN: Yeah. So that's great. So yeah, super happy to have Christian take a break from all the construction work that he's doing in his own home and join us at AU and just take a break from all that.

CHRISTIAN: It's great.

DAT LIEN: So we're glad you're here. Yeah, for sure. Hopefully you can enjoy some of the classes. We're going to revisit with Christian here in little bed. Touch up on where his house is today.

CHRISTIAN: Thanks, Dat.

DAT LIEN: Yeah, no. Thank you. But really quick, let's go back to the scanning issue that we had earlier. Javier?

JAVIER: Yes. So the process that Dat attempted to do at the beginning was taught in [? structure for ?] [? motion ?] [? of ?] photogrammetry using photogrammetry concepts. So we're going to run our-- fourth, third, experiment of the day. I can't remember.

DAT LIEN: This is old school BIM for those guys who, you know.

JAVIER: We did this is in the hotel room. So just to explain real quick.

DAT LIEN: We need a couple of volunteers.

JAVIER: She's a very good volunteer.

DAT LIEN: Anyone? Anyone?

AUDIENCE: [INAUDIBLE]

DAT LIEN: Hold this rope.

JAVIER: Yes. So just to explain real quick what's going on in the back end. So let's say that you're pirate number one and you're going to be pirate number two. Is that OK? Yeah. OK. So those are lines of sights and those are cameras taking pictures of the same area right here, right? Those are pixels that they're being shared at this moment.

So because we know the location of those cameras, we're able to infer where this point is located in a 3D coordinate system. So those cameras are recognizing patterns and features due to advancements computer science and computer vision. So that's how they can find matches between the images and inferred geometry or do the 3D reconstruction.

So a couple of things that Dat did good was that he was using wide-angled cameras. He took a very large amount of images. But a couple of things he didn't do well is that he was walking and some of the images were blurry. So that introduced some errors to the model.

The lighting wasn't good. And he didn't take pictures in a consistent pattern, per se. So photogrammetry for interior is very challenging. Hopefully around us-- I mean, around us-- we are lucky that we have a lot of partners. So the software can recognize those features and create those unique features and do the matching with the other images.

DAT LIEN: So we went back a second time.

JAVIER: So we went back a second time. And he was more prepared. He took a tripod and he was walking and taking pictures as he walked in a more consistent pattern.

DAT LIEN: Got much clearer pictures with just this thing.

JAVIER: Exactly. So what else did you do? Can you tell us a little bit?

DAT LIEN: We also went in a very methodical pattern, making sure that we had plenty of overlaps. I think the first time we did it, we were really maybe stepping too far in-between. And so you really kind of got to take baby steps. Even though it's going to take a little bit longer, but the results were a lot better.

JAVIER: Yes. So what he was trying to attempt there-- remember those shared pixels I was talking about before-- is to try to have a good overlap. About 90% overlap between each image that he was taking. So if I take an image here and capture in this area here and then I move and take the next image, I want to make sure that there is a redundancy of 90%.

DAT LIEN: Here's a preview of what Christian's house looks like if you guys want to see it.

JAVIER: This is the bad preview.

DAT LIEN: This is the bad preview.

JAVIER: So that's--

DAT LIEN: As you can see, we took almost 800 pictures and the photogrammetry process only used 100 of those-- about 13%, which is really bad. So we only caught a portion of the house. We'll go to the next one so [INAUDIBLE].

So you guys can scan this QR code. You can experience, the better of the two models. The better of the two point clouds. I'll give you guys a moment scan that. You want talk about control points really quick?

JAVIER: Yes. So the other thing that Dat did good at this time was that he placed those controlled points on the walls because they break up the lack of randomness that we get on interior walls.

DAT LIEN: Yeah. This looks like a QR code, but it's a control point.

JAVIER: So control points-- and also, because usually for interiors we don't have the location. We will have a GPS receiver so [? we will ?] have locations of the cameras. So the model comes without a scale, so we cannot measure things. So we have to use different ways, and one of them is placing QR codes to do the measuring afterwards.

DAT LIEN: You can print these out from certain sources. I think Pix4D had these control points for us.

JAVIER: Yes.

DAT LIEN: And the funny thing is we had these control points all situated around the house. And Christian, I remember your wife saying the contractor showed up and they were tearing things down and they didn't know what to do with them. They were like, well, can we move this?

CHRISTIAN: They left them up.

DAT LIEN: They left them up? That's great. So real quick, was everyone able to follow along with the QR code exercise? So it's pretty cool. You can spin around that particular point cloud model. On the top left hand corner, you can control the density of the point cloud. And you can do even some quick measurements, too, as well.

So as Christian's at Home Depot or whatever wondering how much drywall to get or if something is going to fit, he can just pull up his phone, go to this link, and just do a quick measurement and say, hey. Yeah, this is going to fit.

So after we did the interior stuff, we went outside. And what we saw was even worse. It's really bad. Similar to the videos that you saw earlier on. So we went and flew the drone to kind of get some video footage of what was going on.

And as we mentioned before, Christian had to rip all of this stuff out of his house. And you can see all the floorboards, all studs-- anything that wasn't structural he had to rip out. And when we we're talking to him afterwards, he even mentioned that there were things that he didn't really want to throw away but he had to threw away because the water was infested with, you know, God knows what.

CHRISTIAN: I can still smell it.

DAT LIEN: Yeah. And the smell was so bad. I mean, I got out of the car and I wanted to hurl right there. Everyone was walking around with masks on, both inside and outside of the house. It was pretty horrendous.

And then it was kind of at that time that we started to realize that this is huge. This is not just a-- Christian's family is just one out of many families, thousands and thousands of families. And if you look around the room, what we did to kind of help honor some of these victims that are still rebuilding today, the stories of where they are, and how they got to where they were, and just to remind ourselves that we can be pretty vulnerable.

And it doesn't matter how rich you are, if you live in a poor neighborhood, in a rich neighborhood. You know, Mayor Bill White-- he had to evacuate his home. And he was seen carrying his bag of clothes, trudging through water, being rescued by boats. So really, you're not you're not immune to all this at all.

And so what we really wanted to do there was to do some exterior photogrammetry, but we just didn't have time and the situation wasn't right. So we want to show you some examples of exterior photogrammetries on a city-wide scale that Javier has done in the past.

JAVIER: Yeah. So as we stepped out and we realized that everybody was suffering from the same things, back in Ecuador, I was introduced when I was in high school to this concept of cadastre. So basically, it's measuring or collecting data of the real estate assets of the municipalities so they can charge fair taxes. So this brings equality to the community because everybody is charge based on what they actually own.

So a couple of things that we can learn from this. I'm going to start throwing at you a couple of experiences and tricks, and tips and tricks in general. So this is what we handle after cadastre projects. It's just a drawing, but we handle geographic information systems to the municipalities so they can perform the evaluation of the real estate assets.

So in terms of the vehicles that we want to use, we're going to start talking about the vehicles and the payloads. So we want to use fixed wings when we're talking about small to medium size cities, right? Because they can cover more space. We can cover more area.

They can cover about a square kilometer per flight. In contrast, we have this type of aerial vehicles that Dat is going to talk a little bit more about it. But so everybody in the field counts, so we want to make sure that we get the most out of it every time that we're in the field.

So these are examples of sensors that we can place. A fixed wing gives us a little bit more flexibility in terms of the camera that we want to use, and we can go from the visible spectrum to the non-visible spectrum.

So here, a little bit about preplanning. So this is in the office. So the polygons that you see there that are popping up-- what we're trying to attempt is that those are an area not that big, so we can fly it with one battery. And of course, we need to make sure that those polygons are overlapping between each other.

The other thing that we have to consider is if we have mountains like this. What we have here is that we are flying on an altitude that is going to-- we have to make sure that we're going to fulfill the requirements in terms of the minimum resolution.

It depends on the job you're doing. Could be 5 centimeters per pixel, 10 centimeters per pixel. But anyways, it's way better than Google Earth, right?

DAT LIEN: There are no mountains in Houston, are there?

JAVIER: Yes.

DAT LIEN: But there are power lines. Lots of power lines and other things that could mess up your photogrammetry.

JAVIER: So this is the way that usually the aerial vehicles fly . They do a grid and they start taking pictures. This is, we're trying to represent here is they overlap. We usually want to have a 70% upfront overlap.

So what it means is that when you send the data to the aerial vehicle, it produces the-- display this again. It sends the data. So the autopilot is triggering the camera based on when we have to take those pictures. So we made sure that we have enough overlap. And the second part is the side overlap which is around 60%.

So another thing that we've learned is that on those polygons, in the border of those polygons where they match, usually you might have change in light because you're not flying at the same time all the time. So make sure that you're flying between hours that are not too early or too late.

Another thing is that we also have here-- remember those control points that we placed on the wall? We can put it on the ground as well. So we place them with high-precision instruments. Differential GPS receivers. And make sure that some of them, they are shared between these areas that overlap-- between the missions that you're flying.

Another thing to take from this is that when you're performing a job, usually we have to do it for yesterday. So we don't want to wait until the end to process all the images. Some of these projects we worked in the past, they can go from 10,000 images to 20,000 images. And so we process them separately and then we put it together.

And as I said, it's important to-- you want to start digitizing. Wherever you're mapping, you want to do it in small chunks and then put it together. But make sure that in the borders there is not a lot of shift, because two separate processes don't make one process.

So here we say just, we can see that, of course, at the perimeter of the polygon, we have less overlap. The green is best overlap. The red is bad overlap.

DAT LIEN: So another example not quite as complicated as scanning an entire city, but scanning a smaller coastal community. This particular example comes from us. It comes from Texas A&M in Corpus Christi. And that's my Alma mater. I graduated from A&M as well.

So it was great that they were able to share this information with us. And Professor Joon over there was gracious enough to do so. But anyway, what they did was they used old scan data and then they overlapped, they overlaid some of the new scan data on top of that.

And on the right hand side during this web interface, what you can do is you could turn on and off certain times. And so go back in time and be able to see exactly what happened and how the hurricane impacted this particular community.

So this was before. And so this was right after. You can see the blue tarps on some of these buildings here. And then as it progressed, more blue tarps as people were doing their renovation work. And then finally, some of the buildings that were so bad were actually torn down.

And so this is an important concept, because without this imagery it'd be very difficult for us to really assess the damages and whatnot from the hurricane . We'll talk a little bit more about some of those applications in a little bit.

JAVIER: So remember, we were planning our flight. So now we can use mission planning software that will allow us to send the data to the autopilot so it can fly. So I know that Dat is a very good RC pilot but I don't really want him to fly my drone. Because he he's not going to make sure that those lines are not-- the separation between them is not perfect, so we can ensure the proper overlap and whatnot.

So a couple of pieces of software. The one that we've been using lately is Pix4D Capture. Another well-known open source one is Mission Planner and QGroundControl.

DAT LIEN: As Javier mentioned, there's all these parameters, right? So it's important that your mission planning software has all of the parameters of your vehicle as well as the parameters of the camera, the payload that's on there, in order to ensure that proper overlap.

JAVIER: Exactly.

DAT LIEN: As I was [? tweaking ?] those sliders, it was able to give you more lines based on the desired overlap. And then it also tells you that, hey, you're about to run out of batteries if you try to do this in one mission so you might have to do two missions.

JAVIER: So one important input is that we know based on what is it resolution that we want to get after the aerial image, we need to-- it's going to program how high we're going to have to fly in order to get those minimum requirements.

DAT LIEN: And that was part of the reason why we didn't fly at Christian's house, because we just didn't plan it ahead of time and we didn't want to just take the drone in the air without doing all that. So we've got a lot of images now. We talked about doing scanning and interior applications. We talked about scanning on an exterior application and best practices for some of those.

And so what do you do with all those images? So naturally, you need to put them into a program to process them. And you're probably familiar with this program. It's probably a more beautiful rendition than you think of. But This is ReCap. ReCap Photos, to be exact.

So basically, you could take something like an object on a table and you walk around the object. And as long as you get that overlap, you can reconstruct something in this amount of detail very quickly and very easily. And ReCap has some limitations. There's about 300 pictures that you can upload when it comes to objects.

When it comes to aerial images, you're limited to about 1,000 pictures or so. It's a cloud-based process, so you're using your Cloud Credits. It's cool, though. If you've already got a subscription, it's already free. And that's a great Autodesk software.

For some of the stuff that we use, we use a slightly different program, though. And--

JAVIER: Yeah, but before going to that point, I just want to tell that there is, under the [? 4G ?] platform, there is a reality capture API. So if you're familiar with sending HTTP requests, you can, in five steps, you can upload your photos, create a photo sync, and then get the results of other programs. They made it really easy.

So the other piece of software that we've been talking about for larger projects is Pix4D. It's basically the same process. Upload the images. However, it gives us more flexibility in terms of the options that we have. How to configure the results and the type of results that we want to get in terms of the resolution and type of formats.

So here's just a little table that is a comparison table matrix.

DAT LIEN: And we talked about the limitations of some of these other software. But if you want to go really high-end, you can go ContextCapture. Super high limit.

JAVIER: Well, nothing is no limit, of course. But this is the software that can get more-- can process more data sets. OK? Something happened here.

OK.

DAT LIEN: OK Yeah. So we talked about acquiring the images. We talked about processing the images. And now we're going to talk about how you can share some of these images, as well as what some of the output that you can get.

So this is some pretty exciting stuff. So the computer's been churning. It can be churning all night long. It can be churning for two or three days in cases where you're scanning entire cities.

And so what do you get out of that output? So first of all, you get something called a point cloud like we mentioned before. Basically, it's the triangulation of every single little point.

Now out of the point cloud you can get a number of other things as well. So one of those things is a 3D mesh. And a 3D mesh is nothing more than just connecting those dots to form triangles we call faces. And with faces and surfaces, you can bring them into rendering programs and you can get the nice, rich 3D object that we're talking about.

And something else you can also get is something called an orthomosaic. So an orthomosaic looks like it just stitched all those pictures together. But actually, the orthomosaic was created from the point cloud. And it's a very nadir shot, or a straight down shot, so you don't get any of that parallax. You don't get any of those issues with buildings shifting.

And then finally, you get something called a digital elevation model, which only civil engineers can understand.

JAVIER: Yeah. So some of these things are a result of the point cloud. And these elevation models, we can have the [? utilitarian ?] models, which basically, we didn't have buildings or vegetation. It's based on the point classification. And also, we have the detail surface model, where we include all of those objects as well.

I'm having technical issues with this.

DAT LIEN: Oh. Yep. So how do we share this information once we've got it? Because these files can be pretty large. So if you can, go ahead and scan this QR code.

JAVIER: So we don't want to send an email with a point cloud because it would be a pretty large file. So what we're going to see here is a mesh. We upload it to a platform called Sketchfab. It's a good way to share meshes, texturized meshes. And what you see here is a water treatment plant.

DAT LIEN: Yeah. So it's very similar to the water treatment plant that Christian was talking about earlier, the one that flooded and all the muck and everything else came to his house. And you can see why. A lot of the water treatment areas are just exposed. And that's just part of the aerobic and anaerobic process of cleaning that water.

And so when this floods, you can imagine, right? All that gunk is going to go out. And we did this scan really early on. And so this particular water treatment plant was able to use some of this data in order to provide training for their guys and in order to just be able to communicate what they do a lot better. Another example here.

JAVIER: Yeah. So in the next example, we're going to see Potree. It's the way that we've been sharing the point clouds with you. It basically converts LAS, LAZ files into WebGL that can be viewable on the browser.

DAT LIEN: So this is the Child's Hope orphanage example that we showed at the very beginning of the class. And the importance of this is this project is all the way in Haiti. So imagine if you were a solar designer and you had to design solar panels for this particular building.

So you went over there. You did some measurements. But then upon returning home, you realized you forgot a few dimensions. And so what we're able to do is we're able to provide this point cloud to Freedom Solar, who helped install the solar panels and provide electricity to the mountain.

And they were able to use our point cloud to do some of those extra measurements. They didn't calculate how much conduit that they would have to need to rise over to the top of the building. And there's a power building down below, so they had to dig underground conduits in order to bring power to them.

Those are just some of the things that they just forgot to measure. And so luckily, we have the photo scan so they can take those measurements after the fact.

So let's talk about how some of this data, all this information can actually be used during disaster recovery. If there's one thing that's good about hurricanes it's that we're pretty good at predicting when they're going to be here. And it's giving folks a little bit of time to prepare ahead of time.

So you can see-- this was actually taken from Harvey. So Harvey hit just to the west of Houston, which was terrible because that kind of put us on the, quote unquote, dirty side, if you will. And so that's what caused the torrential rainfalls for four days.

Something like 15 trillion gallons fell on the city of Houston. Can you guys picture 15 trillion gallons? I can't. So that's like having Niagara Falls just parked on top of the city and just dumping water on the city for 19 hours. But this was spanned out over four days and flooding the bayous.

So preplanning and preparation we've talked about over and over again. But if you can do a good job with, like for instance, this particular project in Miami where this particular area floods pretty frequently. What they've done is they've combined Lidar data along with photogrammetry data in order to recreate most of the city as they can.

And then they were able to do some simulations with storms. And in this case, you can see what happens when a category 2 versus a category 5 storm surge were to go through the city. And so it's very important that, if we take it as it is, then we're not going to be able to get a lot of places without thinking ahead of time.

So another thing that we did was we kind of did some photogrammetry of the Addicks Reservoir. That's one of the reservoirs that they haven't released.

JAVIER: And we just have a small example here. But before going to that, I just want to mention that we love Lidar. We love laser scanner. In general, that technology is way more precise than photogrammetry and [INAUDIBLE].

But what we are trying to conceptualize here is to-- it's about the location, and let the people grab those tools so they can make it themselves. And they can, whomever is affected by the natural disaster, if they can be educated, they can make good use of the tools. And they can create maps of their communities, their neighborhoods, and they can be more involved in terms of the [? plannification ?] of their spaces in general.

So here's just one example of a couple of cross sections of the berm. So one of the applications the [INAUDIBLE] engineers can do is slope stability analysis based on these sections and whatnot, and calculate volumes and in general, all those things that we can do and see with 3D.

The other example-- so we are all about graphics and using geometry to visualize things. So another good thing that we can do in Civil 3D, [INAUDIBLE]. And Revit is that we can import those Civil 3D surfaces into Revit. And we can create a toposurface based on good, quality data.

We don't have to worry about those triangulations in the toposurface that usually they don't have brake lines or things like that. This is much better.

DAT LIEN: So essentially what we're saying is that this is not that difficult to do. So if you look at Lidar, you look at laser, you look at some of the other methods, the equipment can be pretty costly. But what we're saying is that pretty much anyone can do photogrammetry. All you need is an input device. All you need is a camera.

And so here's another example of what you can do if you were able to capture the data. So this is Google Earth data. And then on top of that is the higher resolution orthomosaic that you can get from a scan-- from a photogrammetry scan.

Something else you get as a side effect or as an extra added bonus is the ability to spin around and do analysis on destruction and things like that. So again, this is the same Texas A&M example that we saw earlier. But now you're able to really assess the damage just from one flight. There's a lot of details to be had there.

So that's before the hurricane. So what happens during the hurricane?

[INAUDIBLE]

So there were a ton of rescue efforts that were happening during the hurricane. And you can see here a pretty dramatic rescue effort. Why didn't they have boats in this particular area? Probably because the waters were probably flowing at a rate that was too dangerous to have boats, so they had to bring in helicopters.

But we were sitting there at night watching the news. And I live up north of Houston. And as the rain was coming down, you can hear reports of people just getting trapped in their homes. They had to evacuate to their attics and to the roofs, and things like that.

And so what they ended up doing was they started forming ad hoc call centers and rescue centers using nothing more than just laptops and their cell phones. And because of that, they needed some way to kind of help organize the efforts.

And so they created a website.

[INAUDIBLE]

So that what they did was they created a website called Neighbors Rescuing Neighbors. And as you could see, they relied heavily on the Google Map data in order to identify that, hey, via social media or via phone calls, hey, my grandma's stuck. I can't get her out. she's there by herself, and things like that.

And so everyone else was able to tap into this website in order to rescue both rescuers and people who needed rescuing. And we just can't help but wonder, what if they had a lot more information and would have had more updated maps? More detail and higher detailed maps? Some of those efforts could have been better.

Another thing that happened was the water was freezing cold. And so there's boats that were just relentlessly just going up and down neighborhoods looking people. We talked about payloads. And what if you mount an infrared camera on-- drones were not allowed to fly at the time, so you couldn't use drones.

However, you can mount them on helicopters. You can mount infrared devices on boats and other rescue vehicles. And you can easily see that a first responder could easily spot someone that's either hidden amongst trees or shrubs, or on top of the roofs, or possibly even in their homes and whatnot because electricity is out, and water is cold, and bodies are warm. You can quickly identify some of those contrasts.

So then let's talk about the recovery efforts and the cleaning efforts. So one of the biggest complaints from the residents is, Christian included, was that no one would come and pick up all of their debris. And so the city had no idea in terms of which areas to focus their efforts on.

So imagine if we're able to fly drones right afterwards and then be able to do all that all that great imagery. And then they can identify, the decision makers can quickly identify, hey. We need to pull all of our efforts into this in one area because there's a ton of debris here and we can easily see that.

So what you're seeing here is a non-public map released from Solid Waste Management. I just happened to find it in Google. But what this indicates is the gray areas that didn't have any coverage, but some of them that needed coverage. And then some of the colored areas where there were different levels of efforts going on.

And if we zoomed in a little bit more, we can see how many trash trucks that they had in those areas. So there's a ton of logistical issues that we think that photogrammetry and images can help with some of these efforts.

So we have a problem. So we can't scan the entire city ourselves as much as we would love to. And so we have to ask the neighbors to do it. Your family, your friends. Anyone with a drone, anyone with a camera. Anyone with a GoPro cameras to do it. To us, that seemed like the most logical way for us to help with some of these efforts moving forward.

JAVIER: Yeah. So it's about empowering communities. So here's an example of how we can create our own server using Amazon Web Services or Google Cloud. And we were able to install what is called-- this piece of software is called OpenDroneMap.

And we can create our server, process our own imagery. And afterwards we can share it, and we will get similar results-- point clouds, meshes, orthomosaics-- out of it that we can share. And we can create our own platform based on open source efforts that we can use it for communities in general.

DAT LIEN: Anyone use NextDoor? Does anyone have the NextDoor app on your phone to find out what's going on in your neighborhood? So imagine NextDoor but for mapping, essentially.

OK. So let's revisit Christian's house. So we recently visited him prior to this presentation just to kind of get an update on how things were going. And lucky for him, the drywall started going up. Not so lucky for us because it made the scanning of his home a little bit more difficult.

JAVIER: Yeah, I do remember in the second scan the studs were exposed. So that was easier for the software to process. In this case, the dry walls were placed and they were all just one white color. It was more challenging.

However, in this case, we used 360 degree cameras. And that really helped in terms of the areas that we were able to capture. The density of the point cloud was not as great as the first one. In the first one, we had around 16 million points. On this one, we have around a million points. But in terms of the area that was covered and the accuracy, it was better.