How to Create 3D Digital Cities Using Drones

PRESENTER: So this application is very helpful, especially in different scenarios for urban growth of the city, and at the same time evaluating different alternatives for having any new development inside the city.

OK. Now we will talk about different level of details of any building. By the way, you can ask any question at any time. OK? So we will have open discussions.

So there is a standard definition for the level of details of any building inside the city. Whenever I talk about 3D digital city, it comes to, how much details do I need to have in our 3D model? So there is a standard called CityGML. It's a well known standard to define how we are going to build and model our cities.

Actually, it comes with four different level of details. Level of details number 0, which is basically the footprint, 2D footprint, of all the buildings. Level of details number 1, it's extracting footprints to a certain height that represents the mass of the building. Level of details number 2, we start adding pictures, and we start adding details of the roof, just the outer form of the roof. Level of details number 3, actually here we model the full details of all facades and roof. So I model all the opening, doors, windows et cetera. For a level of details number 4, we start adding more details for interior buildings.

In this diagram, actually we are showing the relation between level of details and the interior space. So for level of details number 1 and 2, we don't have any details inside the buildings. But level of details number 3 just reveal the openings. You can see the openings from inside the building. But level of details number 4, it comes with full details, exterior and interior.

So inside citygml.org website, you have different samples and different level of details. For example, we have here New York level of details number 1. And once you click on any building, you will reveal some information. We will come to this after a while. Helsinki, it's another example, but for level of details number 2. What I have, just the massing of the building with the roof details and pictures.

What about drone? How can I use drone to build my 3D digital model? So actually, it's very simple process, three steps. Step number one, scanning. I use my drone to scan the whole city. But I have to consider different things. First item we need to consider and we need to take care of-- while I am scanning the city, I have to scan it using drones with a specific overlap in two directions, x and y. And the more you have in your overlap ratio, the more details you will get, and the more accurate model you will have. In our examples, and in our standards inside our company, we have overlap 60% to 80% in x and y.

After you have thousands of images, we combine all these images together using a specific software in step number two, which is processing. So I have here tons of gigabytes of images. Using a specific software, we combine all images to come up with different outputs and different products. One of these products-- this image, we call it also photo image-- this is a georeferenced photo for the whole city, true color, and rectified, so I can use it in any GIS system.

After I have this kind of output, I can extract a 3D model, a 3D autodetected mesh, out of the data. And actually, the example you see right now, this is a real example from a city in Saudi Arabia. It's called Diriyah. Diriyah, this is the oldest town and historical government town inside KSA. And this is a real model of the area with accuracy 2 centimeter x and y and 5 centimeter in z directions.

OK. Falconviz. So I will give you a small brief, short brief, about Falconviz. Falconviz is a drone company specialized in aerial surveying and mapping and for true 3D development. So I was lucky. I start with the company four years ago. At that time, we have just one small drone. And it was a very basic drone at that time. Now we have a fleet of drones.

And drones come with different types. We have multi-rotor drones, like this one and this one. So it comes with quad-- four motors-- or maybe hexa-- six motors-- and so on. And there is another type, we call it fixed wing, like the orange one and the white one. And they have different applications.

For example, we use the fixed wing drones to have a huge area scanning. And for this type of drones, I will be able just to capture nadir images. But for other type, which is multi-rotor, I will be able to capture nadir and oblique images. We will come to this different types of capturing after a while.

At the same time, we have different applications for each drone. For example, this drone, we call it Hexacopter. We are using it for oblique scanning and high precision scanning. But for this one, DJI [INAUDIBLE], we are using it for different cinematic shots and aerial shots. So each drone, it comes with its unique applications.

Actually, this is our CEO, Neil Smith. He is one of the co-founders, and he is a research scientist inside KSA, King Abdullah University of Science and Technology. And this image actually in front of our visualization lab inside the university. And Neil trying to play with our models in front of the screens.

Up to now, we have been engaged in more than 100 projects across the region-- Middle East. And we have been lucky to work with these kind of clients, actually government clients, consulting clients, semi government. So we have been very lucky to work with-- and owner, actually, to work with these names.

OK. Today we have two case studies. The first one, it refers to Al-Balad, which is the historical area or historical downtown inside Jeddah, the second big city inside KSA. The total area we cover, it's more than 200-- 2.6 million square foot. And the level of details we already model was level of details number 3. And we use nadir technique and oblique technique. And actually, the main purpose of modeling this city is cultural heritage practice.

So as a workflow, very easy, very simple. It's just five steps. Step number one, pre-scanning, which include activities that we need to have before we go to the field and scan the area. Scanning using the drones. Processing, actually this include activities that's already being done inside the office to make sure that we extract the accurate output from the images that we already scanned in the previous step. Then modeling and visualization.

First step, pre-scanning. So after I have my drone, and we are ready to go to the site, we have to make sure that we select accurate positions for what we call ground control points. What is the ground control points? It's points inside the city where I use it while I am doing my processing to make sure my model is georeferenced.

So I am using this ground control points to georeference my model at the end of the project. So without this ground control points, I will have the 3D model, but it will not be in its real location. It will not be georeferenced. So it's very important to define these locations before I go to the field.

And we will stake them out using our GPS device. Then I got all these locations as coordinates, as real coordinates. And I have it in my Excel sheets, or I will save it as CSV file. Then I use it in the processing step afterwards to make sure that my model is georeferenced and true model.

At the same time, before I go to the field, I have to design-- we have to design the best effective path for the drone to make sure that we cover the whole area, so we don't have gaps, and at the same time, we capture it in less time with the maximum number of images, as I already mentioned before, with the specific overlap in x and y. So this is animation diagram, just to show you how we are capturing Al-Balad using DJI M600. And we did nadir and oblique captures.

So we used just one drone in this project, and we capture the whole area in one day, with two different types of capturing, nadir and oblique. So the camera inside that already attached to M600, it gives us the ability to capture nadir and oblique images. So nadir images or nadir capturing, actually it's very important to capture the land features, and at the same time all buildings' roofs. But for oblique, this is very important to capture the facades detail of each building. So this is representing the oblique capture using the drone. And from the nadir captured, I got all the land features and all the building roofs.

So it was eight oblique flights. So total was 10 flights. And we did it in one day. Eight oblique flights, more than 4,000 images captured. Two nadir flights, more than 1,000 images. Total images was 5,500.

Processing. So now I got all the images from the drones, and it's already geotagged. So for each image, I should know where this image already captured inside the city. So we use different softwares to have different outputs. We use Pix4D to process all the nadir flights, to extract the orthophoto. At the same time, we used reality captured software to process all the oblique flights, which was more than 4,000 images, to have a 3D automatically generated mesh.

So here we got the 3D mesh model, and it comes with very high very high accuracy and texture at the same time. We will see a video after a while. So this is a screenshot from Pix4D software, where I have all the model. And you can see here on the blue and green color balls. So for each ball of these small points, it represents the image that already captured from this location. And actually, in this specific screenshot, we were-- during the processing phase, we were just about 42% of the processing. And we got some initial results of the whole area. The results here would be orthophoto, as I mentioned before.

And at the same time, a 3D point cloud file. How many of us know about point cloud? OK. So as point cloud, you already know about it. It's a big file. It comes with gigabytes of data. It holds full information for each point in the area, which is x, y, and z of the point, and at the same time RGB, or the color value of this point.

So as output-- this is a map of the area. All the highlighted buildings, this is a historical area. And this is one screenshot of the exported or the extracted point cloud of the area. So it was more than 500 million points. The data was about 50 gigabyte of data.

Yeah. So the orthophoto we are generating and we are extracting from our data, actually it comes with very high accuracy and very high resolution. We always have questions from our clients. What is the difference between your orthophoto and a satellite image?

In a very simple way, the satellite image, in best case scenario, it comes with 30 centimeter resolution. In our case, it comes with 2 centimeter plus or minus for sure centimeter resolution. So you can tell the difference between two images. And for sure, the satellite image has a specific group of applications. But at the same time, when it comes to high detail precision scanning, we are talking about using drones.

Modeling. In this case study, we extract two different types of model. The first one is automatically generated, which is a 3D texture mesh that we already generated from reality capture software. And at the same time, the client asked us to provide them with a high detailed 3D solid model for different applications, for selected buildings.

We will see two types right now of this model. So I will start with the first one, which is a 3D texture automatically generated mesh. And what you can see here is this is the full area, the whole area, exported as a 3D mesh OBJ file with texture. And this is just a visualization of the whole area. And you can tell how much details we can capture using the drone technique.

[MUSIC PLAYING]

Actually, the area was very challenging because this is very old area, historical area-- very narrow streets, very narrow lanes. It's not easy to capture by drones, but we did our best, and we succeed to capture all the details. For example, you can separate different buildings, and you can start adding different layers-- for example, adding different analysis for building heights, building measurements, extracting pictures. So it has different applications. OK.

That was one example of the first type of the 3D model that we extracted in this case. Another example, the client already select two buildings to model a 3D solid out of it. So we use 3D Studio Max to model post buildings. The first one is Bait-Al-Batarjy. This is the first US embassy in KSA. So actually, the modeling phase here took two working weeks to model the whole building in details, just two facades of the building. This is just the work in progress. And this is the final output of the building with real pictures that already captured from drones.

For sure, after we finished the model, we did short animation to reveal all the details that we already modeled for this specific building. How many of us already visited KSA? OK. Very good. So actually this building style is-- they call it Hijazi style, which is the eastern coast of Red Sea that relating to Arab Peninsula. So this is very unique architecture style.

Another building, which is Bab Jadeed-- so basically, it's the old town. It has different gates. This is one of the old gates for the old town in Jeddah. And we used 3D Studio Max and for sure REDCap to model the building. As you can see here, this is the model without texture, after we add in the texture, and this is as-built drawings that extracted from the building using AutoCAD. Another visualization, just to give the client an idea about how we visualize the 3D model inside 3D Studio Max.

So actually, after that, the client asked us-- I just have to lower the music. OK, no problem. So the client asked us to have interactive presentation of our final data. So we did what we call Al-Balad virtual museum, where you have this interactive map and all the important buildings already highlighted. So you can select whatever buildings you need to visit, and it reveal all the information that already came out of our data. For example, the 3D animation that we already show in the last slide-- different still images, but around 180 panorama and different applications.

After this stage, we already developed-- we take this into another stage. So right now, we are building a Saudi virtual environment platform. It's a web enabled platform, where we add all our information, all our scanned area, inside KSA and Middle East. And right now, it's under development. So this screenshot of the web platform-- so it's device independent. So you can work with it in your mobile and your laptop, even in the desktop application. And it can be displayed in virtual reality devices. Any questions?

Another case study-- actually, it's totally different. So this is model building, model area. This is King Abdullah Economic City. It's 20 kilometer away from King Abdullah University. And basically it has been built since 10 years. So all the buildings are already modern buildings, referring to architecture, new styles of buildings.

So the total area was 26 million square foot. But most of the area was empty, because it's still under development. We use level of details number 2. And the main purpose of this application of this modeling was interactive visualization. So mainly, the client want to have a 3D interactive map for the whole area to help in communication, to help in guiding the visitors while they are visiting the city.

This is a screenshot of a part of the area. So the workflow here, it's a little bit different-- six steps. Again, pre-capturing. And here I call capturing, not scanning, and I will say why after a while. Then capturing, 3D modeling, texturing, geo-referencing, visualization.

The number of buildings was not very high. The total number of buildings was about 70 buildings. But unique buildings was 21. So basically we need to model 21 building to have the whole city, the 3D digital city. Here we use the different drone, which is DJI Inspire 2, with Zenmuse X7 camera. It's very high resolution camera. And basically, for each building, we had to capture the building from different angles to reveal all textures, including the roof, as we can see in this slide. So for this specific building, I have it as an example.

So we captured different images from different angles. So at the end of the day, we have a folder of images for each building. And we start the modeling process inside AutoCAD. So we insert the orthophoto or the roof image of the building, and we start tracing the footprint of the building as polyline.

Yeah. Make sure we close all the polylines because we will use this inside 3D Studio Max. So after we finish the whole footprint for this building, we import it inside 3D Studio Max to start the extrusion process. I will start to select each polygon, and adding the extrusion value based on our captured images.

And in this specific application, we have to make sure that we are modeling our building in a very low polygon count. So make sure when you have cylindrical shapes or circular shapes, you have the minimum segments in this 2D shapes. Because this will increase the number of polygons if we didn't put this into consideration. And this will cause some problem and some challenges in visualization.

So here we finished the 3D model. Then it comes to texturing, how we are going to add texture to our model. So we used a very famous technique. All gamers and all 3D models using it, which is [INAUDIBLE] UVW. Basically, you select any object inside the building, and you export the texture map. Then you use Photoshop to add your textures that already captured by drone as a new layer inside this map after you finish.

So this is one example inside Photoshop. We just captured this sample area, and we brought it and its specific location inside the texture map. Then I load it again using the same modifier to my model. After we finish this texturing process for each building, we come to the geo-referencing step. How I can make sure that my model is geo-referenced to real coordinates?

So we used here different techniques. So we used plug-in called Simlab. You can install it inside 3D Studio Max to extract or to export KMZ file from 3D Studio Max. Then you open it inside SketchUp using a very unique feature called Add Location. So inside SketchUp, you can add location, add geolocation. So you just select the area that's already including the real city you are modeling. And you'll start adding your models to its correct location.

And for visualization, we used Cesium. How many of us know about Cesium? OK. So Cesium, it's a web platform. It's already developed based on OpenGL and WebGL libraries. And actually, it helps us a lot to visualize all our data, our 3D model data, geo-referenced data, inside the web. It's similar to Google, but for this specific application, it's free application, open source. So you can add whatever features you need. For example, this example-- so here in this specific example, this is New York. So we are visualizing-- Falconviz did not provide this example. This is example. It comes from internet. But just to show you what Cesium can do.

So 1.1 million buildings already visualized in this simulation, level of details number 2. And you can build whatever scenarios you would like to have inside Cesium. At the same time, you can have a visualized queries. For example, you can ask the simulation, please show me in a different color all buildings that already has a specific height, or all buildings that already has a specific application. For example, I would like to know all the residential buildings inside an area. So I will use this simulation query to visualize it and export it as 3D file.

How to do this? Actually, it's very simple. Cesium make it very simple. You have to have your own account inside Cesium ion website. Then you start uploading your models in 3D tiles, or in different files. So imagine that we have all New York City model. I cannot model it as one file. I have to partition the whole area into different tiles. Then I upload each tiles as different file.

So here we have one tile include four buildings. We upload it as one file. Then it gives me a code, like embedding code in YouTube or so. So I can have this code embedded into my project and visualize it at once inside Cesium. So that was for visualization. Thank you so much. Any questions?

[APPLAUSE]

Thank you. Yes, please.

AUDIENCE: [INAUDIBLE].

PRESENTER: Yes. Because when you upload it inside Cesium ion, it will reveal its location. But if it's not, it will go to a very different location, and you cannot move it inside Cesium. OK. Any questions? Please.

AUDIENCE: When you were doing the [INAUDIBLE] for the building, could you have just done [INAUDIBLE]?

PRESENTER: Yes, but in this specific example, we actually tried to have another technique. So that's why we have it in a different technique. We didn't have a 3D automatic mesh that already generated from the drones in this example.

AUDIENCE: [INAUDIBLE].

PRESENTER: Yes. Yeah. Yeah. OK. Good? Thank you. Yes, please.

AUDIENCE: [INAUDIBLE].

PRESENTER: So actually, for this build, for this area, we have eight oblique flights. So four x-- four directions, east, west, north, and south, and we have two diagonals. Two diagonals in different ways, so four directions, basic directions, and four diagonals. Yeah. Please.

AUDIENCE: [INAUDIBLE]?

PRESENTER: So Cesium, you can implement and install your server inside your server workstation. So you can host all the data inside the buildings or inside the city. So nobody else can access that. Yeah. OK. Please.

AUDIENCE: [INAUDIBLE]?

PRESENTER: The maximum error. So actually, we are talking about the accuracy here, yes? The accuracy, it comes with different-- there is different limitations and parameters that already defines the accuracy. First of all, the height or the altitude of the drone; the resolution of the cameras that you already use. So different parameters. But in Falconviz, we stick with standard accuracy, which is 2 centimeter x and y and 5 centimeter in z. Yeah. OK. Thank you. Thank you so much. Thank you.