Venice2020 Building Heights Detection

Introduction

The main goal of our project is to obtain the height information of buildings in Venice. In order to achieve this goal, we construct a point cloud model of Venice with drone video from youtube and google earth with the help of a photogrammetry tool developed at DHLAB.

Figure 1:

Motivation

Building height is one of the most important information to consider for urban planning, economic analysis, digital twin implementation, and so on. For Time Machine implementation, it is important to compare different historical data and current data based on different times. Drones and Google Maps provide 3D views of buildings and surroundings in detail which are useful resources to keep track of changes in a target place. In this project, we aim to detect the building heights of Venice. Many geographical areas, which are either not famous or not accessible easily are accessible by Google Earth 3D views. We aim to make a 3D model of Venice with every detail of the city and calculate point clouds to detect the heights of the city. This information can be used to understand current details of the city and also will be useful in the future as historical data to compare with.

Figure 1: YouTube Video

Methodology

Images Collection

To build three dimensional point cloud models

Youtube Video Method

Our initial plan was to download drone videos of Venice on Youtube, extract images from the videos, and generate point cloud models based on the images that we obtained. However, the videos we could get from drone videos are very limited: Few of the videos are of high quality, and among those eligible videos, most of the videos only focused on monumental building such as Saint Mark Square. Therefore, a large part of Venice will be omitted if we only adopt Youtube Video Method.

Google Earth Method

In order to access more area of Venice, we decided t

Photogrammetry

Once the images of different parts of Venice are prepared, we use a photogrammetric reconstruction tool to generate point cloud models. Photogrammetry is the art and science of extracting 3D information from photographs.

Plane Generation

To calculate the height of building, we have to find the plane in the very beginning, which could be realised by a tool provided by Open3D. Open3D supports segmententation of geometric primitives from point clouds using 'RANdom SAmple Consensus'. To find the plane with the largest support in the point cloud, we can use segment_plane by specify 'distance_threshold'.

Height Calculation

After we obtained the plane equation in the previous step, we calculated the relative height of the building using the equation of distance from point to plane, the coordinate information of each point could be accessed by the 'points' attribute of the point cloud. However, we do not have the 2D coordinate information of the point since the point cloud is a 3D object, we could not just visualise the height information on a 2D map. An alternative way is to mark the height information of each point through the color attribute of the point cloud. For each point in the point cloud, it has a 'colors' attribute, which returns an array of shape ( 1 , 3 ), range [ 0, 1 ], containing RGB colors information of the point. Therefore, by normalising the distance array to the range of [0,1] and expanding the distance array to three dimensions by replication, we get a 'distance array' in the form of 'colors array', lighter the color, higher the point.

Milestones

Milestone 1

• Get familiar with OpenMVG, Open3D, OpenGM, Point Cloud Library, Agisoft Metashape, Blender, CloudCompare and QGIS
• Collect high resolution venice drone videos on youtube and record Google Earth birdview videos as supplementary materials

Milestone 2

• Align photos of the same location in Venice, derive sparse point clouds made up of only high-quality tie points and repeated optimize the camara model by reconstruction uncertainty filtering, projection accuracy filtering and reprojection error filtering.
• Build dense point clouds using the estimated camera positions generated during sparse point cloud based matching and the depth map for each camera
• Evaluate point clouds objective quality and select high quality models by point-based approaches which can assess both geometry and color distortions

Milestone 3

• Align and compose dense point clouds of different spots to generate an integrated Venice dense cloud model (implement registration with partially overlapping terrestrial point clouds)
• Build Venice 3D model (mesh) and tilted model according to dense point cloud data
• Generate plane of the surface ground

Milestone 4

• Build the digital elevation model of Venice and align the model with open street map and cadaster in the 16th century to obtain building heights
• Assess the accuracy of the building heights detection

Planning

Week	Tasks	Completion
week 5	Brainstorm and present initial ideas for the project	✓
week 6-7	Configure the environment and master the related softwares and libraries Find suitable drone videos on YouTube and obtain photos for further photogrammetry work	✓
week 8-9	Align photos, generate and denoise dense point clouds Assess the quality of point clouds both subjectively and objectively	✓
week 10	Implement ground segmentation Collect screenshots of buildings from Venice on Google Earth Prepare for the midterm presentation	✓
week 11	Calculate the height of buildings and generate map layers with building height information Evaluate the models based on the overlapped parts in the map layer with building height information
week 12	Build 3D model, tiled model of Venice Align the height information layers with OpenStreetMap
week 13	Write final report, refine the map layers with building height information and the final models
week 14	Refine the final report and code Prepare for final presentation