Progress Report #1 – 3D Reconstruction Using Images From Aerial Vehicles

Overview:

In our project we aim to design a system for automated 3D reconstruction of buildings and monuments of historic importance in Venice. The 3D models will be generated from images taken by an aerial vehicle (drone). To simplify the process of data collection the drone will be asked to scan the object autonomously. The original project description can be found here

Progress:

We are now in the third week of the semester. At this point we were able to achieve the first milestones and adapt/concretize others future ones. Below is a detailed summary of our current results (#numbers correspond to positions in the original work plan).

#1 [Done] Regulations regarding UAV flights in Lausanne and in Venice

  1. Legislation in Switzerland allows the operation of drones weighing less than 30 kilograms without any license as long the drone is always visible to the operator.
  2. Regulations for operating drones in the city of Venice is more restrictive since Italy has introduced new law in 2014 . According to the law, drones with a takeoff weight lower than 25kg (“Light Drones”) could be operated based on self-certification (attesting that the drone complies with the Regulation) filed with ENAC (Italian Civil Aviation Authority) along with some technical documents;  Drones with a weight of lower that 25 kg can fly in only restricted air space which is  70 m maximum height above the ground and 200 m radius.  Italian Civil Aviation Authority provides more  simplified attestation procedure for the drones with weight less than 2 kg.

#2 [In progress] Reconstruction software

We will be using the following tools for dense 3D reconstruction from multiple images:

Bundler

This is an open-source SfM library which takes in images taken from different viewpoint and outputs 3D reconstruction of camera (camera poses in the scene) and sparse 3D geometry of the scene.

Clustering Views for Multi-view Stereo (CMVS)

This tool takes in the output of the Bundler and creates dense 3D point cloud of the scene.

Meshlab

Finally we are planning on using Meshlab for creating polygon meshes out of point cloud for surface reconstruction.

There are many other options available in terms of tools for 3D reconstruction such as openMVG, VSFM but they require either more effort on building the reconstruction pipeline or they are already fully integrated and do not allow for the modifications we would like to add.

#3 [In progress] Hardware – selection and ordering

  1. Optimal choice: AscTec Hummingbird
    Initially we considered the Parrot Bebop or the DJI Phantom to be the most suitable solutions for our purposes. However, the Bebop is a new product that has recently come to the market receiving numerous negative reviews due to software and hardware problems. The DJI Phantom is the most popular solution for aerial photography so far offering first person view (FPV) from an HD camera, long flight time (~20 minutes) and long communication distance (>500 m). Both Drones offer autonomous flight modes by traveling a given set of GPS waypoints. However, the accuracy of GPS (especially in city environments) does not suffice to guide the drone closely around a building. In order to have more control over the drone’s trajectories without doing it manually, we need to be able to program the drone itself.
    With the AscTec Hummingbird we are choosing a powerful quadcopter with the strengths of the DJI Phantom combined with an on-board computer and the opportunity to add additional hardware like a camera for visual guidance.
  2. High costs: looking for collaboration
    We are currently looking for collaborations with EPFL laboratories that dispose over AscTec Hummingbirds. Thereby we hope to avoid high costs and also to have references when we are working with the drone.

#4 [Started with an idea] Localization techniques
Even though this topic was intended to be tackled later we already sketched an idea. Our primary goal is to give a simple and, most importantly, practical solution to automated scanning of buildings. To achieve this we plan to equip our drone with a down-facing camera. The drones desired path around an object is drawn on the ground with a wide tape that must be visible to the camera. Using line-tracking the drone will then be able to follow the tape around the building. To scan buildings at several heights the drone will do multiple laps and vary its own altitude.
More sophisticated methods were discussed but discarded due to their complexity (e.g. the triangulation methods that were originally planned or encoding the ground markings indicating altitude and position on the tape).

[Not yet treated]

#5 Write a robot controller that makes the robot go around the bounding box in an automated fashion (lateral scans at different heights)  [3 weeks]

#6 Flight testing and debugging. [1 week]

#7 Deployment : Scan selected buildings and monuments in Venice [1 week]

Intermediate resume:

We have finished investigating on regulations, hardware and software. Also we found an idea that appears feasible for the purpose of autonomous navigation. Due to the availability of tools that handle the largest part of image reconstruction our focus will be more on the implementation of autonomous navigation. Selecting the right images to feed into the pipeline will be treated when our hardware components are set up and ready to test. As most commercially available drones are not customizable enough for our needs we are now depending on the availability of an AscTec Hummingbird at EPFL, which is the strongest drawback at the moment. Also we might be interfering with Italian law.


 

References:

http://www.cs.cornell.edu/~snavely/bundler/
http://www.di.ens.fr/cmvs/
http://meshlab.sourceforge.net/
http://www.asctec.de/en/uav-uas-drone-products/asctec-hummingbird/
http://www.dji.com/product/phantom-2-vision-plus/feature
http://www.parrot.com/de/produkte/bebop-drone/
http://www.swissinfo.ch/eng/successful-takeoff-for-swiss-commercial-drones/35863208
http://www.enac.gov.it/repository/ContentManagement/information/N1220929004/Reg%20SAPR%20english_022014.pdf