Procedural reconstruction of Venice

Project definition and deliverables

From books to movies via video games and research, 3D representation of objects, buildings and even whole cities are everywhere. To create such 3D representations, one has to use a modeling technique. The most common one is the so called “by-hand” which involves numerous hours of meticulous work using a specialized software, such as Maya or Blender. A highly detailed model created that way can take a huge amount of disk space and may be difficult to work with when rendering it. A lesser known technique is called procedural modeling. This technique uses algorithms and a set of predefined rules (called grammars) to create the geometry from scratch. One of the most common example of procedural modeling is fractals.

The first 4 stages of the Pythagoras tree, a procedurally modeled tree [Wikipedia]

A grammar can express a very complicated object using only a few axioms (base rules) and thus, using only a very small amount of disk space create a very detailed model.

While the creation of 3D representation of unique or uncommon objects like the Venetians palazzi involves some modeling by hand or 3D scans of the entire building, creating the minor buildings of a city is the perfect case where one could use procedural modeling. Indeed, the quantity and variety of these constructions make them nearly impossible to model them all by hand.

The objectives of our project are the analysis of the minor structures of Venice at different periods and the extraction of grammars to allow the generation of similar constructions using a few parameters such as the century or the architectural style. The ultimate goal of our project is to create a user interface that could allow a parameterized generation of an entire Venetian neighborhood according to the era.

As an additional motivation for our project, it’s known that the Venetian architects have used several patterns when building the minor Venice along the ages. During the city’s history, there’s been several big fires (in particular in 1106, 24 churches were destroyed along with a lot of minor constructions) and parts of the city have been rebuilt very fast using variations of a small amount of blueprints.


Our initial task will be the gathering of a sufficiently large quantity of data in the form of drawings, pictures, video footages and eventually existing historically accurate 3D models. One issue we will face is the small amount of sources about the minor buildings in contrast with the major buildings like the palazzi. Indeed, there is a large collection of references on the later but only a small number of books about lesser known constructions hidden in the shadow of the palazzi and churches. We are currently evaluating the possibility to use Google Street View, crossed referenced with two books we currently have on the minor Venice [1&2], as a solution to this lack of content. We will need the help of a Venetian specialist to identify and filter modern buildings versus historical ones and to help us group them according to the century or era as precisely as possible.

Assassin’s Creed II Venice walkthrough, not 100% historically accurate

As identifying multiples characteristics in several documents and linking them together by date for example, we are planning to use DHCanvas (an annotation and transcription tool, using open annotation data model, created by the DHLAB at EPFL) to ease this task.

After having classified all our sources by date and eventually by architectural style or neighborhood location in Venice, we will begin the image annotation to extract specific features of each building type and to find similarities between them. This task must be done conscientiously as the quality of our grammar will highly depend on it. Each components (windows, balcony, doors, etc) of the facades will be annotated and linked to a probable style and/or construction date.

When the quantity of material about one era will be sufficient, our next task will be to extract the grammar for this building type and era from all these annotations. Typically, a facade will be divided in 3 floors and on each one of these floors will be 3 windows. The ones on the first floor are always bigger than the ones on the third floor, etc. At this stage of the project, our goal is to have a simple grammar for several buildings (the grammar will not only allow the generation of a facade but also a whole building). Our inspiration for this grammar extraction will be the shape grammar for CG Architecture [Procedural Modeling of Buildings].

The final stage of our project will be the actual generation of a 3D mesh starting from our grammars. To accomplish this task, we will, depending on the remaining time, either use a framework/rendering engine like Unity or Unreal Engine or we might also use the ability to import our own grammars in CityEngine. Another possibility we could explore is the creation of a plugin for Maya or Blender that could allow the generation of 3D geometry from our grammars. We will separate the mesh creation task in two smaller steps. First the generation of 2D facades procedurally, using the algorithm from Müller’s paper [Procedural Modeling of Buildings] and then, merge this representation in 3 dimensional space.

Finally, a bonus task, would be to generate several buildings starting from a Venice map and generate plausible neighborhood with spaces left for the palazzi. To realize this step we could adapt our previous tool or explore the existing open source tools available, for example this python procedural city generation tool.


  • Weeks 1, 2, 3 : Source gathering and analyzing
  • Weeks 3, 4 : Sources annotating on DHCanvas and identify features
  • Weeks 5, 6, 7, 8 : Grammars creation (with a strong focus on a few building style)
  • Weeks 7, 8, 9 : Generation of 2D facade procedurally
  • Weeks 10, 11 : Generation of one 3D building
  • Week 12 : Creation of the GUI for our application
  • Week 13 : Generation of a small set of buildings
  • Week 14 : Wrap up and final presentation


[1] Trincanato, Elge Renata. Venezia minore. Verona: CIERRE EDIZIONI, 2008. Print.
[2] Foscari, Giulia. Elements of Venice. LARS MULLER, 2014. Print.

Group members

Gaspard Zoss, Frédéric Moret, Pierre Sarton