Procedural Venice

Introduction

Procedural modeling is an umbrella term for a number of techniques in computer graphics to create 3D models and textures from sets of rules. In this project, we are going to randomly generate the Venice building models using Houdini, with VEXpression script and visualized nodes. Our project will mainly focus on the shape part of the models, but not the texture part.

We use Houdini for procedural modeling:

(1) Based on OpenStreetMap data, which is the original location of Venice buildings, to randomly create the Venitian style buildings.

(2) At this stage, considering the difficulty and time of the project, only focus on the Venetian roof and window style.

(3) Divide the city generator into building generator, divide building generator into roof generator, window generator etc(Figure 1).

Figure 1 Architecture of the procedural Venice

Motivation

The creation of compelling models is a crucial task in the development of successful movies and computer games. However, modeling large three-dimensional environments, such as Venice city, is a very expensive process and can require several years worth of labor.

As for Venetian buildings, except for some landmark buildings such as churches and palaces, most of them are similar Venetian style buildings that could automatically generated using the procedural method. By using the procedural generating method, it will save a lot of artwork and cost.

Project Plan and Milestones

Finished: Finshed as planed

Delayed: Not finished as planed

Changed: Replaced by other method or abandoned

Date	Task
By Week 4	Collect information for the Procedural city generator and GAN painting generator. Prepare for the project kick-out slides.
By Week 5	Discuss with TA the implementation plan of procedurally generated Venice Obtain the original TA's method and engineering files of the procedural Venice. Read books about architecture and start to use Google Maps to classify Venice architecture.
By Week 6	Completed the basic classification of Venetian architecture. Determined the classification of simplified Venetian buildings, palaces, and complex churches. Distinguished and classified various components in simplified Venetian buildings and palaces.
By Week 8	learned the TA's city generator. learned the Houdini Basic operation and programming. Made some basic demos following the tutorial on youtube.
By Week 10	Create simplified building creators using Houdini. Create a basic roof generator. Create city generator with SideFX Labs tools. (We didn't use this method because we think it is difficult to make Venetian style building roof using this method)
By Week 11	Use Maya to build some windows and chimneys, import them in Houdini. Add more details to the basic roof generator( including add vertical roof structure, tiles, chimneys, attic). Solve the problem that the lab OSM node cannot connect to the self-written building generator. Create the basic city generator. Create a basic wall generator, with window models. Create another fence style roof generator.
By Week 12	Improve the building generator, which could adjust building height, windows num etc. Random assign the building height in the city generator. Implement web using Streamlit python package, deep learning method will be used on the web. input to Unreal and make a short video about the final scene (Abandoned because we want to focus on Houdini modeling generator, importing to Unreal might face new conflict and Unreal5 which comes next year will be much more powerful than Unreal4)
By Week 13	Push the models and engineering file to the GitHub. Finish the report. Testing Video
By Week 14	Final Presentation

Methodology

Classification of Venetian buildings

Correctly classifying Venetian buildings is very important in this project. Using Google map of Venice, pictures on Google, we classify Venetian buildings into 3 types, which are churches, simplified normal buildings, and palaces (as the pictures below). For churches and palaces, due to their sophisticated design, they have more features and types than simplified normal buildings. Especially for churches, because of its uniqueness, procedural modeling is of little significance compared to manually modeling. After considering the technical difficulties and workload, we decided to focus on simplified Venetian buildings.

Venetian building types:

Churches (Need to model manually because of the uniqueness of the church)

Different types of churches in Venice

Palaces

Different types of palaces in Venice

Simplified buildings

Different types of simplified buildings in Venice

We can see that compared with palaces, simplified buildings have regular cubic shapes. The roofs are mostly sloping roofs with chimneys. The windows are simpler than palaces, and there are fewer windows with carved rails. Simplified Venetian style buildings has different components. We can classify them through texture(color, wall damages), chimneys, windows, doors.

Different component types:

Windows

Different types of windows in Venice

Chimneys

Different types of chimneys in Venice

Roofs

Different types of roofs in Venice

Roof generator

Two Roof types

roof generator type 1
roof generator type 2

Roof details

We used the 3D modeling software Maya to build some chimney models and exported the FBX file to Houdini. A certain density of points can be randomly generated on the plane of the roof of the house so that the chimney with the normal upward is assisted on the corresponding point.

chimney model 1	chimney model 2

Window generator

For the window generator, in the actual project, we did it in two methods at the same time:

One is to use the Maya modeling software to model the Venetian windows then import it into the Houdini engine. The advantage of this method is that it is relatively straightforward and we can use the Maya for complex window structures. However, the disadvantage is that there are not many variables that can be programmed and automatically controlled.

Window model 1	Window model 2	Window model 3	Window model 4	Window model 5

Another method is to use Houdini to program the windows. We chose the arched window structure to generate it, using the tangent relationship of the circle to generate a total of 7 types of arched structures. But due to the time limitation, we did not finish the whole window structure.

Window generator type 1	Window generator type 2	Window generator type 3	Window generator type 4
Window generator type 5	Window generator type 6	Window generator type 7

Simplified Building model

simplified building generator

Venice city generator

OpenStreetMap of Venice is used as a reference to pinpoint the location of every building. Using the sideFX Labs tools, the Venice city buildings' locations could be pinpointed in Houdini, and the street information and building inner parts are filtered using OSM filter node. Because there are some overlaps between different surfaces, so we add a boolean node and union the overlap surfaces.

Venice OSM data

In this step, we connect the OSM data to our building generators. First, we traverse all OSM surfaces and use each OSM surface as the bottom surface to reconstruct the building generator. We

Bird view of Venice
Bird view of Venice
Different height of houses

Quality Assessment

Roof generator assessment

We have made two types of roof generators. For different shapes of ground, our roof generator can automatically generate the corresponding shape of the roof.

It can be seen that for a single roof, the roof can be automatically covered with tiles if the amount of computer calculation allows. On the roof of Venice, there are tiles neatly arranged on the roof, but due to the age, some tiles have faded and some degree of damage has also appeared.

real Venice roof
roof generator model

Among the buildings in Venice, a very small part of the roofs (mostly on the riverside) are fence-style. You can compare the following two pictures. Our second roof generator restores this type of roof style.

real fence roofs
roof details model

We can see that the roofs of Venice show irregular shapes. The roofs are vertical and inclined, but in our city generator, there are only inclined ones. That is because our vertical roof structure has some bugs when dealing with a large number of irregular models. So we commented it out temporarily. It can be seen that the effect of our roof generator results in fitting irregular roofs is very close to the real situation.

real roofs
roofs' model
Rendered roofs

We compared the top view of the roof near San Giacomo di Rialto in the real Venice with roofs generated by our roof generator, and found that from the top view, the shape and style of the roof are basically similar to the real Venice(the green box parts below). But if you look carefully, there will be some differences, for example, if there is an open space in the middle of the building, this area may be filled in the generated model(the red box below). At the same time, the roof will also have some randomly generated structures, just like the strange structures that would appear in real Venice.

Comparation of roofs near San Giacomo di Rialto from the top
Rendered roofs from the top

Building generator assessment

we used the building generator randomly generating two buildings for assessment, with different types of windows and roofs.

For buildings near the river, there will be a small number of fence-type roof buildings. When classifying, we classify them as palaces. In our single building generator, this type of building can be reproduced well. The disadvantage is that there are still some details missing, such as flower pots outside the windows, corner decorations, etc., which can be attached to the building generator in the same way as windows.

Simplified Building 1
Simplified Building 1
Simplified Building 2

For simple buildings, which are also the buildings with the most in Venice, we can see that our randomly generated buildings can also reproduce this type of building. However, in a real Venetian architecture, there will be many details, such as clothes poles, telephone poles, etc., which need to be constantly improved to look more real. Compared to the real Venice, our generated models are still too clean, too neat, and lack traces of years.

Simplified Building 3
Simplified Building 3
Simplified Building 4
Simplified Building 5

City generator assessment

We used Houdini to generate 1,407 buildings in the central area of Venice. Compared with Google Maps 3D, from the perspective of a bird's-eye view, we can see that the generated city are very similar to Google 3D map.

Real venice city from Google Map 3D
Generated venice city in Houdini
Rendered venice city in Houdini(white model)

Limitation

Running time limitation

For procedurally generated roof with tiles, if all building tiles in the city are generated procedurally, it will cost a lot of computing time and resources. For a tile, there are more than 10 faces, and it takes much time to generate a roof covered with such tiles using Houdini(with GTX1060). Our test city OSM scene has 153 building planes, which is quite small compared to the real size of Venice. If we generate all the roofs covered with tiles, then it will definitely take hours or even days. We think Applying UV mapping to the roof can be an alternative method to deal with the roof tiles.

Archetecture Precision limitation

At present, the procedural generation of Venetian buildings can only correspond to the bottom contour of the building. However, it cannot automatically match the number of building windows, window styles, roof shapes, and other factors of the buildings. We have not build a connection with data points or pictures of Venice, which is very difficult in Houdini engineer.

Dilapidated

We did not think about the dilapidated problem when we actually modeled. Actually, many tiles in Venice buildings are damaged, many walls are worn, many bricks are missing, and many chimneys are damaged.

Github Link

Procedural-Venice

References

Literatures

[1] Tavakkol, Sasan & Han, Feng & Mayer, Brandon & Phillips, Mark & Shahabi, Cyrus & Chiang, Yao-Yi & Kiveris, Raimondas. (2020). Kartta Labs: Collaborative Time Travel.

[2] Răzvan, Cristea. (2020). Procedural Generation of Architecture & Props.

[3] A history of world architecture (before the end of 19th century), in Chinese.

[4] Procedural Modeling of Buildings. Article in ACM Transactions on Graphics · July 2006