Venice Families and Real Estate

Introduction

In this project we take the reader back in time, through our Venetian Time Machine, to two critical points in Venice’s history: 1748 and 1808. Rife with conflict, development, and Venetian love, these two period offer a look into the life before us. The composition of cities, the family power dynamics, and the cities progressions. We’ll call the data from 1748 the Catastici the data from 1808 the Cadaster. The Catastici and the Cadaster hold information about who owned buildings in Venice in the years 1741 and 1808. In order to find out how power shifted between families in this time, we will look at the change in real estate between these years.

Motivation

“Life in the state of nature is solitary, poor, nasty, brutish and short.” - Hobbes

History has repeatedly shown us that empires rise and fall, cities evolve and change, and families battle for power. Up until now, historians have relied largely on diaries, archives, interviews, and city plans to qualitatively track these dynamics. But, while effective at understanding a static picture of a city through the lens of a few families, it becomes increasingly complex to track the evolution of an environment. The Venice Time Machine Project has aimed to digitize large swaths of archival data to permit new forms of analysis that up until now have not been possible. In our research we focus on two new sources of data, Venice’s 1748 Catastici and 1808 Cadaster. These two geographic datasets contain information about parcels of land across Venice like ownership, tenants, area, and category of property. In this project we use these data sources to paint a static picture of Venice in both periods, and then track its evolution during a period rife with conflict, like Napoleon’s invasion in 1797. After conducting the analysis, we create a website to present the data and findings in a visually appealing way, which can be of use for future Venice researchers.

Research Questions

More specifically, we tackled three main questions.

• What was the static distribution of power in Venice in 1748 and 1808.
  • Who was the most powerful?
  • What were the relationships between families
• How did the dynamics between families change?
  • Biggest upsets!
  • New faces.
• How did the landscape of Venice change?
  • Were locations nationalized or increasingly privatized?
  • How did the type of location change?

Plan and Milestones

First Steps

The first step working on the project was to familiarize ourselves with the datasets. We started working on the cadaster, as we assumed it would be easier to handle. After extracting some general information about the owners of the parcels, we quickly ran into our first problem: the parcel owner names are not constructed in a homogenous way. This is due to the way the data was extracted, and as it is based on scans of handwritten documents, we found we had to deal with all kinds of problems associated with handwritten documents.

The Sudetto problem

The “suddetto problem” appeared quite early when working with the cadaster from 1808. In Italian, suddetto means “aforementioned” and was used whenever a family name was consecutively mentioned in the dataset.

When multiple consecutive parcels have the same owner, most of the later entries would have no owner specified anymore but the value of the column was 'suddetto'. In theory, this issue would be easily fixed through a forward fill, but unfortunately this was not uniformly conducted on the dataset. Instead, some locations utilized the suddetto method, others reverted to properly naming all locations. Fixing this issue required doing a location specific forward fill to align each parcel of land with the correct owner. 

Family names

Regex search: Since most of our analysis is based on families and we assume that if an owner of a parcel has the same surname as another, they belong to the same family, extracting the family names was one of the first tasks we tackled.

The datasets were both cleaned using OpenRefine, in order to cluster similar entries for the ‘parcelOwner’ feature. The text facets were clustered using key collision and fingerprinting as the keying functions. Using the functionality of OpenRefine, we managed to rid the noise in both the Catastici and the Cadaster datasets. For the Cadaster dataset we assume that for most of the areas (all except ‘Canareggio’) the surnames of parcel owners are written in capital letters. Using this information, we tried to extract some family names from the cadaster using a simple regex pattern matching. However, some other entities are also written in all capitals, especially public institutions. In order to filter the public institutions and other noise that might be in the extracted family names, they were compared to a list of venetian family names. The list contains information about families in old Venice like their noble status. Unfortunately, the list also contains all names of public institutions, so instead we filtered the family names using a handwritten list of public institutions.

For the catastici from 1741, it was found that approximately 40% of the entries for entry owner column are under the format 'name surname' so they were rather easy to extract. Additionally, assuming that the order of the names is correct, it was simple to extract family names and even names of the owners. For the other 60% of the dataset, it is not yet clear if there is a specific format. It contains a lot of additional information about the relations of the owners of the parcels. For example the entry “Giovanni Paolo e fratelli Palmerini” suggests that Giovanni Paolo and the Palmerini brothers owned the parcel together. There are many more entries of similar format, giving this kind of information that we haven’t attempted to extract at this point.

Fuzzy name extraction: Instead of a simple regex search, we turn to a fuzzy name extraction method that’s capable of finding a name’s closest match in a list. Given a family name, we found the closest match in the set of known Venetian family names. Fuzzy name extraction conducts the string matching by choosing the family name with the minimum Levenshtein distance (shown below) and returns a probability that the name is a match. To limit noise we removed names that were not names, like “di” and “scuola” and then found the closest names. Python’s fuzzywuzzy package provided a simple and quick method of applying fuzzy name extraction on the large name column, and it is also capable of doing substring matching, which is what we needed to extract only the last name. We were able to extract the names for 16,000 rows (88%).

Italian name translations Connecting the catastici and the cadaster dataset proved more difficult than expected. This is largely due to the differing origins of the datasets. Whereas, for the catastici, surveyors went out and manually validated data, the cadaster was automatically generated from texts. This asymmetry leads to difficulty when merging the two datasets. One of the examples of the issue came from the way parcels were categorized. In the catastici a simple english naming was used and in the cadaster we had a series of 20 different Italian names, written in an archaic language Google Translate failed to understand. To fix this issue, we attempted to manually translate the Italian words into the corresponding English names.

Analysis

Static image

After the data cleaning done above, we turned to analyzing the datasets. We commenced with a static image of both locations. This included the most famous families, the composition of the land, and relationships between the tenants and owners.

Jumping into the time machine: back to 1748

To begin, we visualize the map of the Catastici data below.

Date	Task	Completion
Week 5	Start exploring the data Use OpenRefine to clean some of the data	✓
Week 6	Understand rules of family names in Cadaster and Catastici	✓
Week 7	Solve sudetto problem Use library of venetian names to filter the extracted family names filter public institutions from family names	✓
Week 8	Define rules to extract family names from cadaster and catastici	✓
Week 9	Set up angular project for website Start extracting datasets for website	✓
Week 10	Midterm presentation Extracting some overview data and plots
Week 11	Example webpage for one family Extract comparable datasets for each family using names Start building backend for the website
Week 12	Extract data for overview page Generalize website for all families Generate maps using extrated data
Week 13	Integrate data into website Use plotly or similar tool to generate plots Start writing project report
Week 14	Finish design and details of website Finish project deliverables