Venice Families and Real Estate: Difference between revisions

From FDHwiki
Jump to navigation Jump to search
Line 58: Line 58:
* Midterm presentation
* Midterm presentation
* Extracting some overview data and plots
* Extracting some overview data and plots
*
| align="center" |  
| align="center" |  
|-
|-

Revision as of 17:45, 24 November 2021

Introduction

The Catastici and the Cadaster hold information about who owned buildings in Venice in the years 1741 and 1808. In order to find out how power shifted between families in this time, we will look at the change in real estate between these years.

Motivation

Plan and Milestones

First Steps

The first step working on the project was to familiarize ourselves with the datasets. We started working on the cadaster, as we assumed it would be easier to handle. After extracting some general information about the owners of the parcels, quickly we had to realize that the parcel owner column of the dataset is not homogenous in the way it is constructed. This is due to the way the data was extracted, and as it is based on scans of hand written documents, we found we had to deal with all kinds of problems associated with handwritten documents.

The Sudetto problem

The sudetto problem appeared quite early when working with the cadaster from 1808. Whenever multiple consecutive parcels have the same owner, most of the later entries have no owner specified anymore but the value of the column is 'sudetto'. Sometimes the later entries still contain the actual name of the parcel owner, in addition to 'sudetto'. In order to solve this problem, we iterated once through the entire dataset and filled in all the 'sudetto' with the value of the previous parcel owner.

Family names

Since most of our analysis is based on families and we assume that if a owner of a parcel has the same surname as another, they belong to the same family, extracting the family names was one of the first tasks we attempted to tackle. In the Cadaster we assume that for most of the areas (all except Canareggio) the surnames of parcel owners are written in capital letters. Using this information, we managed to extract some family names from the cadaster using a simple regex pattern matching. However, some other entities are also written in all capitals, especially public institutions. The extracted names were then compared to a list of familiy names we received. The list contains information about families in old venice like their noble status. Unfortunately, the list also contains all names of public institutions, so it became redundant to our purposes. Hence, a list of names of public institutions (by no means complete) was used to filter the extracted names.

For the catastici from 1741, it was found that approximately 40% of the entries for entry owner column are under the format 'name surname' so they were rather easy to extract. Additionally, assuming that the order of the names is correct, it was simple to extract family names and even names of the owners. For the other 60% of the dataset, it is not yet clear if there is a specific format.

Date Task Completion
Week 5
  • Start exploring the data
  • Use OpenRefine to clean some of the data
Week 6
  • Understand rules of family names in Cadaster and Catastici
Week 7
  • Solve sudetto problem
  • Use library of venetian names to filter the extracted family names
  • filter public institutions from family names
Week 8
  • Define rules to extract family names from cadaster and catastici
Week 9
  • Set up angular project for website
  • Start extracting datasets for website
Week 10
  • Midterm presentation
  • Extracting some overview data and plots
Week 11
  • Example webpage for one family
  • Extract comparable datasets for each family using names
  • Start building backend for the website
Week 12
  • Extract data for overview page
  • Generalize website for all families
  • Generate maps using extrated data
Week 13
  • Integrate data into website
  • Use plotly or similar tool to generate plots
  • Start writing project report
Week 14
  • Finish design and details of website
  • Finish project deliverables

Methodology

Limitation

Results

Reference

Links