Extracting Toponyms from Maps of Jerusalem

rubric

Written deliverables (Wiki writing) (40%)

Projet plan and milestones (10%) (>300 words)

Motivation and description of the deliverables (10%) (>300 words)

Detailed description of the methods (10%) (>500 words)

Quality assessment and discussion of limitations (10%) (>300 words)

The indicated number of words is a minimal bound. Detailed description can in particular be extended if needed.

Project Timeline

Timeframe	Task	Completion
Week 4	Finalize and present project proposals. Toponym extraction project selected.	✓
Week 5	Survey SOTA toponym extraction tools.	✓
Week 6	Port MapKurator's Spotter tool and model weights into Windows-based Python. Select two (later four) maps to use when implementing, evaluating, and fine-tuning MapKurator's model.	✓
Week 7	Create ground truth labels for first map with VIA's online interface.	✓
Week 8	Create ground truth labels for second map. Implement 1:1-matched precision and recall via IoU (geometry) and normalized Levenshtein (text). Calculate baseline accuracy statistics.	✓
Week 9	Implement multi-layer pyramid application of MapKurator's Spotter.	✓
Week 10	Create ground truth labels for third map. Implement toponym rectification and amalgamation on pyramid-derived toponyms.	✓
Week 11	Calculate pyramid accuracy statistics. Fine-tune toponym rectification and amalgamation. Deliver Midterm presentation.	✓
Week 12	Launch Wiki. Group words into toponyms via polygon size and location. Apply NLP tools to correct toponyms based on MapKurator strategy.
Week 13	Create ground truth labels for fourth map. Calculate final accuracy statistics. Hierarchize final toponyms and develop Voronoi map.
Week 14	Prototype toponym-disagreement visualizer. Finalize Wiki and deliver presentation.

Introduction

We aim to programmatically and accurately extract toponyms (place names) from historical maps of Jerusalem. Building from mapKurator, a scene text recognition pipeline developed specifically for toponym extraction by the University of Minnesota's Knowledge Computing Lab, we develop a novel label extraction and processing pipeline capable of significant accuracy improvements relative to mapKurator's spotter module alone. We then explore the success of our pipeline during generalization, describe the limitations of our approach, and suggest possibilities for future progress in accuracy or end-user interactivity.

Motivation

A sample of the linguistic, geometrical, and typographical diversity of toponyms in 19th-century maps of Jerusalem.

Development of an accurate, accessible, and generally applicable tool for extraction of map toponyms is an unrealized goal. Indeed, despite the deep-learning induced boon to scene text recognition (STR) seen in the past decade, STR in general remains difficult due to the sheer variety of shapes, fonts, colors, languages, and text backgrounds encountered. In the context of historical maps and Jerusalem especially, STR issues are compounded by the fact that there is not necessarily agreement between cartographers over toponym spellings (in the case of toponyms transliterated from arabic, for example) or, as is true among biblical toponyms, even the precise location of certain places. Properly collecting toponym locations and spellings from historical maps of Old Jerusalem would thus permit researchers to understand how such ambiguities have manifested as agreement or disagreement in geographic depiction throughout time.

In an effort to automate text recognition on historical map archives - our exact use case - the University of Minnesota's Knowledge Computing Lab has developed an end-to-end toponym extraction and rectification pipeline dubbed mapKurator. While we do build on their methods, we identify three issues with simply implementing mapKurator on historical maps of Jerusalem to achieve our intended result. First, some elements of the full mapKurator pipeline (namely the use of Open Street Map to correct extracted labels) are not suitable for historical maps of Jerusalem due to the ambiguities described above. Second, the success of the chief tool used to extract text in mapKurator depends heavily on its model weights - weights that users may not have the computational power or ground truth labels to train. Third, mapKurator's text spotting tool only extracts individual words, requiring further processing to capture multi-word toponyms. We therefore decide to augment mapKurator's text spotting tool with a pipeline involving repeated sampling and methods for recovering proper toponyms from said sampling.

The contributions of this project are thus inroads into the collection of accurate toponyms from historical maps of Jerusalem and a general, accessible pipeline that can be mounted on practically any pre-trained STR label extractor to improve accuracy.

Deliverables

- A novel, general-use, and computationally accessible toponym extraction pipeline capable of augmenting pre-trained SOTA neural network models.

- A set of X ground truth toponyms across Y historical maps of Jerusalem.

- A set of X extracted toponyms across Y historical maps of Jerusalem.

- A set of X labels denoting font similarity across Y images of toponyms.

Methodology

Extracted Label Processing

Multi-Line Sequence Recovery

Single Line

Multiple Lines

Curved Line

Limitations

Only X maps and Y labels for ground truth

Imbalanced font similarity labels, + the fact that font similarity labels may not generalize

No Open Street Map equivalent - and developing one would require a custom set of ground truth possibilities

Github Repository

Jerusalem Maps EPFL DH405

Acknowledgements

We thank Professor Frédéric Kaplan, Sven Najem-Meyer, and Beatrice Vaienti of the DHLAB for their valuable guidance over the course of this project.

References

Webpages

• MapKurator GitRepo
• MapKurator Introduction

Chief Literature

• Kim, Jina, et al. "The mapKurator System: A Complete Pipeline for Extracting and Linking Text from Historical Maps." arXiv preprint arXiv:2306.17059 (2023).
• Li, Zekun, et al. "An automatic approach for generating rich, linked geo-metadata from historical map images." Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020