Universal Aesthetics (Multimodal Focus)

Methods

Data

As for the convergence of language models, we need both plain texts and aesthetic texts. For simplicity, we reuse this text-image dataset, which is also used in Huh et al.'s paper, and then add another poem dataset.

Poems

For poems, we use the Poems dataset from Kaggle. We find this dataset ideal for this project because of the following reasons:

• As the plain-text dataset contains 1,024 entries, it provides enough poems to yield a substantial amount of data.
• It categorizes the poems into 135 types based on their form (haiku, sonnet, etc.), which could facilitate our further studies.

However, this dataset still needs to be cleaned before usage. We identify two problems with the raw dataset. First, some poems contain copyright notices at the end, which introduce noise into subsequent processing. However, because the copyright information is clearly marked with a special mark ©️, it can be easily removed through rule-based filtering. Second, although most poems are in English, a small portion is not. Since the plain-text dataset contains almost exclusively English texts, we should also remove the non-English poems from this dataset.

Afterward is an unknown term in future
Before that we face the present,
Coming at well future depends on present;
Dismissing hazardous future
Endeavor best early at present.
Copyright © Muzahidul Reza | 29 November,2017

The text above shows an example of poems with copyright information. We assume that the mark © does not appear within the poem itself and remove all the content starting from any line that begins with this symbol.

To filter out non-English poems, we use the word frequency list as an auxiliary resource and construct an English lexicon by selecting only the words whose frequencies exceed a certain threshold (10,000). For each poem, we compute the proportion of lemmatized words that appear in this lexicon and apply a threshold to identify English poems. We initially experimented with this English words list, but it was overly inclusive and contained many non-English words such as bonjour. This caused some non-English poems to match a large number of dictionary entries. Therefore, we adopted a frequency-based filtering approach to exclude words that may have been borrowed from other languages and appear in English text only occasionally, despite being included in comprehensive dictionaries. The table shows how poems with different proportions of English words detected look like. To minimize bias while filtering out as much non-English data as possible, we chose a threshold of 0.8.

Proportion	0.00	0.40	0.50	0.60	0.70	0.75	0.80	0.85	0.90	1.00
Poem	आज अपने ही खटकने लग गए रिश्ते नाज़ुक थे चटकने लग गए रास्तों की मुश्किलें हल हो गईं आके मंज़िल पर भटकने लग गए क्या कभी पहले भी ऐसा था हुआ ...	illusionary triskaidekaphobia's unaccountab le	Nazakat husn se mashroot hoti gar zamane main. To ye muflis pre paker na bikte aane aane main.	Woyese to hum mile na kahin, ajnabi se they, Rishte na jane kaise kahan ke kabhi ke they. Dekha jo unko aankhon ne chupke se keya kaha, Alam ajeeb dil pe mere bebasi ke they. Majboor kar ke jane kahan ja ke chup gaye, ...	Nu scylun hergan hefaenricaes uard metudæs maecti end his modgidanc uerc uuldurfadur sue he uundra gihuaes eci dryctin or astelidæ he aerist scop aelda barnum ...	Ne vous étonnez pas, objets sacrés et doux, Si quelqu'air de tristesse obscurcit mon visage. Quand un savant crayon dessinait cette image J'attendais l'échafaud et je pensais à vous.	ALLAS! my worthi maister honorable, This landes verray tresor and richesse! Deth by thy deth hath harme irreparable Unto us doon: hir vengeable duresse Despoiled hath this land of the swetnesse ...	N-ew acrostic quatrain I-s brought in for the first time; C-ombination of these two forms K-eeps the beauty so sublime. Topic: Birthday of Nicole "Nick" Asuncion (March 20) ...	(Queer In Quatrain) Now so near, Now so far, You and I are In what a queer! ...	Promise Of A Child (Dramatic Monologue) March 31, 2020 Believe me my, tribe I'm your child I know your dream ...