1. Exploring Similarities in Themes and Style across Age Groups in Children’s Artwork using CLIP 2. Generative Fashion and Assisted Clothing Design with DragGAN

"ANYTHING TO IMAGE from Culture Perspective"

Current State & future plan:

Finished:

ImageBind Model Deployment State Diffusion Model Deployment Website Demo Website Frontend Design

Ongoing (1 - 8 Dec)

Dataset Collection (Deadline December 5) Wiki Page for the project (Deadline December 6) Website Improvement (Deadline December 8)

Future weeks (8 - 15 Dec)

Search Culture Benchmark (Deadline December 10) Generation Culture Benchmark (Deadline December 15)

Bases:

Part 1 ImageBind & Stable Diffusion Introduction Part 2 Website Part 3 Experiment Plan for Search Function Part 4 Experiment Plan for Generation Function

Part 1.-

The Introduction of GenAI Models

Imagebind’s joint embedding space enables novel multimodal capabilities

This is a model that can be used to generate and modify images based on text prompts. It is a Latent Diffusion Model that uses a fixed, pretrained text encoder (OpenCLIP-ViT/H).

Part 2- Website

Creation of website of the code, mis front end with back end to hace a results: https://dataoni.com/fdh-project-frontend-page/

Part 3: Experiment Plan for Search Part & Data Set construction

Disseminate the diverse cultural perspectives of the world

"From Cultural Perspectives": this involve recognizing and accurately depicting cultural symbols, costumes, architecture, and other elements unique to various cultures.

Cultures around the world Civilizations through time

3.2 Dataset

Data set generated manually, with more than 300 samples analyzed between different cultures over time and also that occupy a different geographical space.

Each sample comprises at least 2 of the followeing elementos Audio, Image, Tec=xt.

Instruments:

China:

String instrument: guzheng, pipa, erhu, gaohu, harp , dulcimer Blowing instrument: flute, xiao, suona. Percussion instrument: gongs, drums, and wooden fish.

European:

String instrument: violin, cello, guitar, harp Woodwind instrument: Flute, clarinet, oboe. Brass instruments: trumpet, trombone, horn Keyboard instrument: piano, organ

America:

String instrument: chalapatita, bajo, cuatro. Blowing instrument: Pan Flute, Saxophone. Percussion instrument: Conga drums, marimba.

Clothes:

Historical period: Ancient Clothing: Egyptian, Greek, and Roman clothing. Medieval Costume: Renaissance Clothing: 17th to 19th Century Costumes: 20th Century Clothing:

Geography & culture:

Asian Costumes: African clothing European clothing: Middle Eastern and Arab clothing: Costumes of the Americas:

Paintings:

Oil paintings: Renaissance: Baroque: Rococo: Romanticism: Impressionism: Post-Impressionism:. Modern and contemporary art:

Sketching

Chinesse pictures:

Lanscape Flowers and Birds Characters

Food

Asia America European Middle Eastern

Experiment Plan for Search Part:

Remark: Only Imagebind

Basic Perception Distinguish between different types of items such as food and paintings or musical instruments and clothing

General Culture Perception Distinguish between same type of items but from different areas or times such as Chinese food and Mexico food or Medieval Costume and 20th Century Clothing

Accurate Culture Perception Distinguish between same type of items but from different specific culture such as Renaissance Oil Painting and Impressionism Oil Painting or Chinese New Year and Mexican Day of the Dead

Part 4

Approach to studying AI model’s intelligence

Key questions: How can we measure the intelligence of an LLM that has been trained on an unknown but extremely vast corpus of web-text data?---The Standard Benchmark

This approach is designed to separate true learning from mere memorization, and is backed up by a rich theoretical framework [SSBD14, MRT18].

Definition of intelligence: Someone who:

Reasoning Planning Problem-solving Abstract thinking Understanding complex ideas Fast learning Ability to learn from experience.

4.1 Approach to studying AI model’s intelligence

They propose a different approach which is closer to traditional psychology rather than machine learning, leveraging human creativity and curiosity.

They aim to generate novel and difficult tasks and questions that convincingly demonstrate that GPT-4 goes far beyond memorization, and that it has a deep and flexible understanding of concepts, skills, and domains

Experiment plan:

Clothes: The ability of the models to recognize, differentiate, and accurately generate traditional and culturally specific attire. Paintings: Assessing the models' capability to identify and replicate cultural themes, styles, and historical contexts in paintings. of specific cultural art forms. Food: Evaluating the models' proficiency in identifying and visually representing traditional dishes from various cultures.

Multimodal and Integrative Ability Basic Sound & Text & Image Instruments & Text & Image Provide a sample of an instrument's sound, like a piano piece or violin solo. The text describes the instrument and the style of music played, such as "a piano concerto from the Romantic period". Request the model to generate an image that integrates the sound and text, depicting the instrument and the style of music.

4.3 Evaluation and coherence

Evaluation Recognize and Differentiate: distinguish between different cultural elements within the same category Accuracy in Representation: portray the identified cultural elements in terms of visual appearance, cultural context, and associated symbolism. Maintaining cultural accuracy and sensitivity System Coherence Put the generation result back into the search model