Generative AI: 1. Ethics 2.CLIP

Project Plan and Milestones

Weekly Plan

Date	Exploration	Application	Evaluation	Report
Week 4	Paper reading Existing RLHF and RLAIF exploring Red-teaming dataset exploring
Week 5		Familiarizing with Dromedary, SALMON, Llama base models.
Week 6			Evaluation of different base models. Choice of using Llama 2 model as our baseline.
Week 7	Red teaming dataset exploration. Reading about ethical theories.
Week 8	ETHICS dataset discovering.
Week 9		ETHICS dataset formatting for Llama fine-tuning and evaluation. Llama supervised model fine-tuning
Week 10			Evaluation of Llama model before and after fine-tuning with ETHICS dataset.	Mid-term Presentation & Start writing the Wikipedia page with the plan.
Week 11	Read about Reinforcement learning using PPO.	Re-formatting deontology dataset. Creation of the preference model.
Week 12
Week 13
Week 14				Write the Wikipedia page & Final presentation

Milestone 1

• Read papers and define potential directions for our project.

Milestone 2

• Understand different ethical theories.
• Fine-tune supervised model

Milestone 3

• Model Evaluation