Generative AI: 1. Ethics 2.CLIP

Project Plan and Milestones

Weekly Plan

Date	Task	Completion
Week 4	Paper reading. Existing RLHF and RLAIF exploring. Red-teaming dataset exploring.
Week 5		Familiarizing with Dromedary, SALMON, Llama base models.
Week 6	Evaluation of different base models. Choice of using Llama 2 model as our baseline.
Week 7	Red teaming dataset exploration. Reading about ethical theories.
Week 8	ETHICS dataset discovering.
Week 9	ETHICS dataset formatting for Llama fine-tuning and evaluation. Llama supervised model fine-tuning
Week 10	Evaluation of Llama model before and after fine-tuning with ETHICS dataset.	Mid-term Presentation & Start writing the Wikipedia page with the plan.
Week 11	Read about Reinforcement learning using PPO.	Re-formatting deontology dataset. Creation of the preference model.
Week 12
Week 13
Week 14		Write the Wikipedia page & Final presentation

Milestone 1

• Choose the project subject.
• Read papers about the existing studies in this field.
• Define our research questions.

Milestone 2

• Refine our research questions.
• Explore different ethical theories.
• Find an appropriate dataset.
• Evaluate our fine-tuned supervised model.

Milestone 3

• Get our Preference and the Reinforcement learning models.
• Analyze the results.
• Write the Wikipedia page.