Generative AI: 1. Ethics 2.CLIP

Project Plan and Milestones

Weekly Plan

Date	Task	Completion
Week 4	Paper reading. Existing RLHF and RLAIF exploring. Red-teaming dataset exploring.	√
Week 5	Familiarizing with Dromedary, SALMON, Llama base models.	√
Week 6	Evaluation of different base models. Choice of using Llama 2 model as our baseline.	√
Week 7	Red teaming dataset exploration. Reading about ethical theories.	√
Week 8	ETHICS dataset discovering.	√
Week 9	ETHICS dataset formatting for Llama fine-tuning and evaluation. Llama supervised model fine-tuning	√
Week 10	Evaluation of Llama model before and after fine-tuning with ETHICS dataset. Mid-term Presentation & Start writing the Wikipedia page with the plan.	√
Week 11	Read about Reinforcement learning using PPO. Re-formatting deontology dataset. Creation of the preference model.	√
Week 12
Week 13
Week 14	Write the Wikipedia page & Final presentation	√

Milestone 1

• Define our research questions.
• Read papers about the existing studies in this field.
• Explore different ethical theories.
• Find an appropriate ethical dataset.

Milestone 2

• Refine our research questions.
• Finish the whole dataset.
• Run the model and fine-tuned it on the GPU.
• Evaluate our fine-tuned supervised model.

Milestone 3

• Get our Preference and the Reinforcement learning models.
• Analyze the results.
• Write the Wikipedia page.