Generative AI: 1. Ethics 2.CLIP

Project Plan and Milestones

Weekly Plan

Date	Task	Completion
Week 4	Paper reading. Existing RLHF and RLAIF exploring. Red-teaming dataset exploring.	√
Week 5	Familiarizing with Dromedary, SALMON, Llama base models.	√
Week 6	Evaluation of different base models. Choice of using Llama 2 model as our baseline.	√
Week 7	Red teaming dataset exploration. Reading about ethical theories.	√
Week 8	ETHICS dataset discovering.	√
Week 9	ETHICS dataset formatting for Llama fine-tuning and evaluation. Llama supervised model fine-tuning	√
Week 10	Evaluation of Llama model before and after fine-tuning with ETHICS dataset. Mid-term Presentation & Start writing the Wikipedia page with the plan.	√
Week 11	Read about Reinforcement learning using PPO. Re-formatting deontology dataset. Creation of the preference model.	√
Week 12
Week 13
Week 14	Write the Wikipedia page & Final presentation	√

Milestone 1

• Define Research Questions: Establish clear, focused questions to guide the project.
• Literature Review: Conduct a comprehensive review of existing studies in AI ethics.
• Ethical Theory Exploration: Investigate various ethical theories to ground your research in a solid theoretical framework.
• Ethical Dataset Identification: Locate datasets for quantitative AI ethics evaluation, such as red teaming datasets.

Milestone 2

Refine Research Goals: Sharpen the focus and scope of the research based on initial findings. Dataset Finalization: Select the most appropriate dataset after exploration and evaluation. Model Selection and Fine-Tuning: Settle on the LLaMA model and fine-tune it by deploying GPU resources. Model Evaluation: Conduct a thorough evaluation of the model, focusing on its ethical implications and performance.

Milestone 3

• Get our Preference and the Reinforcement learning models.
• Analyze the results.
• Write the Wikipedia page.