Generative AI: 1. Ethics 2.CLIP

Project Plan and Milestones

Weekly Plan

Date	Task	Completion
Week 4	Paper reading. Existing RLHF and RLAIF exploring. Red-teaming dataset exploring.	√
Week 5	Familiarizing with Dromedary, SALMON, Llama base models.	√
Week 6	Evaluation of different base models. Choice of using Llama 2 model as our baseline.	√
Week 7	Red teaming dataset exploration. Reading about ethical theories.	√
Week 8	ETHICS dataset discovering.	√
Week 9	ETHICS dataset formatting for Llama fine-tuning and evaluation. Llama supervised model fine-tuning	√
Week 10	Evaluation of Llama model before and after fine-tuning with ETHICS dataset. Mid-term Presentation & Start writing the Wikipedia page with the plan.	√
Week 11	Read about Reinforcement learning using PPO. Re-formatting deontology dataset. Creation of the preference model.	√
Week 12
Week 13
Week 14	Write the Wikipedia page & Final presentation	√

Milestone 1

• Define Research Questions: Establish clear, focused questions to guide the project.
• Literature Review: Conduct a comprehensive review of existing studies in AI ethics.
• Ethical Theory Exploration: Investigate various ethical theories to ground your research in a solid theoretical framework.
• Ethical Dataset Identification: Locate datasets for quantitative AI ethics evaluation, such as red teaming datasets.

Milestone 2

• Refine Research Goals: Sharpen the focus and scope of the research based on initial findings.
• Dataset Finalization: Select the most appropriate dataset after exploration and evaluation.
• Model Selection and Fine-Tuning: Settle on the LLaMA model and fine-tune it by deploying GPU resources.
• Model Evaluation: Conduct a thorough evaluation of the model, focusing on its ethical implications and performance.

Milestone 3

Develop Advanced Models: Implement Preference and Reinforcement learning models, integrating them with the fine-tuned LLaMA model. In-Depth Analysis: Analyze the models' outcomes, assessing performance, identifying defects, and investigating specific issues like coherence and degeneration. Documentation and Dissemination: Create a comprehensive Wikipedia page summarizing the project's findings. Final Deliverables: Compile all project materials, including a well-documented GitHub repository.