Tracking a Historic Market Crash through Articles

Methodology

Data Collection

Bloomberg Businessweek is an economic and business-oriented weekly magazine published by Bloomberg, a global multimedia corporation. Renowned for its in-depth coverage, analysis, and commentary on global business, finance, technology, markets, and economics, this publication offers a comprehensive view of various industry trends, corporate strategies, and market dynamics. The dataset is generated from Bloomberg website through their public APIs by Philippe Remy and Xiao Ding. It contains 450,341 news from 2006 to 2013.

A sample of the content of Bloomberg newspaper.

Project Plan and Milestones

Weekly Project Plan

Week	Tasks	Completion
Week 4	Read topic-related Acadamic papers to figure basic paradigms Brainstorm and present initial ideas for the project	✓
Week 5	Learn the standard process for NLP preprocessing. Find suitable news datasets and economic crisis labels.	✓
Week 6	Initially define the entire project's workflow. Configure the development environment and master the relevant software and libraries. Finishing data preprocessing on the news dataset.	✓
Week 7	Completing the feature engineering construction and basic pipeline for the TF-IDF based model. (Completed) Completing the feature engineering construction and basic pipeline for the sentiment dictionary-based model.	✓
Weeks 8–9	Choose and train appropriate machine learning models to build feature-to-label mappings. Learn and implement cross-validation of timing models to validate model performance. Analyze the experimental results and summarize preliminary conclusions.	✓
Week 10	Prepare slides for the midterm presentation.(Completed) Fill in information on wiki.	✓
Week 11	Expand the fine-grained news dataset and replenish the economic analysis metrics. Introduce pre-trained models with transformer architecture to optimize the extraction of sentiment features. Explore a variety of deep learning and machine learning techniques for optimization.
Week 12	Complete project workflows on new datasets with new time series models. Compare the results and analyze the correlations between sentiment scores and different financial indicators. Decision fusion for enhancing model performance.
Week 13	Achieve visual representation to display news trends, sentiment analysis outcomes, and predictive metrics. Finalize modifications and refinements for the project's concluding model iterations.
Week 14	Write the report. Prepare for the final presentation.

Milestones

Milestone 1

• Draft a comprehensive project proposal outlining aims and objectives.
• Identify datasets with appropriate time granularity and relevant economic labels.
• Prepare and clean selected datasets for analysis.

Milestone 2

• Master the NLP processing workflow and techniques.
• Construct TF-IDF representation and emotional indicators in news data.
• Conduct preliminary model adjustments to enhance accuracy based on initial data.

Milestone 3

• Implement pre-trained models for sentiment analysis and integrate them into the project.
• Apply decision fusion techniques to optimize model performance.
• Refine and fine-tune the models based on the results and feedback.

Milestone 4

• Prepare the final presentation summarizing and visualizing the project findings and outcomes.
• Create and finalize content for the Wikipedia page, documenting the project.
• Conduct a thorough project review and ensure all documentation is complete and accurate.

References

• Financial News Dataset from Bloomberg and Reuters by Philippe Remy (2015)