Home » Departments and Centres » Academic catalogue » Course detail

Open sidebar menu

Text Analysis and Causal Machine Learning (ECO-AD-TEXTCAUS)

ECO-AD-TEXTCAUS

Description

Advanced Topics in Machine Learning for Social Sciences and Economics

This 10-hour course provides a structured introduction to machine learning methods for natural language processing, dimensionality reduction, and causal inference, with applications in social sciences and economics. Each 2-hour session integrates conceptual discussion and methodological foundation with hands-on coding exercises designed to reinforce understanding through practical implementation.
Note: The syllabus is tentative and may be subject to change.
Prerequisites
Prior exposure to foundational machine learning concepts (as covered in Professor Gottard’s course), linear algebra, regression analysis, and causal inference is assumed. Familiarity with basic programming in Python or R, and with statistical software (e.g., Stata) for data analysis, is helpful but not required.

Learning Outcomes
By the end of this course, participants will be able to:
1. Understand how to preprocess and analyze textual data using standard NLP pipelines.
2. Understand how transformer-based language models work, and their use in text representation, classification, and analysis.
3. Apply principal component analysis for linear dimensionality reduction.
4. Utilize matrix completion techniques to estimate and recover missing values in data matrices.
5. Implement causal machine learning methods to estimate treatment effects using existing software packages.
6. Critically evaluate the research design, assumptions, and empirical findings of published papers utilizing these methods.

Assessment
• Class participation: Active engagement during discussions and coding exercises.
• Referee report: Students are required to write a critical review of a published or working paper related to the topics covered in the course. The objective is to assess their understanding of the methods discussed and their ability to critically engage with applied empirical research in the social sciences and economics. The report will be evaluated based on:
(i) understanding of the empirical strategy, including its assumptions, strengths, and limitations;
(ii) identification of potential limitations in the research design, estimation strategy, or data construction;
(iii) discussion of possible improvements or alternative approaches to strengthen the analysis and robustness of the results.

Topics covered
Lesson 1:
Text pre-processing
Bag-of-words approaches: TF-IDF
Word embeddings (Word2Vec)
Lesson 2:
BERT for text representation and classification
Large Language Models: transformer architecture, prompting, and decoding
Lesson 3:
Principal Component Analysis
Matrix Completion
Lesson 4:
Post-Double-Selection Lasso
Double/Debiased Machine Learning (DML)
Lesson 5:
Post-Double-Selection IV-Lasso
DML-IV

Selected sources:
•           Devlin, J., Chang, M.W., Lee, K., et al. (2019) BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, Minneapolis, 2-7 June 2019, 4171-4186.
•           Vaswani, Ashish et al. (2017). “Attention is All You Need”. In: Advances in Neural
Information Processing Systems 30 (NeurIPS 2017), pp. 5998–6008. url:
https://arxiv.org/abs/1706.03762.
•           Mikolov, Tomas et al. (2013). Efficient Estimation of Word Representations in Vector Space.
arXiv: 1301.3781 [cs.CL]. url: https://arxiv.org/abs/1301.3781.
•           Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. and Robins, J. (2018), Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21: C1-C68. https://doi.org/10.1111/ectj.12097.
•           Victor Chernozhukov, Hansen, Kallus, Spindler, Syrgkanis, (2025), Applied Causal Inference Powered by ML and AI. Chapters: 4, 9, 13.
•           Mutlu Yuksel and Yigit Aydede, 2025, Causal Inference and Machine Learning: In Economics, Social, and Health Sciences. Chapters: 19, 22, 31.
•           Belloni, Alexandre, Victor Chernozhukov, and Christian Hansen. 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects." Journal of Economic Perspectives 28 (2): 29–50.
• Jolliffe I.T., Principal component analysis, Springer, 2002, 2nd ed., Chapters 1,2.