Main project
Due date: 11:59pm on Sunday, 20 December 2026.
Use the corresponding invite link in this google doc (accessible with your EPFL account) to accept the project, and either join an existing team or create a new one. Once this is done, go to the course GitHub organization and locate the repo titled main-project-TEAM-NAME to get started.
Intermediate submission
- By Sunday November 8, you should have chosen a team and a topic.
- On Friday November 20 your team will submit a 1-2 page writeup. Your writeup should provide a preliminary introduction to the topic you will study and provide clear motivation for why they are interesting and/or relevant.
Final submission
You are required to hand in a PDF version of your report report.pdf (max 20 pages) and the source code used. You should not show the actual code in the PDF report, unless you want to point out something specific.
Your README.md should contain instructions on reproducing the PDF report from the quarto file. This can be useful if you have issues with the automatic generation of the PDF report right before the deadline. Your README.md should also include a brief description of the contributions of each team member, if you are a team of three students.
Checklist:
The goal of this project is quite broad, students are free to come up with their own ideas. Simulation studies are the designated topic and while we suggest below a list of topics, groups interested in studying a bit deeper one of the methodological concepts from this course (not listed below) are encouraged to approach the teachers during the exercises and discuss their ideas. Prospective topics for the final project will be gradually revealed during the lectures.
Part of the grade for the final project will be awarded for value added (e.g., simulation study answering a previously unclear question). All of the prospective topics that will be introduced during the lecture will have this element, and by half-way through the semester (when the final project will start) it should be clear through the examples what the project should aspire to. We will also discuss this in person at some point, likely on Week 6. The remaining part of the project will be awarded for
- quality of the report (clarity, readability, structure, referencing, etc.)
- graphical considerations (well chosen graphics with captions, referenced from the main text)
- concepts explored beyond the scope of the course (in the soft sense that they were not fully covered during classes)
- overall quality (scope, correctness, demonstration of understanding, etc.)
A project seriously lacking in any of the criteria above will be penalized.
Use of generative AI tools
The purpose of this project is to develop your ability to formulate scientific questions, design and conduct statistical investigations, and communicate your findings. All intellectual contributions must therefore be your own.
The use of generative AI tools (e.g., ChatGPT, Claude, Copilot) is therefore not permitted for content generation, including proposing research ideas, designing simulation studies, writing sections of the report, interpreting results, or generating code.
However, AI tools may be used for limited technical and editorial assistance, such as debugging code you have written yourself or checking spelling, grammar, and typographical errors in your own text.
As in scientific publishing, transparency and accountability are essential. If you use generative AI tools, you must briefly disclose their use in your report, specifying the tool and the purpose for which it was used.
The project should reflect your own reasoning and understanding. This should be evident throughout the report, particularly in the motivation of your research question, the design of your simulations, and the interpretation and discussion of your results. You are expected to understand and be able to explain every aspect of your submitted work.
Topics for the final project
Unless otherwise stated, you are not required to code everything from scratch (unless there is no available source code).
1. Cross-validation for PCA
Several cross-validation methods have been proposed to select the number of principal components in PCA. The project aims to compare three approaches:
- EM-based CV and repaired CV, covered in Week 4.
- CV based on matrix completion: Implement a third approach that relies on performing a singular value decomposition (SVD) with missing entries. Details are provided in the supplementary notes, with a more detailed treatment in Perry (2009), Section 5.3.
- Compare the three methods through a simulation study.
Students should investigate how the performance of these methods depends on the underlying dimension of the data and the level of noise, using simulated low-rank data matrices. Performance should be assessed in terms of the ability to select an appropriate number of principal components and the resulting reconstruction error.
The project should address the following question: Under which conditions does one cross-validation approach outperform the others in selecting the number of principal components?
An interesting extension would be to investigate how the performance of the methods changes with the proportion or pattern of missing entries.
2. Comparison of variable selectors in regression
Hastie et al. (2020) report some surprising results in their simulation study comparing variable-selection methods in linear regression. However, one important method, the adaptive lasso, is omitted.
The project aims to reproduce and extend parts of their simulation study:
- Reproduce a selection of the simulation settings considered by Hastie et al. (2020).
- Include the adaptive lasso in the comparison (and possibly elastic net).
- Compare the methods with respect to predictive performance, variable-selection accuracy, and sparsity.
Students should investigate how their conclusions depend on the signal-to-noise ratio, the correlation structure of the predictors, and the degree of sparsity of the true regression model.
The project should address the following question: Which variable-selection method should be preferred, under which conditions, and for which objective (prediction, variable selection, or sparsity)? In particular, does the method with the smallest prediction error also identify the relevant covariates most reliably?
The goal is not to identify a universally best method, but to understand the trade-offs between the competing approaches.
3. Comparison of cross-validation methods for temporally dependent data
Standard cross-validation methods can perform poorly when observations exhibit temporal or spatial dependence. In particular, when short-range autocorrelation in the errors is ignored, conventional cross-validation may favor overly complex models, leading, for example, to the selection of excessively small smoothing parameters (overfitting) in non-parametric regression.
Several modifications of standard cross-validation have been proposed to address this issue, including:
- Buffered cross-validation: Excluding observations within a certain distance of each validation point.
- Block cross-validation: Holding out contiguous blocks of observations rather than individual points.
- Neighborhood cross-validation: A recently proposed approach by Wood (2024).
The project aims to investigate how temporal dependence affects the performance of cross-validation methods and under which conditions the modified procedures offer an improvement.
Students should conduct a simulation study in a non-parametric regression setting (e.g., using smoothing splines), considering different strengths and structures of short-range autocorrelation (e.g., AR or MA Gaussian error processes).
The study should address the following questions:
- How does autocorrelation affect the smoothing parameters selected by standard cross-validation and the resulting estimation accuracy?
- To what extent can modified cross-validation procedures mitigate these effects?
- How does the performance of these procedures depend on the strength and range of temporal dependence?
An interesting extension would be to investigate how the size of the excluded neighborhood or validation block affects the results. In particular, is there a trade-off between reducing dependence and retaining enough observations for model fitting?
References: Chu and Marron (1991), Arlot and Celisse (2010), Roberts et al. (2017), Wood (2024).
4. Statistical inference with missing data: EM versus imputation
Missing observations are common in statistical applications, and different approaches have been proposed to deal with them. Among these, the expectation-maximization (EM) algorithm allows maximum likelihood estimation from incomplete data, while imputation methods replace missing observations with plausible values before applying standard statistical procedures.
The project aims to investigate how different approaches to handling missing data affect the quality of statistical inference.
Students should:
- Consider a statistical model with a known data-generating mechanism (e.g., a multivariate Gaussian model).
- Simulate missing observations under different missingness mechanisms (MCAR, MAR, and possibly MNAR), varying the proportion of missing values.
- Compare maximum likelihood estimation using EM with alternative approaches such as single or multiple imputation. A description of imputation methods can be found here.
- Evaluate the methods in terms of bias, mean squared error, and, where appropriate, confidence interval coverage.
The project should address the following question:
Under which assumptions and missingness scenarios can reliable statistical inference be obtained from incomplete data, and what are the consequences of using an inappropriate method?
An interesting extension would be to investigate how violations of the assumptions underlying the different methods affect their performance.
5. Diving into one of the course topics, e.g., MM algorithms or Monte Carlo integration (Week 8)
The project should be centered around a clearly formulated research question that can be investigated through a simulation study. Simply implementing or illustrating an algorithm is not sufficient: the project should aim to reveal something about its behavior, limitations, or performance under different conditions.
Students choosing this option must discuss their proposal with the teachers to ensure that the research question is sufficiently interesting and that the project is feasible within the available time.