Justin Deschenaux

My research focuses on generative models (diffusion, flow matching, GANs, VAEs, autoregressive LLMs, ...). While I primarily supervise projects in these areas, I'm open to other topics if you have relevant experience and a specific research idea. For example, if you've taken an advanced course and want to build on topics explored in this course, let's discuss your interests.

1. We should expect to have weekly 30-45 minute meetings. I can allocate more time when we have results to discuss or are working on a paper. If you don't have meaningful progress to report in a given week (due to illness, exams, or other circumstances), don't worry, just let me know in advance so that we can reschedule or cancel the meeting. You can always message me on Slack if you need something.

2. I may need to cancel, shorten, or reschedule meetings during busy periods (e.g., conference deadlines). Again, feel free to message me on Slack if you're stuck or have urgent questions. When asking for help, please try to explain:

   1. What you're trying to do
   2. The errors you're encountering
   3. What solutions you've already tried and why they didn't work

   This helps me provide more targeted assistance rather than suggesting solutions you've already attempted.

3. Try to come prepared to meetings with a summary of your key progress and challenges. You don't need to detail every step or small issue that you solved (like cluster setup problems that you fixed). Instead, we would rather spend most time focusing towards the project goals.

4. Our lab holds regular group meetings on various topics (RL, sequence modeling, etc.). These meetings are optional, and you are welcome to attend them based on your interests and project relevance. Occasionally, we will schedule short presentations for you to present your project to the other students.

1. Let's discuss your availability and experience at the start of the semester. This helps me scope the project appropriately:

   1. If you're new to research, I'll try to assign you more focused tasks
   2. If you have research experience or want to allocate more time to the project, I can give you broader responsibilities and more autonomy
   3. Note that EPFL sets specific time requirements for research projects (detailed below). Hence, we will generally expect you to meet these.

2. My role is to guide your research and help you overcome obstacles. The goal is for you to develop independent research skills. When facing technical problems, being able to figure out a solution yourself is an important skill to develop. At the same time, if you get stuck and have no idea how to proceed, or your debugging ideas don't work, you should reach out to me. For example, if you have problems training your neural networks, you could:

   1. Test on simple datasets (e.g. on 2D distributions), or even try overfitting a single batch
   2. Adjust hyperparameters (e.g. learning rate and regularization)
   3. Implement common fixes (e.g. gradient clipping, this page talks about ideas to debug neural networks)

3. Early in the semester (usually in the first two weeks), you should give a short 5-15 minute presentation about your project to other students in the lab. This helps ensure you understand the background material and project goals, while also getting valuable feedback from peers working on related topics. Keep the presentation simple - focus on explaining the key ideas, for example using figures, tables, and equations from the background material. As your project progresses, you may present updates to get additional feedback from the group. While preparing presentations takes some time, the insights and suggestions from these discussions can significantly improve your project. The presentations should be clear, but don't spend excessive time on them. The main focus is to advance your project.

4. Publications are not a requirement for your project. While we may discuss submitting a paper early in the project, research is inherently uncertain - it's difficult to predict which ideas will succeed. If you expected to publish, but the results do not lead to a publication, don't be discouraged. Instead, focus on the skills and knowledge you can gain for future projects. That said, publications can strengthen your applications for future research positions in both academia and industry.

5. Feel free to message me on Slack anytime. If you're struggling with any aspect of the project or have concerns, make sure to let me know. Don't hesitate to send reminders if I forget to follow up on something.

• Expected time commitments:

  1. Master's thesis: 4 months full-time
  2. 12 ECTS project: 14 weeks (312 hours total)
  3. 8 ECTS project: 14 weeks (228 hours total)

  Note: If we aim to publish your work at a top conference with you as first author, expect to spend additional time beyond these hours, particularly on paper writing and revisions.

• It is a good idea to start writing your report early and document your progress regularly. For example, you could:

  1. Write down experimental details while they're fresh (model architecture, hyperparameters, training steps)
  2. Include key results and generate plots as you train/evaluate models
  3. Initial drafts don't need to be polished - focus on the technical details
  4. This makes the final write-up much easier than trying to remember everything at the end

  Below, I outline a suggested structure that your semester project report could follow. You can find a LaTeX report template here.

• Introduction: Describe the problem you're addressing and summarize relevant prior work.

• Background: This section should explain the prior methods that your project builds upon. For example, if your project uses GANs, explain what GANs are and which specific GAN architecture you're building from. Focus on what's directly relevant to your project.

• Method: Clearly describe your approach and how it differs from existing work. This could include:

  1. Changes to model architecture
  2. New training objectives
  3. Novel combinations of existing techniques
  4. Any other technical innovations

• Experiments: Document your experimental setup and results clearly:

  1. Describe your datasets, metrics, and evaluation protocol
  2. Detail important hyperparameters and implementation choices
  3. Present results with clear visualizations (plots, tables, examples)
  4. Compare against relevant baselines when possible
  5. Include ablation studies to understand which components matter

• Conclusion: Summarize your project outcomes and critically analyze your results:

  1. Were the initial goals achieved?
  2. What challenges did you encounter?
  3. Highlight any suspicious results that need further investigation (e.g., unexpected patterns that might indicate bugs)
  4. Discuss potential future directions for the project

• Remember that it's perfectly normal if the approaches didn't work, this is the nature of research. Be honest about what worked and what didn't. Failed experiments often provide valuable insights for future work. The key is to clearly document your observations, successful or not, and write down possible explanations if you have any.

• We will use Notion to maintain a document for your project that includes:

  1. Key papers and references
  2. Meeting notes
  3. Results (plots, tables, experiments)
  4. Project goals and milestones

  This document should also allow us to quickly access important information like experimental settings and implementation details from previous work. If you're sharing with me a draft of your report, there is no need to add plots and tables in notion as well.

• It would be ideal if you created a Notion sub-page before each meeting with:

  1. A brief summary of main points to discuss
  2. Updates on goals from previous meeting
  3. Any important questions or blockers

  This would help us prioritize our meeting time effectively.

• Other resources on how to organize yourself for projects:

  • 1. Grigoris Chrysos' advice
  • 2. Alex Hägele's advice