Samy Jelassi Headshot

Samy Jelassi

I am a Research Fellow at the Center of Mathematical Sciences and Applications (CMSA) at Harvard University. My hosts are Boaz Barak and Sham Kakade.

I am working on problems related to large language models, spanning problems from creation and optimization of architectures to length generalization. Previously, my PhD thesis focused on understanding the implicit bias induced by optimization algorithms and architectural choices.

Prior to coming to Harvard, I did my PhD at Princeton University advised by Boris Hanin. During that time, I interned at Facebook AI Research, Google Deepmind and Google Research. And before that, I undergraduated at Ecole Normale Superieure de Lyon in France.

Selected Works

(full list)

Mixture of Parrots: Experts improve memorization more than reasoning
Samy Jelassi, Clara Mohri, David Brandfonbrener, Alex Gu, Nikhil Vyas, Nikhil Anand, David Alvarez-Melis, Yuanzhi Li, Sham M. Kakade, Eran Malach
In submission, Oral presentation (top 10%) at the Mathematics of Modern Machine Learning (M3L) workshop, (NeurIPS 2024).
[Blog]

Repeat after me: Transformers are better than state space models at copying
Samy Jelassi, David Brandfonbrener, Sham M. Kakade, Eran Malach
41st International Conference on Machine Learning (ICML), 2024.
[Blog]

Universal Length Generalization with Turing Programs
Kaiying Hou, David Brandfonbrener, Sham Kakade, Samy Jelassi*, Eran Malach*
In submission.