Knowledge Gradient Ascent
Working notes on what I am currently learning: generative models, efficient ML systems, and the occasional fight with a GPU cluster. I write them to understand things properly, and leave them here in case they are useful to you.
-
OCT 2026entry 07 · 9 parts
Latest Trends in Diffusion Language Models
p.1the year text diffusion shipped p.2borrowing an autoregressive brain p.3the parallel decoding wall p.4taking it back: editing and self-correction p.5what should the noise be? p.6thinking without writing p.7teaching it to reason p.8serving, training and measuring p.9where it is going: the open problems -
SEP 2026entry 06 · 5 topics
How to Prepare for an ML Interview in 2026
A running syllabus for ML interviews: the questions that actually get asked, answered from first principles, one topic at a time, with the diagrams and animations I wanted while I was preparing.
-
MAY 2026entry 05 · 5 parts
Catching up on Flow based LLMs
A five-part build from first principles: why generation is transport, how it survives contact with discrete tokens, and how flow matching, diffusion, and masked diffusion turn out to be one framework.
-
22 MAR 2026entry 04
-
MAR 2026entry 03 · 3 parts
Distributed Training from Scratch
A three-part, from-scratch tour of training large models across many GPUs: the batch-size hierarchy, data and model sharding with DDP and FSDP, and what actually breaks at 256 GPUs.
-
APR 2024entry 02 · 3 topics
The Math Under the Model
LoRA is a low-rank factorisation. Muon is an SVD with the stretch thrown away. RoPE is a rotation. Most of what is new in deep learning is linear algebra, calculus and probability under a new name, and this is that math, one question at a time: figures you drag, guesses you commit to before the answer, and code that runs in the page.
-
1 OCT 2023entry 01
SSH Keys Demystified: A Researcher's Guide to Cluster Access
Picture this. You’ve just been granted access to a shiny HPC cluster. Four GPUs waiting for you. You open your terminal, type ssh cluster: