Ryan PégoudWriting on inference efficiency, compression, and GPU kernels.https://rpegoud.github.io/Cutting LLM Memory by 84%: A Deep Dive into Fused Kernelshttps://rpegoud.github.io/articles/fused-linearce/https://rpegoud.github.io/articles/fused-linearce/Fusing the linear projection and cross-entropy loss into a single tiled Triton kernel cuts peak training memory by 84%, by never materializing the full logits tensor.Fri, 01 May 2026 00:00:00 GMTLearning Triton One Kernel at a Time: Softmaxhttps://rpegoud.github.io/articles/softmax/https://rpegoud.github.io/articles/softmax/A numerically-stable, online softmax kernel fuses the max and sum reduction passes into one, cutting global memory reads from three to two while exceeding PyTorch's throughput.Tue, 23 Dec 2025 00:00:00 GMTLearning Triton One Kernel at a Time: Matrix Multiplicationhttps://rpegoud.github.io/articles/matmul/https://rpegoud.github.io/articles/matmul/An illustrated introduction to tiled matrix multiplication, the memory hierarchy and memory coalescing.Fri, 10 Oct 2025 00:00:00 GMTLearning Triton One Kernel at a Time: Vector Additionhttps://rpegoud.github.io/articles/vector-addition/https://rpegoud.github.io/articles/vector-addition/A first-principles vector-add kernel in Triton demonstrates the GPU execution hierarchy — threads, warps, blocks, streaming multiprocessors — before any optimization is possible.Thu, 25 Sep 2025 00:00:00 GMT