Skip to content
All tags

#triton

3 posts
aiguideReading Stanford CS149 Parallel Computing

, the Fastest Kernel on an H100: Five AI Kernels, Graded on Your Work Log | Stanford CS149 Programming Assignment 5

The last programming assignment in CS149 Fall 2025 is open-ended. Pick at least one of five kernels (Histogram, a 1D occupancy decoder, FlashAttention, a 3D heat equation with RK4, SwiGLU) and make it faster than its PyTorch baseline on an H100. You can write CUDA, Triton, or TileLang, and you may use LLMs. There is no speed threshold. The grade depends on a work log that shows what you measured at each step, what hypothesis you formed, and why you stopped. The H100 job queue and leaderboard need a SUNet ID; outside Stanford you can only run eval.py on your own NVIDIA GPU.

#cs149#ai-course#stanford#gpu#cuda#triton#flashattention#performance
techguideSelf-Hosted Inference

Self-Hosted Inference Overview: When Running Your Own Models Makes Sense

The key question in self-hosted inference isn't how fast the engine is — it's your GPU utilization. A fully saturated A100 costs ~$0.70 per million output tokens; at 10% utilization that becomes $7, more than most cloud APIs. This overview maps seven tools across three layers to help you decide which layer you need.

#self-hosted#llm-inference#vllm#sglang#tensorrt-llm#triton#ray-serve#gpu
aideep-diveReading Stanford CS336 Language Modeling from Scratch

Benchmark and Profile Before Writing a Triton Kernel | Stanford CS336 Lecture 6

Lecture 6 turns GPU principles into kernels: benchmark scaling across shapes, profile actual calls and time, then implement GeLU, softmax, reductions, and tiled matrix multiplication in Triton. Speed begins with measuring correctly.

#cs336#triton#gpu#cuda#performance