Course Schedule
Being less concrete further out, the course scheduling is tentative and subject to changes.
Introduction, AI systems foundation
Week 1
- 08/24
- Fill out background survey by 08/25
- 08/26
Reading: The Bitter Lesson (required); Hidden Technical Debt in Machine Learning Systems (optional); Software 2.0 (optional)
- 08/28
- Inference Speedrun out
Week 2
- 08/31
Reading: Floating Point and IEEE 754 (optional); Using FP8 and FP4 with Transformer Engine (optional); Defeating Nondeterminism in LLM Inference (optional)
- 09/02
Reading: mlsysbook: Quantization and Precision (required); Building a quantization paradigm from first principles (optional)
Week 3
- 09/07
Reading: ZipLLM [NSDI’26] (optional); TensorDex [SOSP’26] (optional)
Pickle safety: Demo
- 09/09
AI infrastructure
Week 4
- 09/14
Reading: mlsysbook: Hardware Acceleration (required); Memory wall UVA tech report (optional); Roofline model UC Berkeley tech report (optional); OpenAI Jalapeño first results (optional)
- 09/16
Reading see 09/14
Systolic array: Demo
Inference Speedrun artifact due (09/16)
- 09/17
Project team signup due (09/17)
Week 5
- 09/21
Reading: mlsysbook: Compute Infrastructure (required); Hoffman et al. Scaling Laws (DeepMind) (required); Kaplan et al. Scaling Laws (OpenAI) (optional)
- 09/23
Reading see 09/21
Reliability
Week 6
- 09/28
Reading: mlsysbook: Storage Infrastructure (required); Marin Hero Pretraining Run (optional); Xiaomi MiMo RL Run (optional)
- 09/30
Reading see 09/28
Project proposal due (09/30)
LLM inference and performance engineering
Week 7
- 10/05
Reading day (no class)
- 10/07
Reading: mlsysbook: Distributed Training (required); ZeRO paper (optional)
Week 8
- 10/12
Reading: mlsysbook: Distributed Training (required); ZeRO paper (required); PipeDream paper (optional); GPipe paper (optional)
Paper presentation signup due (10/12)
- 10/14
Reading see 10/12
Student paper presentations
For guidance on preparing slides and paper reviews, see the Presentation and Paper Review sections of the syllabus.
Week 9
- 10/19
DeepSeek Elastic Compute (DSec): A Sandbox Infrastructure for Effective Agentic Training at Scale
FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution
Review questions: Link
- 10/21
RobustRL: Role-Based Fault Tolerance System for RL Post-Training [USENIX OSDI 2026]
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
Review questions: Link
Week 10
- 10/26
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU [ICML 2023]
LLM-as-a-Verifier: A General-Purpose Verification Framework
Review questions: Link
- 10/28
Project checkpoint presentation
Week 11
- 11/02
A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing
CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving [SIGCOMM 2024]
Review questions: Link
- 11/04
Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms [USENIX OSDI 2026]
FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving [MLSys 2025]
Review questions: Link
Week 12
- 11/09
DFlash: Block Diffusion for Flash Speculative Decoding
H2O: heavy-hitter oracle for efficient generative inference of large language models [NIPS 2023]
- 11/11
Hawkeye: Hardware-Aware GPU Kernel Optimization with Minimal Supervision
PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU [ACM SOSP 2024]
Week 13
- 11/16
Efficient Memory Management for Large Language Model Serving with PagedAttention [ACM SOSP 2023]
Mirage: A Multi-Level Superoptimizer for Tensor Programs [OSDI 2025]
- 11/18
DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving [USENIX OSDI 2024]
CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation
Inference Speedrun report due
Week 14
- 11/23
KernelBench: Can LLMs Write Efficient GPU Kernels? [ICML 2025]
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness [NIPS 2022]
- 11/25
Thanksgiving recess (no class)
Week 15
- 11/30
Project presentation I
- 12/02
Project presentation II
Week 16
- 12/07
Project presentation III
- 12/09
Project everything due