Reading List

Being less concrete further out, the reading list is being incrementally updated to include more papers as we go.

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness [NIPS 2022]

Efficient Memory Management for Large Language Model Serving with PagedAttention [ACM SOSP 2023]

FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU [ICML 2023]

H2O: heavy-hitter oracle for efficient generative inference of large language models [NIPS 2023]

Accelerating Large Language Model Decoding with Speculative Sampling

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management [USENIX OSDI 2024]

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving [USENIX OSDI 2024]

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve [USENIX OSDI 2024]

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU [ACM SOSP 2024]

FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving [MLSys 2025]

KTransformers: Unleashing the Full Potential of CPU/GPU Hybrid Inference for MoE Models [SOSP 2025]

Mirage: A Multi-Level Superoptimizer for Tensor Programs [OSDI 2025]

LLM in a Flash: Efficient Large Language Model Inference with Limited Memory [ACL 2024]

CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving [SIGCOMM 2024]

Understanding Stragglers in Large Model Training Using What-if Analysis [USENIX OSDI 2025]

DFlash: Block Diffusion for Flash Speculative Decoding

DSpark: Confidence-Scheduled Speculative Decoding with Semi=Autoregressive Generation

FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution