Reading List

Being less concrete further out, the reading list is being incrementally updated to include more papers as we go.

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness [NIPS 2022]

Efficient Memory Management for Large Language Model Serving with PagedAttention [ACM SOSP 2023]

FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU [ICML 2023]

H2O: heavy-hitter oracle for efficient generative inference of large language models [NIPS 2023]

Accelerating Large Language Model Decoding with Speculative Sampling

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management [USENIX OSDI 2024]

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving [USENIX OSDI 2024]

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve [USENIX OSDI 2024]

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU [ACM SOSP 2024]

LLM in a Flash: Efficient Large Language Model Inference with Limited Memory [ACL 2024]

CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving [SIGCOMM 2024]

Understanding Stragglers in Large Model Training Using What-if Analysis [USENIX OSDI 2025]

DSpark: Confidence-Scheduled Speculative Decoding with Semi=Autoregressive Generation