LLM Inference Systems Lab
Blog
Four measured studies on a single 12 GB GPU. Each article explains the question, why it matters, what the experiment found, and where the code and raw evidence live.
Does compression actually make LLM inference faster?
Saved bytes became speed only when the kernel and batch regime cooperated. We tested FP16, INT8, NF4, and 2:4 sparsity on real LLM decode shapes.
Lower power, higher energy: the quantization paradox
NF4 drew less board power but used more gross energy per token. Subtracting the GPU idle floor reversed the comparison, while batching produced the largest win.
How much KV cache can you prune before quality breaks?
A recent-window cache removed one-third of the context with almost no perplexity change, while bounded caches kept long-context memory and decode cost flat.
What breaks first when an LLM server is overloaded?
First-token latency failed before decode latency, and fixed bursts showed why average request rate is not enough for capacity planning.