# LLM Academy — LLM Production Ops Guides

> Practical guides for engineers shipping large language models to production. Covers inference engines, gateways & cost, observability & evals, and inference optimization. Plus interactive deep dives on attention and KV cache techniques. No fluff, just what works in production.

## Start Here — Content Hubs

- [Home](https://llm-academy.dev/): All LLM Production Ops guides — inference, gateways, observability, optimization
- [Inference Engines Hub](https://llm-academy.dev/inference/): Production guide to vLLM, SGLang, TGI, and TensorRT-LLM
- [Gateways & Cost Hub](https://llm-academy.dev/gateway/): Route requests across providers, cache calls, and cut API spend
- [Observability & Evals Hub](https://llm-academy.dev/observability/): Trace every call, evaluate quality, and catch hallucinations
- [Inference Optimization Hub](https://llm-academy.dev/optimization/): Flash Attention, KV cache, prefix caching, speculative decoding

## Inference Engines

- [vLLM vs SGLang vs TGI: 2026 Benchmark](https://llm-academy.dev/inference/vllm-vs-sglang/): Throughput, latency, and VRAM comparison on H100
- [How to Deploy vLLM in Production](https://llm-academy.dev/inference/deploy-vllm/): Docker setup, tensor parallelism, GPU memory tuning, OpenAI-compatible API
- [SGLang RadixAttention Explained](https://llm-academy.dev/inference/sglang-radixattention/): The radix-tree prefix cache behind SGLang's throughput edge
- [vLLM PagedAttention Explained](https://llm-academy.dev/inference/vllm-pagedattention/): The paged KV cache inspired by OS virtual memory — 96% GPU utilization, 2-4x throughput
- [TensorRT-LLM vs vLLM](https://llm-academy.dev/inference/tensorrt-llm/): When NVIDIA's lowest-latency engine wins, and the deployment cost of vendor lock-in
- [LLM Inference GPU Benchmark: H100 vs A100 vs L40](https://llm-academy.dev/inference/gpu-benchmark-h100-a100-l40/): Cost-per-token, memory bandwidth, FP8 throughput, and which GPU wins for 7B/70B workloads

## Gateways & Cost Optimization

- [LiteLLM vs Portkey vs OpenRouter](https://llm-academy.dev/gateway/litellm-vs-portkey/): Routing, fallbacks, cost control, self-hosting, and pricing compared
- [LLM Cost Optimization Playbook](https://llm-academy.dev/gateway/cost-optimization/): The 5 levers that cut API spend 70-85%
- [How to Self-Host LiteLLM](https://llm-academy.dev/gateway/self-host-litellm/): Production Docker Compose setup with budgets and rate limits
- [Semantic Caching for LLM APIs](https://llm-academy.dev/gateway/semantic-caching/): GPTCache, Redis VL, embedding model choice, and the similarity threshold that intercepts 25-40% of redundant calls
- [LLM Model Routing](https://llm-academy.dev/gateway/model-routing/): Rule-based vs classifier-based routing that cuts cost 50-70% by matching prompt complexity to model
- [OpenAI vs Anthropic vs Gemini Pricing](https://llm-academy.dev/gateway/llm-pricing-comparison/): 2026 per-token cost, prompt caching discounts, batch API savings, cheapest provider by workload

## Observability & Evals

- [LangSmith vs Langfuse vs Phoenix](https://llm-academy.dev/observability/langsmith-vs-langfuse/): Tracing, evals, pricing, and open-source status compared
- [How to Self-Host Langfuse](https://llm-academy.dev/observability/self-host-langfuse/): Production Docker Compose setup for open-source LLM tracing
- [LLM-as-a-Judge Methodology](https://llm-academy.dev/observability/llm-as-judge/): Using a strong model to score outputs, biases, and human correlation
- [Tracing LLM Agents](https://llm-academy.dev/observability/agent-tracing/): Multi-span traces, tool calls, sub-agent spans, and where latency hides in agent pipelines
- [LLM Cost Monitoring Dashboards](https://llm-academy.dev/observability/cost-monitoring/): Per-team, per-model spend visibility and budget alerts with Langfuse, LiteLLM, and Grafana
- [Hallucination Detection in Production](https://llm-academy.dev/observability/hallucination-detection/): RAG grounding checks, self-consistency, LLM-as-a-judge verification, and guardrails (NeMo, Guardrails AI)

## Concept Foundations

- [What is RAG? Retrieval-Augmented Generation](https://llm-academy.dev/guides/rag-explained/): How RAG grounds LLM responses in retrieved documents, cutting hallucinations 40-70%. Production stack, vector DB choice, RAG vs fine-tuning.
- [LoRA Fine-Tuning in 2026](https://llm-academy.dev/guides/lora-fine-tuning/): Low-Rank Adaptation cuts fine-tuning memory 10-100x. Rank/alpha choice, QLoRA for 70B models, multi-LoRA serving in vLLM and SGLang.

## Inference Optimization

- [Flash Attention Explained](https://llm-academy.dev/optimization/flash-attention/): The tiled attention kernel that cut memory O(N²)→O(N) — FA1, FA2, FA3
- [Prefix Caching in vLLM & SGLang](https://llm-academy.dev/optimization/prefix-caching/): Block-level hashing vs radix tree prefix caching
- [Chunked Prefill Explained](https://llm-academy.dev/optimization/chunked-prefill/): How splitting prefill into blocks and interleaving with decode cuts Time-To-First-Token for long prompts

## Inference Optimization Deep Dives (Interactive)

- [Speculative Decoding](https://llm-academy.dev/speculative-decoding/): Draft-verify decoding for 2-3x decode throughput with zero quality loss
- [Multi-Head Latent Attention (MLA)](https://llm-academy.dev/mla-attention/): DeepSeek's KV cache compression — 93% memory cut vs standard MHA
- [Ring Attention](https://llm-academy.dev/ring-attention/): Scaling context to millions of tokens across GPUs via sequence parallelism
- [KV Cache Quantization (FP8/INT4)](https://llm-academy.dev/kv-cache-quant/): FP8 and INT4 KV cache quantization that slash memory 2-8x

## Calculators & Tools

- [KV Cache Calculator](https://llm-academy.dev/kv-cache-calculator/): Compute exact KV cache memory footprint for any model
- [Quantization Calculator](https://llm-academy.dev/quantization-calculator/): Compare model size and memory across FP16, BF16, INT8, and INT4

## Optional

- [About](https://llm-academy.dev/about/): About LLM Academy
- [Contact](https://llm-academy.dev/contact/): Get in touch