Fast Inference skills

GPTQ Quantization

davila7

Efficiently deploy large LLMs on consumer GPUs with minimal loss.

SGLang

davila7

Fast structured generation and serving for LLMs.

A

AWQ Quantization

davila7

Optimize large models with 4-bit quantization.

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention, Jacobi iteration, parallel token generation, and production deployment strategies.