VLLM 0.32.0 Release Adds New Features and Optimizations

What happened

The vLLM 0.32.0 release introduces a new endpoint, POST /v1/systemone, designed to answer questions about structured state, including choice, noul, and score. This endpoint is available by default and runs on platforms such as Cloudflare/clef-omni, LiquidAI/d1-3B, and DiffusionGemma.

Additional updates include the hf tool and reasoning parser, which read the checkpoint's response_template and eliminate the need for model-specific parsers for compatible models. These tools are now available in both Python and Rust frontends.

Why it matters

The new features and optimizations in vLLM 0.32.0 aim to improve efficiency and throughput across various workloads. Enhanced support for low-precision KV caches and specific GPU architectures contributes to better performance for models like GLM-5.x and GLM-5.3-Flash.

Key details

The release includes 814 commits from 363 contributors, with 121 being new contributors. Support for NVFP4 KV cache is now available on SM8x, SM12x, and SM90 GPUs, with XQA decode enabled on SM12x. MiniMax-M3 sparse attention is now supported on SM100 GPUs with NVFP4 KV cache.

GLM-5.x models default to FP8 KV cache on SM100, reducing memory per rank by half and increasing throughput by 2.3-5.5% at the cost of a 7-11% TTFT increase. The fp8_ds_mla feature on SM90 boosts output throughput of GLM-5.3-Flash by up to 31%. ROCm now supports a 4-bit ultraquant_4bit KV cache with twice the density compared to previous versions.