VLLM 0.33.0 Release Adds New Models and Tools

What happened

The vLLM project released version 0.33.0, which includes 927 commits from 389 contributors, 122 of whom are new. The release features several new models, such as Reflection Beam-1.0, Yandex AliceAI-Foundation-80B-A3B, and Recursive Flint, along with tools like the ColUltraVBERT visual document retriever and Domyn Edge.

The update also upgrades PyTorch to version 2.14.0, torchvision to 0.29.0, and Triton to 3.8.0. Additionally, CUDA 12.9 is no longer supported, and all release wheels and images are now based on CUDA 13.0.

Why it matters

The inclusion of new models and tools expands the capabilities of vLLM for developers and researchers working on large-scale language processing tasks. The transition to CUDA 13.0 and the removal of support for CUDA 12.9 reflect ongoing efforts to align with the latest hardware and software advancements.

Key details

Among the new models are Reflection Beam-1.0, a 501B MoE model with 23B active parameters; Yandex AliceAI-Foundation-80B-A3B, a 80B parameter model; and Recursive Flint, a 40B MoE model with 2.5B active parameters. Other models added include Muse Spark Open, Step-5 Preview, NVIDIA Alpamayo 2 Super, and Nemotron-Labs Dory1.0-4B.

The release also introduces an AWS EFA image, named vllm/vllm-openai:v0.33.0-efa, for use with EFA-enabled EC2 instances. All release images are now published under the shorter name vllm/vllm, such as vllm/vllm:v0.33.0.