vLLM v0.34.0: Major Performance Boosts and New Model Support
vLLM v0.34.0 brings massive speedups for running large language models, alongside native support for 10 new models like Mistral Large 4 and Nemotron-3.5-Transcribe. This update turns on the extensible KV cache by default to give your GPUs a bigger memory cache and higher overall throughput.
Original image temporarily unavailable
Release Overview
vLLM v0.34.0 marks a significant milestone driven by 1,061 commits from 412 contributors. This release completely phases out Model Runner V1, making Model Runner V2 the exclusive GPU runner for maximum efficiency. It also introduces major speedups tailored for Blackwell and MI355X hardware.
Core Performance Improvements
Performance is a major focus in this version. Users can now experience up to 1.95× decode acceleration on GB200 systems, alongside a 1.6× throughput boost for models like GLM-5.3-Flash. Additionally, long-prompt Time-to-First-Token (TTFT) has been slashed, running 1.24 to 1.40 times faster.
New Model Integrations
The update adds seamless support for 10 new state-of-the-art models, covering text, speech, and vision modalities. Highlights include Mistral Large 4 (featuring FP8 and NVFP4 support), MiniMax-M3.1-Flash, NVIDIA Nemotron-3.5-Transcribe, and Liquid AI's LFM3 series.
Breaking Changes & Upgrades
Several important changes require attention before upgrading. The default block size is now 64 instead of 16, and huggingface_hub >= 2.0.0 is strictly required. Furthermore, the extensible KV cache now automatically sizes itself based on actually free memory after warmup, maximizing your hardware's potential.
View selection rationale
Importance 88/100
About this brief
- Content type
- News summary
- Source type
- First-party / official source
- Original source
- vLLM Releases ↗
- Source published
- Process
- AI-assisted discovery, translation, summarisation, classification and scoring; standards and corrections are maintained by the editorial desk.
- Verification
- The original URL and publication metadata are retained. This brief is not independently fact-checked unless explicitly stated.
- Published by
- Polyemergence Editorial Desk · PolyEmergence Pte Ltd.
- Conflict disclosure
- No financial or commercial conflict declared.