vLLM Decision 3.0: Five Open Decision Models, 0.8B to 27B

What changed

The vLLM Semantic Router team released Decision 3.0, a family of five multimodal decision models under Apache-2.0 licensing. The models span from 0.8B to 27B parameters and address three core tasks: multiple-choice classification, binary yes/no questions, and numerical scoring on defined scales. Unlike traditional language models that generate text token-by-token, these models return probability scores for each possible answer in a single forward pass.

The five model sizes are d3-lite (0.85B), d3-nano (2.21B), d3-mini (4.54B), d3-flash (8.39B), and d3 (26.09B). Each includes a built-in vision encoder, allowing the models to process text, JSON, images, or combinations thereof. Images support multiple formats including PNG, JPEG, and WebP at resolutions up to 1.6 megapixels.

The models are fine-tuned versions of Qwen base models. The largest d3 builds on Qwen3.8-27B, while the smaller four derive from Qwen3.5 checkpoints at corresponding parameter counts. Vision encoders range from 0.10B in d3-lite to 0.46B in the larger variants.

Why it matters

Decision models address a practical production pattern in AI systems. Rather than prompting a large language model and parsing its text output for routing decisions, decision models return calibrated probability scores that developers can threshold directly. This approach fits guardrails, request classification, and routing without text generation overhead.

The release positions open-source alternatives against proprietary decision APIs, enabling developers to deploy models on-premises or self-hosted infrastructure. The performance advantage affects system design choices: the smallest model processes text in under 24 milliseconds on a single GPU, while the 27B variant requires around 84 milliseconds. Image processing adds latency proportional to model size, ranging from approximately 40 to 342 milliseconds for a single image.

Efficiency gains between generations impact practical deployment decisions. The 9B d3-flash model now outperforms Decision 2.0’s 27B model on internal benchmarks, achieving 59.0 versus 55.9. This allows teams to reduce model size without sacrificing accuracy, lowering compute costs or enabling deployment on resource-constrained hardware.

However, performance advantage is not consistent across all task types and model sizes. The smallest models do not lead every benchmark category they compete in, indicating that different models may suit different workloads and that practitioners cannot assume smaller always means worse.

What to test

Before adopting Decision 3.0, engineers should validate several claims and design tradeoffs:

Benchmark generalization. The team reports internal evaluation scores on the Jev Decision Index. Test whether these numbers translate to your actual use cases. A relevant caveat: Torchcast Decision 27B reportedly scores 65.1 on a public suite, marginally higher than Decision 3.0’s d3 at 64.9 on the same public benchmark. Clarify whether the internal versus public suite difference affects your requirements and how the models rank on benchmarks directly relevant to your domain.

Vision task coverage. The models perform well on some vision benchmarks (97.3 on InfographicVQA, 89.9 on Mind2Web) but underperform on others (46.2 on MMMU-Pro vision, 44.6 on Hateful Memes moderation). Identify whether your image classification tasks align with the tested domains and consider testing on your own visual data.

Hardware and inference speed. Latency measurements come from a single AMD Instinct MI325X GPU. Test performance on your target hardware, including other GPU types and CPU inference if applicable. Batch inference or concurrent requests may change latency characteristics significantly from single-request measurements.

Dependencies and setup. Models require transformers version 5.17.0 and must load with trust_remote_code=True. The optional flash-linear-attention package speeds up linear-attention layers. Verify these dependencies integrate cleanly with your infrastructure and that the security implications of trust_remote_code align with your risk tolerance.

Model output format and calibration. Confirm that probability scores require the thresholding and decision logic your system expects. Test whether the models’ confidence estimates correlate with actual accuracy on your data, since poorly calibrated scores can mislead downstream decision logic.

Class coverage and edge cases. The 0.8B d3-lite leads on text tasks but ranks third on vision, and the 2B d3-nano leads vision but trails on text. Validate which model variants fit your specific task profile before committing to a single size.

The conclusion

Decision 3.0 provides a deployable alternative to proprietary decision APIs. The range of model sizes offers flexibility to optimize for latency, accuracy, or resource constraints. The multimodal support addresses real use cases where decisions depend on both text and visual input.

The efficiency gains since Decision 2.0 are measurable, though unevenly distributed across all task types. Smaller models still underperform on certain vision tasks, so practitioners cannot automatically select the smallest option and expect top performance across all workloads.

The open license and Apache-2.0 terms eliminate vendor lock-in concerns, meaningful for production systems. The architecture supports local inference through standard interfaces, though the team notes planned native vLLM Semantic Router integration and Open Decision API support for future releases.

The models represent a concrete tool for practitioners building routing and guardrail systems, with the caveat that real-world validation remains necessary. Watch for community adoption patterns and third-party benchmarking on production routing tasks beyond the published evaluation sets. Success will depend on whether the project becomes a standard decision layer in mixture-of-models architectures.

Related reading

• Nace AI Open-Sources Drex 1.5 Decision Model for Option Scoring
• Liquid AI Releases Multimodal Open D1 Decision Models for Edge
• Microsoft releases Decision-1 routing and classification model