d1-ultra decision model scores 65 on AA index

What changed

Liquid AI released d1-ultra, a multimodal decision model built for classification, routing, and probability-based decisions on complex real-world tasks. The company designed the model to handle harder cases like identifying defects on circuit boards or validating damage claims against customer photos.

The model accepts text, images, or both as input and returns probabilities for each possible answer in a single forward pass without generating output tokens. This architecture means developers pay only for input tokens, not generated text.

Liquid reports that d1-ultra scores 65 on the Artificial Analysis Decision Index and ranks first among all decision models on that benchmark. The index combines results from four evaluations: AA-Omniscience-Decision, HLE-MultiChoice, When2Call, and WildGuardTest, each weighted at 25 percent.

The model is available now on the Liquid API at $0.042 per million input tokens. Liquid indicates that Vercel AI Gateway and OpenRouter support are coming soon.

Why it matters

Decision model benchmarks matter because they measure a specific task: answering bounded questions with calibrated probabilities instead of free-form text. This use case is growing as teams replace large language models with cheaper, faster alternatives for routing, moderation, guardrails, and triage.

Liquid’s reported 65 score on the Artificial Analysis Decision Index positions d1-ultra ahead of other options in a relatively new category. The company emphasizes calibration alongside accuracy, noting that most decision models are overconfident and that properly calibrated probabilities are essential for setting routing thresholds.

For practitioners, the appeal lies in cost and speed. With zero output tokens, a decision model call has different scaling characteristics than a language model call. Processing 1,000 images to classify them costs only input tokens, not the token cost of generated explanations. This model is useful for high-volume classification in manufacturing, insurance claims, content moderation, or document triage.

The multimodal capability represents progress. Earlier decision models worked on text alone. Text-plus-image support means a single API call can now handle visual inspection tasks, damage assessment, and other cases where understanding images is necessary.

What to test

Before deploying d1-ultra in production, teams should verify several vendor claims:

• Calibration in practice: Liquid emphasizes that its probabilities are well-calibrated. Test this by setting a routing threshold (for example, “send to human review if confidence is below 0.85”) and measure how often the model is incorrect at that threshold. Poor calibration causes downstream problems.

• Multimodal accuracy on your data: The model ranks first on the Artificial Analysis Decision Index, but that index measures performance on a curated evaluation suite. Test d1-ultra on images from your domain: circuit boards from your factory, claim photos from your customer base, documents in your workflow. Generic benchmarks often do not predict real-world performance.

• Latency and throughput: The announcement does not disclose parameter count, context length, or model size. Request performance data showing latency per request, throughput per second, and tail latency at your expected query volume.

• Image token counting: Liquid’s billing model counts images at 1.5 tokens per 32×32-pixel patch. A 1024×1024 image costs 1,536 tokens. Verify that your image sizes and batch sizes fall within expected cost ranges before scaling up.

• API-only availability: d1-ultra runs on Liquid’s hosted API only. No downloadable weights or quantized variants exist. If you need on-device deployment, local inference, or private hosting, this model cannot meet that requirement.

• Request limits: The API documentation caps requests at 8 images and 10,000 image patches per request, with a full JSON body size limit of 4.5 MB. Each question is billed separately, including all images in that question.

Common questions

How does d1-ultra compare to Microsoft-Decision-1?

The source does not provide a direct comparison between d1-ultra and Microsoft-Decision-1. The Artificial Analysis Decision Index lists them on different versions of the benchmark, making a straightforward head-to-head comparison difficult from the available data.

What are the input limits for d1-ultra?

The API documentation caps requests at 8 images and 10,000 image patches per request, with a full JSON body size limit of 4.5 MB. Billing counts each question separately, including all images in that question.

Does d1-ultra support open-weight deployment?

No. d1-ultra is available through the Liquid API only. Liquid built it on a larger open-weight backbone but has not announced downloadable weights or self-hosted variants for d1-ultra itself.

The conclusion

d1-ultra represents progress for decision models as a product category. The benchmark score is credible within its scope: Liquid tested across 112 evaluations and reports wins across all tasks. The emphasis on calibration is notable because probability scores matter as much as accuracy in production systems.

The real question is whether those benchmark gains translate to your specific use case. Generic decision-model benchmarks measure performance on curated, balanced datasets. Real-world images, blurry photos, unusual angles, lighting changes, and domain-specific nuances often behave differently. Test the model on your data before committing to it.

Pricing at $0.042 per million input tokens is competitive. The zero-output-token model makes economic sense for high-volume classification. Deployment is API-only, which means no local inference, no fine-tuning, and ongoing dependency on Liquid’s infrastructure and uptime.

Watch for three things in the coming months: deeper per-benchmark breakdowns showing where d1-ultra wins and loses, real-world case studies demonstrating how calibration holds up in production, and availability through Vercel and OpenRouter to reduce friction for new users. If you are already familiar with decision model options and tradeoffs, d1-ultra is worth testing against your current choice.

Related reading

• Liquid AI Releases Multimodal Open D1 Decision Models for Edge
• Nemotron 3.5 Super VL 120B: Open Multimodal Agent Model
• Timestral time-series forecasting model ranks first on three benchmarks