Nemotron 3.5 Scores Higher, But The Open-Weight Gap Is Still There

🔍 Read the full analysis: Nemotron 3.5 Scores Higher, But The Open-Weight Gap Is Still There on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

NVIDIA released two open-weight reasoning models on October 15, and an independent evaluation by Artificial Analysis scored its flagship Nemotron 3.5 Super VL at 26 on the Intelligence Index, twice the 13 recorded for Nemotron 3 Super. The results mark a substantial improvement for NVIDIA’s models, while the highest-scoring open-weight models listed by Artificial Analysis still come primarily from China.

Super VL is a 120-billion-parameter mixture-of-experts model, with 12 billion parameters active for each token. It accepts text and images, has a reported one-million-token context window, and uses NVIDIA’s OpenMDW-1.1 license, which permits commercial use. Artificial Analysis lists its API pricing at $0.20 per million input tokens and $0.575 per million output tokens.

Artificial Analysis’s comparison with Nemotron 3 Super shows output token use per index task falling from about 114,000 to 49,000, while the estimated cost per task fell from $1.64 to $0.25. These are evaluation-specific measurements, not a guarantee of cost or performance for every workload. NVIDIA’s other release, Nemotron 3.5 Bolt 74B-A9B, scored 21. It has 74 billion total parameters, nine billion active parameters, and a 256,000-token context window. Artificial Analysis said Bolt had no public API or listed license on its model page at publication time.

Artificial Analysis also evaluated Super VL in NVIDIA’s NVFP4 format, a four-bit representation intended to reduce memory use. The evaluator reported a score within 0.1 points of full-precision BF16 weights, with no individual evaluation moving by more than 2.6 points. In a pre-release DeepInfra deployment, it measured about 630 output tokens per second on 100,000-token prompts and 1.8 seconds to first token. The model page’s median across six providers was 196 tokens per second, indicating that serving conditions affect measured speed.

NVIDIA’s Gains Meet a Higher Bar

The release offers developers a stronger US-made option in a market where model quality, deployment cost and licensing all affect adoption. Super VL’s reported $0.25 cost per index task is below the $0.60 reported for NVIDIA’s larger Nemotron 3 Ultra, which scored 23. That comparison suggests the newer model can score higher on this index while using fewer parameters and costing less on the evaluator’s task measure.

That progress does not amount to a change in the overall leaderboard. Artificial Analysis lists China’s MiMo-V2.6-Pro at 46 and several other Chinese models at 44 to 45; the UAE-developed K2 Horizon 375B A23B scored 31. Super VL placed 18th among listed open-weight models and 11th when the comparison was limited to permissively licensed models. The scores measure performance on a particular index, while license restrictions affect who can use a model commercially. Those are related but distinct considerations for organizations choosing what to deploy.

How the Models Compare

The October 15 release included two models with different intended footprints. NVIDIA describes Super VL as its flagship, while Bolt is the first model in a new family designed to run on less hardware. Both accept text and images and produce text. Their mixture-of-experts designs activate only part of the network for each token, though that design alone does not determine real-world hardware needs or total serving cost.

Bolt’s score of 21 is 1.5 points below Nemotron 3 Ultra’s 23, despite having roughly one-seventh as many total parameters. Artificial Analysis reported that Bolt outscored Ultra on AutomationBench-AA and AA-Briefcase, but trailed it on long-context reasoning and Humanity’s Last Exam. Bolt also produced about 82,000 output tokens per index task, compared with Ultra’s 28,000. Without a public API price, its cost per task cannot yet be compared on the same basis.

The same day, Meta released Muse Spark Open, which scored 39 and has 816 billion total parameters, with 42 billion active. Artificial Analysis listed its cost at $0.84 per index task. The source account says it uses the Muse Model License, which has commercial restrictions, while Super VL’s OpenMDW-1.1 license allows commercial use. License terms therefore shape what “open” means for different users.

Limits of the Current Rankings

The scores and cost figures cited here come from Artificial Analysis’s evaluations and model listings. The source material does not establish how well the rankings predict results across every task, prompt or deployment setup. Provider speed measurements vary, and the reported 630-token-per-second result came from a pre-release DeepInfra endpoint, while the listed provider median was lower.

Bolt’s public API availability, pricing and license listing were not available in the cited account, leaving its deployment cost and usage terms unresolved. The source material also does not provide further details on how the Muse Model License’s commercial restrictions apply to particular uses. Model rankings and access terms may change after publication.

Pricing and Deployment Details

For Super VL, the next practical comparisons will come from deployments across providers and from users measuring performance on their own workloads. The reported results give a release-day benchmark, but the model page’s provider median already shows why buyers should check the serving option they plan to use.

For Bolt, a public API, posted pricing and a clearly listed license would allow developers to assess its total cost and conditions of use. Further updates to Artificial Analysis’s model listings may also change the relative rankings as additional evaluations and providers become available.

Key Questions

What score did Nemotron 3.5 Super VL receive?

Artificial Analysis gave it a rounded Intelligence Index score of 26; the unrounded score was 26.5.

Did Super VL close the gap with the highest-scoring open-weight models?

No. It improved NVIDIA’s position, but Artificial Analysis listed several Chinese models above it and a UAE model at 31. Super VL scored 26.

How does Nemotron 3.5 Bolt differ from Super VL?

Bolt is smaller, with 74 billion total parameters and nine billion active, and scored 21. Super VL has 120 billion total parameters, 12 billion active and scored 26.

Can Super VL be used commercially?

The source account says Super VL uses OpenMDW-1.1, which allows commercial use. It reported that Bolt’s license was not yet listed by Artificial Analysis.

Is Bolt cheaper to run?

Its cost is not established in the cited material: Bolt had no public API price at publication time. Artificial Analysis reported about 82,000 output tokens per index task, but token use alone does not establish the final cost.

Source: ThorstenMeyerAI.com