Astro Hacker News - Nvidia Nemotron 3.5 Lightning

NitpickLawyer |next [-]

This links to the nvfp4 version of the model, so they only compare it to the bf16 in this model card. If you're looking for other similar model comparisons they are in the model card of the bf16 version here [1].

While it looks "behind" the qwen equivalent model on most benchmarks, a few personal notes:

- nemotron models feel to me a bit less benchmaxxed / "stubborn". That means that they generalise a bit better, or can be tasked to solve similar but not quite identical task types to the training data (something that's hard to do w/ qwen/ds models)

- nemotron series are also open training (w/ open training recipes and some training data public)

- nvda will have an incentive to continue this kind of releases, even if other parties slowly abandon the open release of models. Whatever other incentives 3rd party labs have (i.e. meta, goog w/ gemma, the chinese labs that IPOd, etc) nvda will always want to sell hardware so their incentive to keep pushing open models is evident and will likely continue "forever".

[1] - https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...

adrian_b |root |parent |next [-]

Yes, there are 5 Nemotron-3.5 models:

https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...

to be used for further training/fine-tuning.

https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...

main model.

https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...

quantized version of the previous.

https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...

this "DFlash" model should be used together with one of the previous two "for lower-latency speculative decoding deployments tuned for low-concurrency data center and workstation workflows".

https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...

like DFlash, the previous model above, but optimized for DGX Spark.

beklein |root |parent |next |previous [-]

Thanks for pointing that out!

I actually copied the link from NVIDIA's Technical Blog post:

- https://developer.nvidia.com/blog/nvidia-nemotron-3-5-lightn...

You can also try the model via a free API endpoint from Openrouter, would be interesting to see if it's the BF16 or NVFP4 version:

- https://openrouter.ai/nvidia/nemotron-3.5-lightning:free

ggcr |root |parent |previous [-]

> nvda will have an incentive to continue this kind of releases, even if other parties slowly abandon the open release of models

This! It's literally in their best interest for open-weights models to succeed

happycube |root |parent [-]

Commoditize your compliments.

simonw |next |previous [-]

Runs fast on my Mac - ~100 tokens/second - but it's a bit of an over-thinker. It sketched out four different SVGs in the reasoning trace before returning a (bad) pelican riding a bicycle: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

I used LM Studio and this model file: https://lmstudio.ai/models/nvidia/nemotron-3.5-lightning

I got a pretty solid (and reasonably fast) result for this simply coding agent activity (with simonw/llm-coding-agent):

  llm code -m lmstudio/nvidia/nemotron-3.5-lightning \
    'how does auth work?'

https://gist.github.com/simonw/a8741f79280cedc86bcb8d90edfb4... - took just under a minute.

dofm |root |parent |next [-]

I tried that exact model, I get about 50 tokens/sec on an 64GB M1 Max MBP.

It writes pretty good code; it does seem to second-guess itself in thinking traces and I wonder if it just needs a reasoning budget and message.

Each time I test a model quickly in LM Studio, I ask it:

- to write a tiny little wordpress "last login" tracker plugin, asking me clarifying questions first. I ask it to use old PHP, break the WP coding guidelines to use inline anonymous functions in the hooks, avoid custom SQL, I see if it can write something useful in a singleton class, and what questions it thinks to ask. Qwen 3.6 does this very well, Nemotron has done OK, though it's a little less effective at reading between the lines, maybe.

- to offer an answer for a SQL puzzle about finding max score per category on old MySQL (5.0) without using subquery/derived tables -- it did a good job, picked up the nuances in the prompt that allow a particular solution, didn't go on a tangent about how it would be nice to have window functions or use subqueries, did a tool call to check like I asked. With this puzzle, if the model doesn't offer up an index for performance optimisation, I nudge it; this time it didn't volunteer one but when prompted about performance it offered an index and a bunch of other nice solutions, and only there did it round up the options for subqueries and derived tables, which is fair game.

(It did badly fail the car wash test, though, even on repeated nudging, where it gets more and more insane, doubling down and never getting the point, whereas Muse Glimmer solved it and well, with a thinking trace that didn't particularly suggest it had been post-trained)

I need to test it in Pi or opencode. I've been trying to motivate my brain to move to pi, but this model supports a longer context window so maybe opencode's overlong system prompt is less of an issue.

Of the 30B models in the last 24 hours (!) I think I prefer working with Muse Glimmer, which is slow but very good, and writes rather well with just a hint of being a bit of a cheeky monkey. Not tested that in an agentic coding setting yet.

Interesting times.

EchoVoicy |root |parent |previous [-]

>~100 tokens/second That sounds great to me!

>Runs fast on my Mac What specifications (CPU/RAM)?

simonw |root |parent [-]

M5 Max MacBook Pro, 128GB of RAM.

kristianp |root |parent [-]

Does it get loud and hot with a smaller MOE model like this?

bearjaws |next |previous [-]

Crazy to see how well Qwen3.6 35b-a3b is holding up, sure it is ~20% larger but it's scores are also ~20% higher with the same number of active params (excluding the IFBench).

Hopefully Qwen follows up their 3.8 launch with a new 35b-a3b

WalterGR |root |parent [-]

What’s expected in an updated 3.6 release?

hadlock |root |parent [-]

4 months of progress. Qwen 3.8 Max is, depending on who you ask, what you're doing, standard disclaimers, really really close to Opus 4.8. That is a huge dump from 3.6 Max. Presumably improvements in their training librar(y/ies), training improvements, etc will "trickle down" to Qwen 3.8. We are using 3.6 35B internally and it is in most cases better than Sonnet 4.6 which already makes it "good enough" for most data enrichment tasks. I would be a lot more excited about a MoE Qwen 3.8 35B-A3B than the "announced" 27B dense model. I have 50 million records I need to keep up to date, an A3B MoE is waaaay more performant than a 27B dense model.

gaodean |next |previous [-]

Developing on the Mamba 2 architecture is a really interesting point to note. It seems to be catching up to “regular” transformer architectures.

kamranjon |next |previous [-]

You might look at this and and be a bit disappointed by the performance against qwen and gemma models - but this is an entirely open source training pipeline, this is quite impressive and I don't think another model this performant exists with fully open source data and recipes alongside the weights.

hadlock |root |parent |next [-]

>I don't think another model this performant exists with fully open source data and recipes alongside the weights

If you're already selling shovels, you might as well provide maps to the gold mine with every purchase.

lysace |root |parent |previous [-]

Nemotron 3.5 Lightning Training Recipe:

https://github.com/NVIDIA-NeMo/Nemotron/blob/main/docs/nemot...

thisisauserid |next |previous [-]

I know very little about this but why isn't it mamba-3? Isn't it... uh, more gooder?

bcatanzaro |root |parent [-]

The Nemotron 3 family uses Mamba-2 because that is what we had validated when we started building the family.

Changing the SSM backbone is a major revision, not a dot release. At least for us.

judem |next |previous [-]

Interesting to see in the Agentic Coding Benchmarks that the codex harness is so demonstrably lower than any of the other harnesses, even Claude Code another lab harness. Does anyone know why that would be?

e2e4 |root |parent |next [-]

Could you please provide the link.

killix |root |parent |previous [-]

[flagged]

dofm |next |previous [-]

Car wash test was amusing with the LM Studio version of the model simonw mentions below.

Bad, bad failure and no amount of nudging helps it see the point. Comical.

But it did OK on other mini tests.

piyh |next |previous [-]

That SWEBench drop going from bf16 to fp4 is massive

ggcr |next |previous [-]

Nice cadence of releases by the Nemotron team :)

lostmsu |next |previous [-]

> nvfp4

Still trying to lock in, huh?

239127 |next |previous [-]

[flagged]

varispeed |previous [-]

Nvidia just throwing something "for peasants" to stay relevant. Where is competition spirit? More importantly why Nvidia is gatekeeping computing for everyday people?

I find these releases are bad taste.

Make 1TB DGX priced affordably, not some crap model for people to waste time on.

tasty_freeze |root |parent |next [-]

First convince the DRAM suppliers to drop their prices for you and then maybe nvidia will drop the DGX price for you too.

varispeed |root |parent [-]

nVidia could make DRAM themselves, but I heard it is much more complex than making a GPU...

nicman23 |root |parent |next |previous [-]

they are selling the shovels not the gold

timmmmmmay |root |parent |next |previous [-]

you wouldn't be happy then either

jazzyjackson |root |parent |previous [-]

imo a 512GB DGX cluster for $20k is not a bad price for an enterprise that wants to do on prem LLMs. Gatekeeping is a weird way to put “running a business”