Hacker News

Vibecoding an inference chip

56 points by ankitvgupta ago | 18 comments

cpldcpu |next [-]

I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c

It seems implementing transformers is the new "hello world" of AI assisted coding.

datadrivenangel |next |previous [-]

Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.

risyachka |root |parent |next [-]

Fpga is software-driven, model can verify its output easily.

Zero surprise it did it

datadrivenangel |root |parent |previous [-]

it only cost them $10,000 in Claude API usage. :)

hmate9 |root |parent [-]

You get that much usage in a monthly $200 subscription.

dvh |root |parent |next [-]

So who's paying the cost?

stronglikedan |root |parent |next [-]

Everyone else that doesn't use the entirety of the plan they pay for.

nyc_data_geek |root |parent |next |previous [-]

Retail and society as ANT shares get pulled into index and wealth funds

ralph84 |root |parent |next |previous [-]

API pricing vs. individual plan pricing is just price discrimination. They're both positive gross margin.

surgical_fire |root |parent [-]

People believe in some wild stuff.

exe34 |root |parent |previous [-]

Corporate customers

dofm |root |parent |previous [-]

And a kitten.

codekansas |next |previous [-]

This is cool

Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

chalkycrimp |next |previous [-]

Good chance I'm not the only one thinking this: share the code

spwa4 |next |previous [-]

I must say I do see one application for this, that you can't really match reasonably: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.

(unfortunately for a decent model it will also be price competitive with a big nvidia card)

saejox |next |previous [-]

i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.

Archit3ch |previous [-]

Now do silicon. ;)

IshKebab |root |parent [-]