Hacker News
Vibecoding an inference chip
56 points by ankitvgupta
ago
|
18 comments
cpldcpu
|next
[-]
I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c
It seems implementing transformers is the new "hello world" of AI assisted coding.
datadrivenangel
|next
|previous
[-]
Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.
risyachka
|root
|parent
|next
[-]
Fpga is software-driven, model can verify its output easily.
Zero surprise it did it
datadrivenangel
|root
|parent
|previous
[-]
it only cost them $10,000 in Claude API usage. :)
hmate9
|root
|parent
[-]
You get that much usage in a monthly $200 subscription.
dvh
|root
|parent
|next
[-]
So who's paying the cost?
stronglikedan
|root
|parent
|next
[-]
Everyone else that doesn't use the entirety of the plan they pay for.
nyc_data_geek
|root
|parent
|next
|previous
[-]
Retail and society as ANT shares get pulled into index and wealth funds
codekansas
|next
|previous
[-]
This is cool
Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...
spwa4
|next
|previous
[-]
I must say I do see one application for this, that you can't really match reasonably: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.
(unfortunately for a decent model it will also be price competitive with a big nvidia card)
saejox
|next
|previous
[-]
i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.