Hacker News
0.5B base model trained on 60B tokens rivals Qwen3.5-2B trained on 30T+ tokens
11 points by AZDR
ago
|
2 comments
cobychapple
[-]
it's 2.59B total with 553M active, so "0.5B vs 2B" more like 2.5B vs 2B? the card's own table has OLMo-2 1B (61.1) beating both.
AZDR
|root
|parent
[-]
Fair points. What really matters IMO is total training compute and FLOPs/token. On that front:
Nenya vs Qwen3.5 2B: Nenya has 71% fewer FLOPs/token and at least 99.8% less training compute (conservative assumption of only 10T tokens for Qwen, but the real number is likely 30T+).
Nenya vs OLMo-2 1B: 57% fewer FLOPs/token (2.3×) and 99.4% less training compute (155×).
Another way to look at it: at iso-compute (2e20 FLOPs), OLMo-1B (which is actually around 1.5B) scores 45.7 vs. Nenya at 58.3.
Technical reports for both of these analyses: https://www.arsalanzafar.ai/nenya-technical-report