Hacker News
Toon – Token Oriented Object Notation
19 points by royosherove
ago
|
8 comments
vessenes
|next
[-]
I’ll be interested to see benchmarks. My expectation is that accuracy will take a hit on mid or longer context prompts: I’d bet that the heavy use of JSON in fine tuning will end up impacting quality of a more terse (less reasoning space) novel encoding.
That said: I like the idea!
brian-bk
|root
|parent
[-]
There are a very light benchmarks in the Readme, or are you looking for more?
Mumps
|root
|parent
[-]
Do you mean the [0] Token Benchmarks section? I only see token count numbers.
Which doesn't address the question: do LLMs understand TOON the same as they would JSON? It's quite likely that this notation is not interpreted the same by most LLM, as they would JSON. So benchmarks on, say, data processing tasks, would be warranted.
[0] https://github.com/johannschopplich/toon?tab=readme-ov-file#...
tujux
|root
|parent
[-]
I think they're talking about these sections:
1. Retrieval Accuracy - https://github.com/johannschopplich/toon?tab=readme-ov-file#...
2. Performance by dataset - https://github.com/johannschopplich/toon?tab=readme-ov-file#...