That's like saying coding benchmarks are about memorizing the language syntax. Y... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		esafak 17 days ago \| parent \| context \| favorite \| on: Step 3.5 Flash – Open-source foundation model, sup... That's like saying coding benchmarks are about memorizing the language syntax. You have to know what to call when and how. If you get the job done you win.

YetAnotherNick 17 days ago [–]

I am saying the opposite. If a coding benchmark just tests the syntax of a esoteric language, it shouldn't be called coding benchmark.

For a benchmark named terminal bench, I would assume it would require some terminal "interaction", not giving the code and command.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact