

Fine, a chess engine that is capable of running with affordable even for the time 1970s electronics will best what marketing folks would have you think is an arbitrarily capable “reasoning” model running on top of the line 2025 hardware.
You can split hairs about “well actually, the 2600 is hardware and a chess engine is the software” but everyone gets the point.
As to assertions that no one should expect an LLM to be a chess engine, well tell that to the industry that is asserting the LLMs are now “reasoning” and provides a basis to replace most of the labor pool. We need stories like this to calibrate expectations in a way common people can understand…
I’d say that those details that vary tend not to vary within a language and ecosystem, so a fairly dumb correlative relationship is enough to generally be fine. There’s no way to use logic to infer that it’s obvious that in language X you need to do mylist.join(string) but in language Y you need to do string.join(mylist), but it’s super easy to recognize tokens that suggest those things and a correlation to the vocabulary that matches the context.
Rinse and repeat for things like do I need to specify type and what is the vocabulary for the best type for a numeric value, This variable that makes sense is missing a declaration, does this look to actually be a new distinct variable or just a typo of one that was declared.
But again, I’m thinking mostly in what kind of sort of can work, my experience personally is that it’s wrong so often as to be annoying and get in the way of more traditional completion behaviors that play it safe, though with less help particularly for languages like python or javascript.