About whether LLMs reason or just do statistical optimization, given their gullibility.
Has this happened?
Yes 71 (61%)Not sure 25 (22%)No 20 (17%)
Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.
Hacker News has set AI a lot of challenges over the years. Which ones has it met?
About whether LLMs reason or just do statistical optimization, given their gullibility.
Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.
Sure thing.
People really don't learn the history of AI any more apparently or this question wouldn't come up all the time.
There is basically any number of questions you can ask a two year old human who have never encountered that question nor anything even remotely similar to it and yet they can answer without fail. Meanwhile absolutely no AI can answer these unless the specific question / the rules underlying the questions were previously fed into it. The textbook example is "If Susan goes shopping will her head go with her?" Of course, since this specific question is literally a textbook one, you can't fool an LLM with it but it's easy to come up with brand new ones.
In the early 1980s this stopped Douglas Lenat who has worked very successfully on discovery systems and made him turn to assembling these facts and rules into CyC.