About whether unpredictable LLM errors make reviewing their output a bottleneck for knowledge work.
Has this happened?
Yes 26 (23%)Not sure 33 (29%)No 53 (47%)
Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.
Hacker News has set AI a lot of challenges over the years. Which ones has it met?
About whether unpredictable LLM errors make reviewing their output a bottleneck for knowledge work.
Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.
The first is that the LLM outputs are not consistently good or bad - the LLM can put out 9 good MRs before the 10th one has some critical bug or architecture mistake. This means you need to be hypervigilant of everything the LLM produces
This, to me, is the critical and fatal flaw that prevents me from using or even being excited about LLMs: That they can be randomly, nondeterministically and confidently wrong, and there is no way to know without manually reviewing every output.
Traditional computer systems whose outputs relied on probability solved this by including a confidence value next to any output. Do any LLMs do this? If not, why can't they? If they could, then the user would just need to pick a threshold that suits their peace of mind and review any outputs that came back below that threshold.