Goalposts

Hacker News has set AI a lot of challenges over the years. Which ones has it met?

2025 March__MatrixMan__

About whether LLMs scanning commits could catch supply-chain attacks like the xz backdoor.

Maybe, but good luck getting an LLM (one which does not include analysis of this particular attack in its training data) to spot this attack with a prompt that doesn't also create thousands of false positives when focused on the millions of non-malicious commits out there. I think we're decades away from them being that good.

An LLM, without the xz attack in its training data, flags that backdoor without producing thousands of false positives on normal commits.

Has this happened?

Yes 37 (33%)Not sure 60 (54%)No 15 (13%)

Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.