About whether transformer LLMs can use much longer contexts without context rot.
Has this happened?
Yes 48 (44%)Not sure 39 (36%)No 21 (19%)
Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.
Hacker News has set AI a lot of challenges over the years. Which ones has it met?
About whether transformer LLMs can use much longer contexts without context rot.
Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.
But it's becoming increasingly clear that LLMs based on the transformer model will never be able to scale their context much further than the current frontier, due mainly to context rot. Taking advantage of greater context will require architectural breakthroughs.