About whether LLMs learn new facts at inference time or lose them beyond the context window.
Has this happened?
Yes 12 (10%)Not sure 30 (24%)No 81 (66%)
Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.
Hacker News has set AI a lot of challenges over the years. Which ones has it met?
About whether LLMs learn new facts at inference time or lose them beyond the context window.
Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.
I'm curious if future developments in LLMs will enable them to extract significant/noteworthy info from their context window and incorporate it into their underlying understanding by adjusting their weights accordingly. This could be an important step towards achieving AGI, since it closely mirrors how humans learn imo.
Humans continually update their foundational understanding by assimilating vital information from their "context window" and dumping irrelevant noise. If LLMs could emulate this, it would be a huge win.
Overall, very exciting area of research!