Goalposts

Hacker News has set AI a lot of challenges over the years. Which ones has it met?

2023 FebruaryRC_ITR

About whether GPT multiplying 4-digit numbers would show it learned arithmetic.

GPT saw a bunch of arithmetic and can repeat it. That’s the joke behind the Reddit usernames and /r/counting.

The four digit number thing is just the current lower bound of where it gets confused because a lack of training data.

Once you teach a patient 8 year old the rules of multiplication once, they can multiple any two numbers (that they’ve never seen before) with an arbitrary number of digits. An LLM cannot and will not ever be able to do that because it is a specific tool and it is not designed to do that (doing that would be a bad outcome for an LLM since we have different tools that can do multiplication much more efficiently).

So yes, if a LLM learns rules based math (which it is not intended to do) I’ll eat not only my, but every hat in existence.

An LLM multiplies any two unseen numbers with an arbitrary number of digits by following the rules of multiplication.

Has this happened?

Yes 52 (50%)Not sure 31 (30%)No 21 (20%)

Votes cast 1–2 October 2026: 100,590 votes from 9,694 people.