Slightly less wrong

Predict what comes next. Compare it with what happens. Get slightly less wrong next time.

That’s the basic idea behind next-token training in language models.

I wonder how much of our own learning works that way too. We build a model of the world, make predictions, and adjust when experience surprises us. Provided we’re willing to update.