Learning to Reason: When LLMs Stopped Guessing Words and Entered the Test-Time Compute Era
In September 2024, OpenAI announced o1 with a short description: a model trained with reinforcement learning to reason, one that produces a long internal chain of thought before it answers. Read casually, it sounds like a normal product post. But there is one technical detail buried under the headline worth slowing down for: o1’s performance improves when you let it think longer, and it also improves when you train it longer. Two completely different compute axes, not one.
Read More