Math isn’t enough.
AI systems can now ace the notoriously brutal Putnam Exam. Axiom, a seven-month-old outfit, nabbed 8/12. That’s better than top students. The closest AI competitor? A distant 3/12. This is another notch in AI’s belt, another Goliath to fall. But Axiom’s CEO, Carina Hong, sees this as just another stepping stone, not the summit. Code is good, sure. Anthropic’s gamble on code over cat pictures looks smart now. But code alone won’t get us to AGI. There are gaps, yawning chasms that even LLMs can’t bridge. And those gaps are in the very foundation of reasoning: formal proof.
The Bottleneck is Proof
“Verified AI.” Sounds dull, doesn’t it? Like eating your vegetables. But for Axiom, it’s the golden ticket. It’s about “scaling brilliance, compounding brilliance,” Hong declared. Took me a minute to parse that. It sounded like PR fluff until it clicked. She invoked Srinivasa Ramanujan, the math prodigy who just knew things. When G.H. Hardy forced Ramanujan to write down proofs, to articulate the how and why, he didn’t just convince others. He apparently expanded his own thinking. That’s compounding. Building on bedrock, not sand. Axioms. Formal proofs allow others to build, to scale those insights. This is the engine of progress.
The Math of AI Training
So, how does this translate to AI? Two ways: training and inference. Formal verification, in essence, is like a super-powered type checker. Not for code, but for mathematical proofs. Think of languages like Lean, where every step is meticulously specified. It’s a Herculean effort to translate an “informal” (read: human-readable, but still complex) proof into Lean. Axiom’s open-sourced AXLE toolkit helps with this messy business.
Now, imagine Reinforcement Learning. Instead of fuzzy statistical guesses (GRPO, RLHF, you know the drill), you get a hard, verifiable reward signal. Compile the code, check the proof. Much cleaner. Much more powerful.
The problem? LLMs aren’t great at spitting out Lean proofs. Yet.
But Axiom claims 99% (187/189) on the Verina benchmark. This benchmark requires generating code and its proof of correctness. OpenAI’s best? A measly 4.9%. These numbers are sparse, sure. We’re mostly guessing what the frontier labs are cooking up between math competitions. But Axiom suggests they’re not training LLMs to directly generate Lean proofs. They’re still relying on those messy, informal human proofs.
Scaling and Compounding: The Axiom Way
Hong’s Ramanujan analogy holds up. Better proofs mean better Lean generation, which means better RL. A stronger signal equals more efficient training, higher ceilings.
And scaling? Once a proof is verified in Lean, the output is as reliable as a human’s. Your training data just got a whole lot more trustworthy. This isn’t just about one AI getting smarter; it’s about building a foundation for all future AI. It’s a virtuous cycle, a compounding of knowledge. Can current frontier labs close this gap? Time will tell. But if Axiom is right, the future of AI isn’t just about brute force computation; it’s about rigorous, verifiable reasoning. That’s a bold claim. It’s also the most interesting one I’ve heard about AI in months.
A Different Path to Intelligence
While OpenAI and Google pour resources into ever-larger models, chasing scale with statistical patterns, Axiom is forging a different path. They’re betting on formalism. It’s a painstaking, less glamorous approach than simply scaling up parameters. It requires a deep understanding of logic and mathematics, and a commitment to rigor that many in the AI race seem to have abandoned in their haste for immediate results.
This emphasis on formal verification is, frankly, a refreshing antidote to the often-unsubstantiated claims of emergent capabilities in LLMs. We see models hallucinate, confidently assert falsehoods, and exhibit biases that reflect their messy training data. Axiom’s approach, by contrast, aims to build intelligence on a foundation of demonstrable truth. It’s like the difference between a grand orator who can sway crowds with eloquent but potentially misleading speeches and a diligent scientist who meticulously documents their findings, allowing for independent verification and replication.
Will this meticulous approach ultimately lead to more generalizable and reliable AI? It’s too early to say definitively. But it offers a compelling counterpoint to the prevailing paradigm. If LLMs are like extremely talented improvisational jazz musicians, Axiom is trying to build AI that can compose and rigorously prove symphonies. One is dazzling, the other potentially more enduring.
🧬 Related Insights
- Read more: OpenAI’s $6.5 Billion Windfall: AI’s Grim March to Monopoly Infrastructure
- Read more:
Frequently Asked Questions
What does Axiom Math actually do? Axiom Math is an AI company focused on using formal mathematical proofs to train more reliable and scalable AI systems, aiming to move beyond the limitations of informal reasoning found in traditional LLMs.
Will this formal proof approach make AI more trustworthy? The core idea is that AI trained with formal verification will be more trustworthy because its reasoning can be mathematically proven correct, reducing issues like hallucination and bias.
Is this approach better than just scaling up LLMs? Axiom argues that while scaling LLMs is important, formal proofs offer a more fundamental path to intelligence by ensuring correctness and enabling true compounding of knowledge, potentially surpassing the limitations of purely statistical models.