TL;DR
LLMs are becoming proficient at writing proof scripts, potentially transforming formal verification from expensive research niche to mainstream software engineering practice.
Key Points
- seL4 microkernel required 20 person-years and 200,000 lines of proof code for 8,700 lines of C—ratio of 23:1 proof-to-implementation
- LLM-based coding assistants now capable of automating proof script generation with human expert guidance; full automation extrapolated within years
- Proof checkers provide built-in validation: invalid proofs are automatically rejected, making hallucinations self-correcting and eliminating need for human code review
- Economics shift: formal verification becomes viable when cost of proofs drops below expected cost of bugs; AI-generated code amplifies need for provable correctness
Why It Matters
If formal verification becomes cheap enough through AI automation, it fundamentally changes software development economics and risk management. For security-critical systems (medical devices, aircraft, cryptographic protocols), this could eliminate entire classes of bugs. For AI-generated code specifically, formal proofs provide mathematical certainty where human review fails—shifting from probabilistic trust to deterministic verification.
Source: martin.kleppmann.com