When the Machines Write Faster Than Anyone Can Check Their Work

In a converted office in London, two brothers who once wrote code for Palantir and Dropbox have built a company whose entire premise is that nobody, not even the engineers who write software for a living, can keep up with what artificial intelligence now produces. Meticulous, the company in question, has just raised fifteen million dollars to sell a simple but uncomfortable idea back to the technology industry: that AI now writes code faster than any human team can verify it is safe to ship. Customers who have already paid for such a guarantee include Notion, ElevenLabs, Wiz, and Dropbox itself.

The underlying problem Meticulous was built to solve is not confined to Silicon Valley, and the NHS is walking into a version of it with far less scrutiny than the venture capital world currently affords the issue.

The pattern Meticulous describes is familiar to anyone who has watched NHS digital procurement over the past three years. Systems are commissioned, built and increasingly AI-assisted at a pace that outstrips the institutional capacity to test what they actually do once deployed. The Federated Data Platform concentrates clinical and operational data architecture in fewer, larger systems, which means an untested edge case no longer produces a contained local failure but a systemic one. Ambient voice technology for clinical documentation is moving through MHRA approval and trust-level adoption on a timeline set largely by vendor ambition rather than by the maturity of independent verification processes built to catch what happens when an algorithm mishears a drug dosage or misattributes a symptom to the wrong patient record.

Because their own renewal rates cover the cost of an unconfirmed edge case, commercial software businesses have begun to pay for tools like Meticulous. A bug that reaches production erodes trust with paying customers, and that has a price the market enforces quickly. Clinical software does not face the same commercial discipline. When an NHS trust adopts an AI-assisted system and a rare failure mode goes undetected, the cost falls on a patient, not on the vendor's quarterly numbers. That asymmetry is the real story here, and it is one NHS leaders have been slow to name plainly.

Sir Jim Mackey's accountability agenda has rightly focused attention on financial grip and operational delivery across a reorganised landscape of twenty six integrated care boards. But accountability for digital risk has lagged behind accountability for spend. Trust boards signing off AI-assisted clinical software are rarely asking vendors the question that has become routine in venture-backed technology circles: what is your process for knowing, before a change ships, what it will do to every affected patient pathway, and can you demonstrate that deterministically rather than assert it.

MHRA sits at the centre of this gap. Its regulatory function was designed for a world where software changed in discrete, reviewable releases. That is not how AI-assisted development operates. Code changes continuously, often generated by systems that a human reviewer did not directly write and cannot fully audit line by line. A regulator built around periodic approval cycles is not well suited to governing a technology that behaves more like a living system than a fixed product, and the resourcing question behind MHRA's capacity to adapt has had far less political attention than data platform contracts or leadership reshuffles.

None of this means the digital reform agenda should slow down. The case for AI in clinical documentation, in diagnostics, in administrative efficiency remains strong, and the operational pressures driving NHS digitisation are real and unlikely to ease. But the credibility of that agenda rests on trusts and regulators being able to say, with some confidence, that they know what an AI-assisted system will do before it touches a patient, not after.

Meticulous exists because paying customers in commercial software decided that assumption was no longer safe to make on faith. The first significant AI-related clinical failure will make the case far stronger than any opinion piece can, and the NHS is yet to come to the same conclusion.