The Contractor Who Never Signs the Permit Log: Why Enterprise AI's Real Risk Is Governance, Not Capability

Any facilities manager will tell you the same thing: the risk is never the apprentice who works slowly. It is the contractor who clears three jobs overnight, invoices before sunrise and never signs the permit log — flawless work, right up until the afternoon somebody needs to reconstruct what he touched. Enterprise computing has hired that contractor several million times over. The permits are still blank.

Five developments describe one condition from five angles: California's SB 53 and Texas's TRAIGA reaching effect on 1 January 2026, the European Commission's push to defer portions of the AI Act's high-risk regime, two hyperscaler outages inside nine days in October 2025, METR's randomised trial finding experienced developers slowed by AI assistance, and Stanford's AI Index recording enterprise adoption at 78%. Capability has been procured faster than accountability has been instrumented, and that gap is now a balance-sheet item rather than a research question.

Knight Capital Already Ran This Experiment

On 1 August 2012, Knight Capital deployed new order-routing code to seven of eight servers. The eighth retained a dormant function called Power Peg. In 45 minutes the firm accumulated roughly $7 billion in unintended positions and lost more than $460 million; the SEC's October 2013 order imposed a $12 million penalty for breaching the Market Access Rule. Note what the regulator did not find. The algorithm was neither malicious nor exotic. The failure was a deployment process without a second reviewer and without a rehearsed reversal. Agentic systems reintroduce that precise failure class with weaker determinism and materially worse observability, because the executable now includes a natural-language instruction that can be rewritten by a document the system merely reads.

Liability Is Migrating Downstream, Quietly

Frontier-transparency statutes such as California's SB 53 bind developers above compute thresholds. They do not bind the twelve-person logistics firm whose dispatch agent misroutes a pallet. That firm is a deployer, and deployer duties — human oversight, input-data relevance, log retention, notifying affected workers — sit in Article 26 of the EU AI Act and in the discriminatory-outcome language of state law. The operative enforcement mechanism will not be a regulator's dawn raid. It will be a procurement questionnaire, a renewal negotiation and an underwriter's exclusion clause. Firms that treat provenance as the vendor's problem discover otherwise at deposition.

The Case Against Paper Assurance

The governance build-out deserves scepticism rather than deference. Sarbanes-Oxley Section 404 produced two decades of management attestations without eliminating accounting fraud, and ISO/IEC 42001 certificates can be scoped so narrowly that the certified unit touches none of the workflows that carry risk. A model card is documentation, not a control test. Where compliance spend displaces evaluation engineering, the organisation has purchased moral hazard transfer rather than risk reduction. On that reading, Brussels' deferral is defensible sequencing — notified bodies were not being accredited at the pace the original timetable assumed — rather than capture.

Concentration Risk Is Now an Operating Expense

The AWS us-east-1 disruption of 20 October 2025 and the Azure Front Door failure nine days later were not AI incidents. They became AI incidents because inference endpoints, identity providers, vector stores and orchestration queues share fault domains. A multi-model strategy is not a multi-region strategy. Meanwhile the Stanford AI Index documents inference costs for a fixed capability falling by more than two orders of magnitude, which raises call volume and therefore blast radius. Cheap cognition is abundant. Cheap redundancy is not, and redundancy is what the auditor asks for.

The Productivity Number Nobody Audits

METR's July 2025 randomised controlled trial found experienced open-source developers were 19% slower on real repository tasks when permitted AI tools — while estimating afterwards that they had been 20% faster. A near-forty-point perception gap is not a rounding error; it is the mechanism by which unmeasured expenditure survives budget review. MIT's circulated 2025 finding that 95% of generative pilots produced no measurable P&L effect rests on contested sampling and should be read as directional, not decisive. The direction nonetheless matches the trial data. Adoption at 78%, against 55% a year earlier, measures procurement. It does not measure throughput.

Self-reported speed is a sentiment metric. Cycle time and defect escape rate are engineering metrics. Most AI business cases are still built on the first.

Sovereignty Is an Availability Hedge, Not a Flag

Fragmentation carries a price, and it is regressive. A regional insurer cannot fund three jurisdictions' conformity assessments, and delay is oxygen for firms that would otherwise exit the market outright. Open-weight models on owned hardware genuinely reduce exfiltration surface, remove unilateral deprecation risk and restore leverage at renewal. That is an availability hedge, not autarky. The caveat is portability: sovereignty without an evaluation harness merely relocates the dependency from a vendor's API to a weight file nobody on staff can benchmark.

Before the Next Renewal

• Inventory by telemetry, not survey. Reconcile SSO and egress logs against the approved-tool register. Shadow adoption is the unpriced liability.
• Renegotiate four clauses: incident-notification SLA, model-change notice, indemnity allocation, and data-export format on exit.
• Rehearse the reversal. Scope agent credentials as privileged identity, short-lived and revocable, then run a kill-switch drill quarterly.
• Baseline before you scale. Holdout tasks, cycle time, defect escape rate. The METR gap is only visible to firms that measured first.
• Test failover, don't diagram it. Portable embeddings and a second provider that has actually served production traffic.
• Treat retrieved content as untrusted. The UK NCSC position stands: prompt injection has no equivalent of parameterised queries.

Six Months Out: The Underwriters Move First

Forecast, not reporting: by mid-2027 the binding constraint on enterprise agents will be insurance rather than statute. Expect AI-specific endorsements with warranty conditions on logging and human sign-off, and expect exclusions where the deployer cannot produce an action trace. Procurement questionnaires will converge on a common control set faster than regulators converge on a common definition. Point-tool vendors without evaluation infrastructure will be absorbed. At least one mid-market failure will be attributed publicly to an unmonitored agent action, and the post-mortem will read like Knight Capital's: not a model that reasoned badly, but a change process nobody owned. The firms that profit will be the unglamorous ones that spent this quarter building the permit log.