In 1923, the U.S. Court of Appeals for the D.C. Circuit declined to admit the results of a systolic blood-pressure deception test in Frye v. United States, 293 F. 1013. The instrument was not broken. Its output was merely disconnected from the proposition it was offered to prove: a fluctuating blood pressure reading was advanced as evidence of a mental state. The court demanded that the intervening scientific principle be generally accepted before the output could carry evidentiary weight.
Computer vision is approaching that same boundary. The field has largely stopped asking whether a network can label a pixel and started asking whether it can represent the physical world well enough to act inside it. Foundation models now produce amodal segmentation, metric depth, and occupied-space predictions from a single forward pass — capabilities that resemble understanding closely enough to be procured as though they were understanding. They are not the same thing, and the gap between them is where the next round of commercial failures will occur.
The Arithmetic of Almost Right
Consider a character recognition system with 99% per-character accuracy. Present it with a hundred-character invoice line and the probability that the entire line is correct falls to 0.99 raised to the hundredth power: approximately 36.6%. The component metric is excellent. The system metric is unusable. That arithmetic, not any architectural deficiency, explains most industrial vision deployments that work in demonstration and fail in production.
This epistemic gap survives the transition to foundation models. A segmenter that delineates 99% of visible objects correctly can still fail a bin-picking cell, because the failure mode is not pixel accuracy but relational reasoning: which object is load-bearing, which is occluded, which will shift when its neighbor moves. Mean intersection-over-union, the metric that dominates segmentation leaderboards, measures none of those things.
The consequence for computer vision engineering is an inversion of the familiar scaling thesis. The last fraction of operational reliability now costs more than the first ninety-nine. Marginal gains from a larger checkpoint are decaying while marginal gains from a curated edge-case corpus are compounding. The durable moat is shifting from model architecture to closed-loop operational data — and that is a much less transferable asset than a weight file.
The Annotation Labor Did Not Vanish. It Moved.
The prevailing narrative holds that foundation models eliminated the annotation bottleneck. The primary evidence complicates that claim.
Primary source. Meta's Segment Anything paper (Kirillov et al., April 2023) specifies SA-1B: 11 million images and 1 billion masks, generated through a six-stage data engine alternating manual, model-assisted semi-automatic, and fully automatic annotation.
The human labor was not removed. It was relocated — from drawing masks to designing the engine, auditing its outputs, and defining the ontology of what constitutes an object. A prompt-based interface does not eliminate judgment; it relocates judgment to the person holding the prompt.
The counter-argument matters here. It would be equally wrong to conclude that foundation models changed nothing. DINOv2 (Oquab et al., April 2023) trained on LVD-142M, a curated corpus of 142 million images, and yielded transferable features that required no task-specific fine-tuning across a range of dense prediction tasks. That is a genuine reduction in supervised labeling dependence, not a repackaging of it.
The defensible position is narrower than either camp: foundation models collapsed the cost of generic perception while leaving the cost of domain-specific verification intact. A regional insurer can now obtain a serviceable damage-segmentation model for a fraction of its 2022 cost. It still cannot obtain, from any public checkpoint, a defensible determination of whether a specific hail pattern voids a specific policy.
The Demographic Ledger Nobody Audits
The second blind spot is statistical rather than geometric, and it has been documented for longer than the foundation-model era has existed.
Primary source. NIST's FRVT Part 3: Demographic Effects (NISTIR 8280, December 2019) recorded false positive rate disparities across demographic cohorts reaching an order of magnitude and more, varying by algorithm and comparison condition.
Those findings remain structurally relevant. A model trained on a broader, noisier corpus does not inherit fairness from its scale; it inherits the corpus's sampling decisions. Self-supervision changes the provenance of the signal, not the demographics of the camera.
The procurement consequence is uncomfortable. Aggregate accuracy is now a near-useless purchasing signal. The operational questions are the false positive rate at the deployed threshold, the cohort-level dispersion around that rate, and the rate at which the system abstains. Most vendor decks contain none of those three numbers.
What the AI Act Actually Protects
The EU AI Act, in force since August 2024, applies its general regime from August 2026, with Article 6(1) product-linked high-risk obligations deferred to August 2027. Its Article 5 prohibitions — including untargeted scraping of facial images to build recognition databases and biometric categorisation inferring sensitive attributes — have applied since February 2025.
The mainstream reading is that this imposes a compliance tax on European vision firms while competitors elsewhere scale unchecked. That reading is too convenient.
Counter-argument: the Act may protect incumbents rather than punish them. A firm that already licensed its training imagery faces roughly the marginal cost of documentation. A startup whose entire advantage was a scraped corpus faces the loss of its input. Regulation of this kind converts a data-acquisition advantage into a liability — and data-acquisition advantage is precisely what challengers sell.
The same logic applies to on-device architecture. Prohibitions and documentation duties push sensitive inference toward the endpoint, where retention exposure is lower. That is not compliance theater; it is a forced architectural choice with a measurable privacy benefit. The one-sided framing mistakes a change in the shape of the moat for a reduction in its height.
The Victorville Precedent
The closest historical analogue is not another software cycle. It is the DARPA Urban Challenge of November 2007.
The earlier Grand Challenge had treated autonomous driving as a perception problem. The 2007 event, run on a closed airfield in Victorville, California, exposed the actual constraint: eleven vehicles finished, and the winning platform succeeded less because it saw better than because it integrated sensing, planning, and failure recovery into a system that could complete a route without intervention.
The lesson for vision is direct. Capability demonstrations are not deployments. The organizations that eventually captured value from autonomous perception were not those with the best benchmark scores. They were those that owned the closed-loop operational data — the rare intersections, the ambiguous occlusions, the moments when the system had to stop.
Before the Next Procurement Cycle
For a mid-sized business deploying vision — a distributor auditing pallets, a clinic triaging images, an insurer inspecting claims — the actionable list is short and mostly unglamorous.
First, inventory the pipeline as a regulated artifact. Determine whether the system performs biometric identification, biometric categorisation, or ordinary object detection. These fall under materially different obligations. Conflating them is the most common compliance error in the field.
Second, demand the operating point. Ask for the false positive rate at the deployed threshold, the cohort-level dispersion, and the abstention rate. A model that refuses 8% of inputs is not necessarily an inferior model; it may be a more honest one. Selective prediction is the maturity indicator that accuracy leaderboards conceal.
Third, make provenance a procurement gate. Where imagery becomes evidence — claims, litigation, identity — require Content Credentials conforming to the C2PA specification. Treat missing provenance as a disqualifier. Provenance survives many attacks that defeat post-hoc synthetic-media classifiers.
Fourth, for citizens, the protection is procedural. An automated face match is a probabilistic lead, not an identification. Where an adverse decision follows from one, request the human review pathway and the threshold at which the match was accepted. The right to contest is worth more than the ability to spot a synthetic image.
By April 2027
The decisive procurement question will have shifted from how accurate is it to what does it do when it does not know. Vendors without calibrated abstention will lose enterprise bids regardless of their benchmark position.
Provenance will become the contested layer. Expect a market in credential stripping alongside a market in credential verification, with uneven adoption: strong in journalism and insurance, weak in consumer platforms where distribution incentives run the other way.
The more consequential shift will be geometric. Vision foundation models will move further into occupied-space and world-state prediction for robotics, and the valuation premium will follow whoever owns the closed-loop data rather than whoever owns the largest checkpoint.
The prediction is falsifiable: within two quarters, the highest-leverage asset in a commercial vision stack will not be the model. It will be the labeled record of where the model was wrong, and what happened next.