Pilot purgatory: write the test before you build
Pilot purgatory is what happens when a system works in a room with clean data and never has to work in the workflow that actually runs the place. Most failures sit in that handoff, not in the model. The fix is to write the test before you build.
Escherbridge
A pilot is a polite word for a demonstration that has been given a budget line. In a quiet room, with a patient audience and a dataset that has been cleaned for the occasion, the system does what it was asked to do. It looks as if it works. Then the work of putting it into the workflow that actually runs the place is left for later. Later is where a lot of projects go to live, indefinitely, under a name that sounds like progress: pilot purgatory.
The conversations have already moved on from whether the technology can perform. On November 5, 2026, Preeti Singh, writing in Forbes Business Council, put the question executives are asking in a single line: not whether the technology works, but where the business impact is. The essay is a contributed piece, not a news investigation, and the figures in it should be read that way. It cites Gartner as finding that, on average, only 41 percent of generative AI prototypes make it into production, and McKinsey's global AI survey as finding that more than 80 percent of respondents were still not seeing a tangible enterprise-level impact on earnings from generative AI. It also cites IDC as reporting that 42 percent of organizations worldwide find it difficult or impossible to accurately assess return on investment for digital and AI investments, pointing to inconsistent baselines and limited visibility into long-term impact.
That last point is easy to skip past. It is the whole problem. If you did not write down, before anyone built anything, what would count as a result, you will not be able to say, a year later, whether you got one. A pilot that cannot name its test in advance is a demo that grades its own homework.
The failure, when it comes, usually does not arrive in the lab. On September 1, 2026, Toni Stork summarized the OMMAX AI Trends Report 2026, based on a Statista survey of 250 decision-makers in April and May 2026 across France, Germany, Italy, the Netherlands, and the UK, conducted for OMMAX, Ibexa, and Make. The report finds that 79 percent of AI failures occur during the pilot or in the transition from pilot to production. The largest single failure point is the post-pilot transition, cited by 44 percent of respondents whose organizations use AI beyond isolated tasks. Another 35 percent say initiatives most commonly fail during the pilot. Only 12 percent point to the idea stage, and 10 percent to the scaling stage.
Those numbers come from a particular survey, about a particular kind of software. They are still a useful map of a more general stall. A pilot is allowed to skip the unglamorous parts: the integration, the permissions, the data that is incomplete, the person who has to live with the result, the cost of running the thing when it is no longer a special project. Production is not allowed to skip them. Stork's article is plain about the gap. Integration complexity is the top barrier to stronger AI impact, cited by 40 percent of respondents, ahead of talent shortages and high costs or unclear ROI at 32 percent each, and fragmented data foundations at 30 percent. "What appears to be a model problem," the article concludes, "is often a system and coordination problem."
The same piece offers a better definition of pilot purgatory than a graveyard of slide decks: "If these requirements appear only after the pilot succeeds, the organization has not completed a pilot; it has deferred most of the product." The demo was never asked to survive contact with the systems, the people, or the measures that would have to carry it.
The report also finds that 65 percent of respondents said completed AI projects exceed their original budgets, and that 67 percent believe their infrastructure enables AI scaling even as integration, skills, ROI, and fragmented data remain major barriers. Confidence in the pipes is not the same thing as a path out of the room.
Nobody has to be kidding themselves. A pilot optimized for learning and demonstration will use a narrow dataset, a small user group, and a stand-alone interface. Those choices are reasonable when the question is "can this work under defined conditions?" They become a trap when a yes to that question is treated as a yes to "can this deliver a result, repeatedly, in the workflow that already exists, at a cost someone will still defend next year?" Those are different tests. If only the first one is written down, the second one is postponed, and the postponement gets a name.
The useful move is earlier, and duller, than a new model. Define the outcome, the baseline, and the stop condition before anyone builds. Include at least one realistic integration in the hypothesis, if the thing will have to talk to a real system. Use data that looks like production data, including the messy parts. Decide what the system is allowed to do, what needs a person, and how you will know if it failed. Then let the evidence decide, including the decision to stop. A stage gate without a stop condition does not decide anything.
That is the lab method in plain language: match a real need with tools, write the test first, and let the result decide what happens next. PrimusNeo is organized as a clearing house and an innovation lab on that prior. It has not published a success rate for taking enterprise pilots into production.
AMP, the operational runtime in the stack, is a public toolkit for putting a 3D, geospatial, and media model in view. Active releases live on art-media-platform/amp.SDK; plan-systems/amp.SDK is a fork whose last commit was in July 2025. The planet-scale runtime, amp.planet, is not public. A shared view of a site is not, by itself, an escape from pilot purgatory. It is one of the pieces a production-shaped pilot would actually need: a place where the people who have to live with the result can look at the same object, without the work existing only in a slide.
The demo will keep working in the room. The question is whether anyone wrote down, before the room was booked, what would have to be true outside it.
Sources
- Preeti Singh, “Beyond Pilots: Making AI Pay Off Across The Whole Enterprise,” Forbes Business Council, November 5, 2026
- Toni Stork, “From AI pilot to production: How enterprises can cross the scaling threshold,” OMMAX, September 1, 2026
- art-media-platform/amp.SDK (active upstream of the named AMP fork)
Related
- Why good work goes unbuiltThe paper’s name for the same stall: write the test before anyone builds.
- A digital twin shouldn't live in someone else's cloudNames the habit. A pilot that never leaves the account it was demoed in.
- Paid twice, or never, is the same bugThe same prior, on a ledger. The test is written before the system is allowed to act.