skip to content

The AI Portability Test


Ask an organisation whether it owns its AI system and it will usually point to the wrong evidence. It may own the application, the cloud account, the prompts, or the contract with the model provider. It may even run the model on infrastructure it controls. None of this answers the question that matters.

If the model had to be replaced tomorrow, would the system keep what it had learned?

That is the AI portability test.

Most discussions of portability stop at the technical boundary. Can the application call another endpoint? Can the prompts be exported? Does the orchestration layer support more than one provider? These are useful properties, but they describe whether the system can move. They do not describe what survives the move.

An enterprise AI system accumulates more than code. It learns which context matters, which sources deserve priority, which outputs pass review, which exceptions need escalation, which mistakes recur, and where human judgment changes the answer. That learning appears in evaluation cases, traces, corrections, approval rules, tool permissions, workflow state, and the institutional memory surrounding the model.

If those things live inside the provider’s platform, changing the model may be technically easy and operationally ruinous. The new endpoint can receive the same prompt while the organisation quietly returns to day one. The integration moved. The intelligence did not.

The current ZeroEntropy transition makes the distinction unusually clear. Notion acquired the retrieval company on 24 July, and ZeroEntropy says its managed products will be supported only until 4 September. The models are now available under an open licence, so customers can carry the weights elsewhere. That does not automatically carry their managed search service, indexed data, evaluation history, or operating knowledge with them.

This is the lock-in that matters most. It is not dependence on an API but dependence on accumulated learning that cannot be inspected, replayed, or carried elsewhere. The more the system improves, the more expensive escape becomes. What looks like a compounding advantage is really a growing exit penalty.

The remedy is not to build every layer yourself. Enterprises already rent databases, networks, identity systems, and software while retaining meaningful control of their data and operating rules. AI should be treated the same way. A company can rent models and infrastructure while owning the context, evidence, authority, and feedback that make those components useful.

Ownership, in this sense, is functional. The organisation owns the learning loop when it can substitute the model without surrendering its memory, reproduce the evaluations that justified reliance, retain the traces behind past decisions, and carry forward the rules that govern action. It owns the loop when accepted corrections become portable tests rather than private improvements inside somebody else’s product.

Evidence is the hardest part of this architecture because the component being improved should not be the sole custodian of the record used to judge it. An agent may be allowed to change its prompts, tools, policies, or code. It should not be allowed to rewrite the history that shows why those changes were made or whether they worked. The executor can evolve. The evidence beneath it must remain independently durable.

This separation turns portability from a procurement promise into a testable property. Replace one model with another and replay the same evaluation set. Preserve the same permissions and approval boundaries. Compare the new behaviour with the old traces. Keep the rejected cases as well as the successful ones. If a change creates a regression, the organisation should be able to see it without asking the outgoing provider to explain its own record.

The exercise also reveals whether the system has been learning at all. Many AI deployments collect large volumes of conversation history and call it memory. A history is not a learning loop. Learning requires a mechanism that converts experience into a durable change: a new evaluation, a revised source rule, a narrower permission, a better escalation path, or an amended definition of acceptable work. If none of that survives outside the model session, the system is producing activity rather than accumulating intelligence.

This is why model access will become a smaller part of enterprise advantage even as models become more capable. Better models lift everyone who can buy them. The differentiating asset is the organisation-specific machinery that turns general capability into better work and then preserves what the work taught.

A useful portability review therefore begins with a disruption question. If the preferred model or platform disappeared for thirty days, what would the organisation still possess? Could it route the same work through a replacement? Could it reproduce the evidence behind prior decisions? Could it test whether the replacement preserved the behaviours that mattered? Could it explain what had been lost?

If the answer is no, the organisation does not own an AI capability. It rents a working arrangement whose accumulated intelligence belongs to its continuity.

You do not need to own the model. You need to own the path by which the work gets better.

Related by topic
  1. The Model Is Not the Unit of Return
  2. After the Harness
  3. The SOP Is the Product