Why isolated AI experiments rarely reach scale

The pattern is familiar. An enthusiastic department starts a pilot with an AI tool. The first results are good, there is a presentation to the management team and everyone agrees that more should be done with this. A year later the pilot is still running in the same department, with the same three users, and a second pilot has appeared in another department. Scaling is nowhere in sight.

We see this in organizations of very different sizes, in the public and the private sector. The causes are remarkably consistent. This article describes them, explains why a pilot and an organization-wide application are two different things and describes what a management team can do to get from experiments to working applications.

Why pilots succeed

First, something that is often overlooked: most pilots succeed. They prove that a tool works, that users save time with it and that the quality is acceptable. That is no coincidence. A pilot has everything going for it: motivated users who chose the tool themselves, a small and manageable process, no dependence on other departments, no formal obligations and the freedom to deviate from the standard.

Exactly those conditions disappear when scaling. The users are no longer the enthusiasts, the process touches other departments, the systems have to be connected and the organization's rules start to apply. A pilot tests the tool. Scaling tests the organization. Anyone extrapolating the pilot's result to the whole organization is comparing two different things.

1. The pilot had no owner at the level that matters

A pilot is carried by an enthusiastic employee or team lead. Scaling requires decisions about budget, processes and priorities that can only be taken at board level. As long as nobody in the management team considers the application "theirs", it remains a hobby project with a good story.

You can recognize this in the questions asked in the management team. If the question is "what do we think of this?", there is no owner. If the question is "what do we need to roll this out across the whole process and who arranges that?", there is one. The difference is not a matter of enthusiasm. It is about mandate: someone who can allocate budget, shift priorities and hold other departments to account.

2. The process did not change with it

In the pilot the tool worked alongside the existing process: someone entered data twice, checked manually and cleaned up afterwards. That is fine for a test and untenable for the organization. Scaling means redesigning the process, including the roles, the controls and the surrounding systems. That work is almost always underestimated in the pilot phase.

An example we encounter in various forms: a tool that summarizes case files saves the case handler half an hour per file. In the pilot that is a gain. On rollout it turns out that the summary has to be stored in the case system, that the quality check needs an adjusted step, that the distribution of work changes because handlers finish sooner, and that the manager loses sight of the workload because the old measure (files per week) no longer holds. None of that is unsolvable. All of it has to be designed.

Technically, more changes in that step than it seems. In the pilot a user pastes text into a tool and copies the answer back. In production the tool has to be connected to the case system, with permissions per user, logging of what was generated and a fallback when the service is unavailable. And the data that were cleaned by hand in the pilot now have to arrive automatically in the right shape. That is engineering work, and it is usually the part missing from the pilot budget.

A pilot proves that something can be done. Scaling requires proof that the organization is willing to carry it.

3. Skills are concentrated in a few people

The three users from the pilot know exactly how to steer the tool and when to distrust its output. That knowledge lives in their heads. Without a deliberate approach to AI literacy and work instructions, the knowledge does not scale, and neither does the application.

This is about more than knowing the buttons. Over months the pilot users have developed a feel for the errors the tool makes and the situations in which they are better off working themselves. New users do not have that feel and therefore make the mistakes the pilot discovered long ago. Since 2 February 2025 a sufficient level of AI literacy is also an obligation under the AI Act, which makes the question of how that knowledge is transferred a compliance question too.

4. Governance arrives too late, or not at all

During the pilot the department could decide for itself. Organization-wide use raises questions about data protection, the AI Act, vendor dependence and oversight. If there are no answers, the legal department rightly puts on the brakes. Governance that is light and clear from the start prevents that brake from becoming an emergency stop later.

Governance does not have to be complete before the pilot. The pilot should, from the start, note the questions that will have to be answered when scaling: which data pass through the tool, who is responsible for the outputs, which risk class the application falls into, what the vendor contract says. Anyone answering those questions during the pilot has no surprises when scaling.

5. There is no baseline

A cause that is mentioned less often: many pilots have no measurable starting point. "The users are enthusiastic" and "it saves time" are not figures. Without a baseline a management team cannot judge what scaling delivers, and without that judgment there is no basis for the investment scaling requires. A pilot that measured lead time, error rate and time spent at the start has, after three months, a story a board can evaluate. A pilot without a baseline has only enthusiasm.

The economics of scaling

There is a reason pilots are cheap and scaling is expensive. A pilot costs a license and the time of a few people. Scaling costs process redesign, integration with systems, training of dozens or hundreds of staff, adjustment of work instructions and controls, legal review and a period of guidance. Those costs are a multiple of the pilot, and they largely fall before the returns.

Scaling therefore above all requires choosing. An organization cannot scale ten applications at once. It can do two or three well. The pilots that are not chosen have not failed; they have produced information. But they do need to be stopped, otherwise they keep claiming attention and licenses without ever becoming part of the work.

From pilot to program in four steps

Prove. The pilot, but with a baseline, a success measure agreed in advance and a notebook for the questions scaling will raise. Duration: six to twelve weeks.

Prepare. The choice by the management team, with an owner, a budget and a decision about what will not happen. The redesign of the process with the people who run it. The governance questions answered. The data situation checked. Duration: two to three months.

Embed. The rollout in the first complete process or the first department, with work instructions, training, adjusted controls and a period of measuring and adjusting. This is where it shows whether the design holds. Duration: three to six months.

Expand. Only when the first rollout demonstrably works does the extension to other departments or processes follow, with the lessons from the first round incorporated. Every next round goes faster, because the structure is in place.

Most organizations skip the second step. They go from proving straight to rolling out, and get stuck on everything that should have happened in preparation.

What a management team can do

More pilots rarely solve this. What helps is a number of choices the management team makes explicitly:

• Which two or three applications have enough potential to invest in seriously? Choose those, and end the other pilots explicitly.
• Who in the management team owns each chosen application, with budget and mandate?
• Which process is going to change, and who redesigns it together with the people who run it?
• Which skills need to be spread more widely across the organization, and how?
• Which ground rules apply to new applications, so the next pilot starts in the right structure from day one?
• What is the success measure, and when will it be reviewed?

An organization that answers these questions has, in effect, the core of an AI roadmap. The pilots did what they were supposed to do. What was missing was the structure around them.

Whether the pilot worked is rarely the question. The question is whether the organization is ready to carry what the pilot has proven.