Stop the Centralization Pendulum in AI Teams

The AI center of excellence was created to bring order. Six months later, product teams call it a bottleneck. Leadership responds by embedding AI specialists in business units. A year later, finance finds duplicated contracts, security finds untracked systems, and engineering finds five incompatible ways to evaluate similar applications. The call for centralization returns.

Nothing in that scenario proves either model is wrong. It shows that each model exposes a different cost.

Centralization makes duplication and inconsistency less visible, but distance and waiting become easier to see. Decentralization makes teams more responsive, but fragmentation and weak enterprise visibility accumulate. Leaders often react when one set of costs becomes politically louder than the other. They reorganize, enjoy the immediate relief, and later discover the costs that the new design moved out of view.

This is the organizational pendulum. It affects IT, data, analytics, cloud platforms, AI governance, and now agentic systems. The way to stop it is not to choose a permanent winner between central control and local autonomy. It is to diagnose the complaint precisely, fix the mechanism producing it, and change structure only when structure is actually the constraint.

Failure mode 1: a service problem is mistaken for a structure problem

Imagine that domain teams wait three weeks for access to an approved model. The central AI platform group requires a ticket, a risk questionnaire, an architecture review, and manual credential setup. Product leaders conclude that centralization is too slow and ask for independent model accounts.

The complaint is legitimate. The diagnosis may not be.

The problem could be the placement of authority: perhaps the central group truly owns a decision that domain teams are better equipped to make. But it could also be an immature service with no self-service path, unclear risk tiers, insufficient staffing, or a review process that treats a low-impact drafting assistant like an autonomous financial workflow.

Decentralizing model access would reduce the visible queue. It could also multiply vendor agreements, identity patterns, logging gaps, and incident procedures. The organization would have changed its structure without repairing its service design.

Before moving ownership, inspect the service:

• How much time is active review, and how much is waiting?
• Which requests are routine enough to automate or preapprove?
• Are requirements published before teams submit work?
• Do low-, medium-, and high-impact use cases follow different paths?
• Does the shared team have a service target and enough capacity to meet it?
• Which exceptions reveal missing platform capability rather than unreasonable demand?

A shared function should be judged by the capability it creates for other teams. If it owns model access, its job is not merely to control access. It should make safe access predictable and usable.

This is why an internal platform has to be managed as a product. The note on treating internal AI systems like products goes deeper into discovery, adoption, support, and lifecycle ownership. A mandate can force teams through a central door; it cannot make the door work well.

Failure mode 2: the delayed cost of a choice is treated as new evidence

Every operating model has benefits that appear early and costs that appear later.

Centralization can quickly improve purchasing leverage, inventories, standards, and executive visibility. Its later costs may include a growing backlog, weaker domain understanding, generic solutions, and dependence on a small expert group. Decentralization can quickly improve local speed, ownership, and fit. Its later costs may include duplicated tooling, inconsistent controls, fragmented data, and difficulty learning across teams.

Leaders get into trouble when they compare the mature costs of the current model with the promised benefits of its alternative. The comparison is structurally unfair. The proposed model is still a slide; the existing one has lived long enough to reveal its defects.

A serious review compares the full operating profiles:

The last row matters. A cheaper central service may transfer work to product teams that spend hours explaining context and chasing tickets. A fast local model may transfer cost to security, procurement, and operations. Budget lines show where money is recorded, not necessarily where effort is consumed.

The FinOps Foundation’s current practice operations guidance recognizes that a technology-value practice may use a central team, a virtual team, or a hub-and-spoke design depending on organizational needs. Its 2026 State of FinOps also reports that centralized enablement remains common while hub-and-spoke structures appear more often in large enterprises. These are useful patterns, not universal answers. The more important principle in the FinOps Framework is that accountability is distributed even when enablement is central.

Failure mode 3: louder sentiment outruns operating evidence

Organizational complaints contain information, but volume is not the same as diagnosis.

A product team may reasonably complain that governance slows delivery. A risk team may reasonably complain that local experimentation creates invisible exposure. A central platform group may see irresponsible exceptions. Domain teams may see a platform that ignores real work. Each group observes the system from the place where its costs arrive.

Leadership fails when it turns the most repeated story into the whole truth.

Use complaints as hypotheses. Convert each one into something observable:

• “The central team is slow” becomes time from a complete request to a usable result, separated into work and waiting.
• “Every team is duplicating work” becomes an inventory of repeated components, contracts, evaluations, and support effort.
• “Standards block innovation” becomes the number of exceptions, the reason for each one, and the outcome of approved experiments.
• “Local teams are unsafe” becomes specific missing controls, incidents, unowned systems, or actions outside risk tolerance.
• “The platform does not understand us” becomes rework caused by lost requirements or domain-specific needs the shared path cannot support.

Evidence will not remove politics. It improves the quality of the disagreement. Leaders can decide which tradeoff they are accepting instead of treating a reorganization as a neutral correction.

When I teach data and AI, learners often want the final sequence of steps before they understand the conditions that make a step appropriate. Organizational design invites the same mistake at a larger scale. “Create a center of excellence” or “embed people in the business” sounds actionable, but the label cannot decide whether the conditions fit.

Failure mode 4: AI governance is centralized but accountability is not

AI makes the pendulum more consequential because a system can span several organizational boundaries at once. A domain team defines the workflow. A platform team provides model access and tracing. Security controls identity. A data owner approves sources. A vendor supplies the model. An operations team supports the service. A business executive owns the outcome.

Moving all these people into one department is neither realistic nor necessary. Leaving their decision rights implicit is dangerous.

The NIST AI Risk Management Framework Core calls for documented roles and communication across teams, empowered and trained personnel, multidisciplinary perspectives, and executive responsibility for AI risk decisions. It does not prescribe centralization. It describes capabilities an organization needs to produce through whatever structure it chooses.

For each production AI system, the operating model should make these decisions visible:

1. Who owns the business outcome and acceptable failure boundary?
2. Who owns the data’s meaning, quality, and permitted use?
3. Who defines and tests evaluation cases?
4. Who grants the model or agent access to tools and actions?
5. Who monitors quality, cost, latency, security, and user impact?
6. Who may pause the system during an incident?
7. Who decides whether it should scale, change, or retire?

A central governance group can define minimum evidence and prohibited uses. It cannot generate domain-specific evaluation cases from a distance. A product team can understand the workflow. It should not invent its own identity controls or decide enterprise risk tolerance alone.

This is the difference between centralized accountability and coordinated accountability. The first tries to place the whole problem inside one function. The second keeps each decision near the required context while making the complete control loop visible.

The boundary framework in Centralize the AI Platform, Not Every Decision explains which capabilities usually benefit from shared ownership. The concern here is what happens later: whether leaders improve those boundaries when friction appears or throw them away and restart the cycle.

Failure mode 5: reorganization becomes a substitute for maintenance

Reorganizations are attractive because they are visible. A new reporting line, team name, or executive appointment creates a clear event. Maintaining an operating model is quieter. It requires reviewing queues, retiring obsolete controls, fixing platform usability, moving mature skills into domain teams, updating decision rights, and funding work that prevents future friction.

Without maintenance, any structure decays.

A center of excellence that was useful when AI knowledge was scarce can become a permanent delivery team for work that domains are now capable of owning. Embedded teams can drift apart after the common standards and shared forums lose funding. A hub-and-spoke model can leave people with two sets of expectations and no conflict rule. A platform can accumulate mandatory features while its self-service experience deteriorates.

Review the model on a schedule and after material changes such as an acquisition, regulation, serious incident, rapid increase in AI spend, or expansion from advisory tools to agents that take action. The review should ask:

• Which capabilities are now common enough to share?
• Which skills are now common enough to distribute?
• Which central decisions can become guardrails or automated controls?
• Which local variations are producing learning, and which are pure duplication?
• Where has shared responsibility become unowned responsibility?
• Which committee, approval, platform feature, or exception path has outlived its purpose?

This approach complements designing technology teams around decisions rather than org charts. A reporting line can support the work, but recurring decisions and service flows reveal whether the design functions.

Run the pendulum review before moving people

When pressure for a structural reversal grows, pause long enough to run one bounded review. It does not need to become a six-month transformation study.

Name the triggering complaint. Avoid broad diagnoses such as “the model no longer works.” Identify the delayed launch, duplicated contract, control failure, lost customer context, or unclear decision that made the issue urgent.

Locate the mechanism. Is the cause authority in the wrong place, a badly designed shared service, insufficient capacity, a missing interface, conflicting incentives, or unclear risk tiers? More than one may be true.

Measure transferred cost. Follow work across team boundaries. Include waiting, coordination, rework, shadow tooling, operational support, vendor spend, and risk exposure.

Repair inside the current structure first when possible. Publish service levels. Automate routine controls. Embed a domain specialist temporarily. Create a shared evaluation format. Clarify an escalation rule. A small repair can test the diagnosis without paying the full cost of reorganization.

Set a structural threshold. Move ownership only if the present placement repeatedly blocks the outcome despite credible service and process improvements, or if the required context and authority cannot realistically meet where the decision currently sits.

Preserve the strengths of the current model. If work is decentralized, keep central inventory, minimum controls, shared telemetry, and reusable foundations. If work is centralized, preserve direct domain participation, bounded local authority, and short feedback paths.

Define when to review again. The decision is not eternal. Record the expected benefit, guardrail measures, transition cost, and conditions that would justify another change.

This review makes structural change harder to sell as a quick cure, but easier to defend when it is genuinely necessary. It also reduces the risk described in How to Change Technical Teams Without Breaking What Works: removing invisible knowledge and relationships while improving the diagram.

Progress means learning across the cycle

An organization may still centralize or decentralize after doing this work. Stopping the pendulum does not mean freezing the structure. It means refusing to repeat the same reversal for the same unexamined reasons.

A mature move toward centralization should preserve local context, make the shared service measurable, and give routine work a fast path. A mature move toward decentralization should preserve enterprise visibility, common interfaces, and explicit accountability. In both directions, leaders should be able to state which problem they are solving, which cost they knowingly accept, and how they will know whether the design works after the initial relief fades.

AI organizations need both consistency and adaptation. Security, identity, observability, inventories, and risk tolerance benefit from common foundations. User needs, workflow choices, data meaning, and evaluation depend on domain knowledge. The leadership work is not choosing one side of that tension. It is maintaining a system in which the tension remains productive.

The next reorganization should not be triggered because one word—centralized or decentralized—has become unpopular. It should follow evidence that a decision, capability, or service is persistently in the wrong place. That is how an organization stops circling and starts carrying lessons forward.