Skip to main content

What is an AI control plane, and how does it differ from an AI gateway?

An AI gateway sees the call to the model. An AI control plane governs the agent. Which one you need, and how the enterprise LLM gateways compare.

Product
Novie Control
Format
Guide
Reading time
5 min
Published
2 October 2026
Updated
21 October 2026
In short

An AI gateway sits between applications and model providers and manages the call to the model: routing, keys, rate limits, caching and cost per request. An AI control plane governs what AI does across the organisation: which agents exist, who owns them, what each is approved to reach and spend, and a record of every action. A gateway sees the request to the model. A control plane sees the agent, its permissions and its owner.

Where the terms come from.

Both are borrowed from networking and cloud infrastructure. A gateway is a point traffic passes through. A control plane is the layer that decides policy, as distinct from the data plane that carries the traffic. Applied to AI, the gateway is the path of the call, and the control plane is the set of rules and records about who may make which calls, for what, at what cost.

TakeawayThe gateway is the path a call takes and the control plane is the set of rules and records about who may make it.

What an AI gateway does.

  • Routes each request to a model and provider, with failover when one is down.
  • Holds the provider keys, so applications do not.
  • Applies rate limits and, in some cases, content filters.
  • Caches repeated requests.
  • Records tokens and cost per request.

TakeawayA gateway routes, keys, limits, caches and prices each request to a model.

What an AI control plane does.

  • Keeps the inventory: every agent, its owner, its model and tools, its data and systems.
  • Holds the approvals: the scope each agent was approved for, by whom, until when.
  • Sets and enforces budgets: caps by organisation, team, project and agent, holding before spend is committed.
  • Records actions: what each agent did, in which system, on whose authority.
  • Brings approvals back for review when an agent's model, tools or reach change.

TakeawayA control plane keeps the inventory, the approvals, the budgets and the record of what each agent did.

The difference that matters: the enforcement point.

A gateway can see that a call went to a model and what it cost. It cannot see that the answer was then used to change a customer record, approve a refund or send an email, because those actions happen in other systems, through other tools. A control plane is built around the agent rather than the call, so its question is whether this agent was allowed to take this action, and who answers for it.

That is why the two are often used together. The gateway is a good place to enforce a budget, because every call passes through it. The control plane is where the budget, the owner and the approval are defined, and where the record of what the agent did is kept.

TakeawayA gateway sees the call to the model, and only a control plane built around the agent can say whether the action it then took was allowed.

Enterprise LLM gateway comparison.

The gateway market is growing quickly. In August 2026, Gartner forecast spending on AI gateways of $251 million in 2026 and $429 million in 2027, growth of 70.9 percent, and expects major vendors to keep their lead in gateways by adding AI features to the API and data management products they already sell.

Gartner, August 2026.

Two gateways come up most often in enterprise evaluations: LiteLLM, an open-source proxy, and Portkey, which Palo Alto Networks acquired in May 2026. The table sets them beside the gateway built into a cloud provider's model service and beside a control plane. Details are taken from each vendor's own documentation as of 19 October 2026; check them again before you buy.

LiteLLMPortkeyCloud provider gatewayAI control plane
What it isOpen-source (MIT) proxy and SDK, self-hostedAI gateway with observability and guardrails; part of Palo Alto Networks since May 2026Model access built into one cloud platformA governance layer over agents, owners, budgets and actions
Models100+ providers through one API250+ models through one APIThat cloud's model catalogueWhichever models the organisation chooses
Spend limitsVirtual keys with budgets per key, team and userCost or token limits per integration and workspace, on Enterprise and some Pro plansQuotas and budget alerts by accountCaps by organisation, team, project and agent, with a named approver
DeploymentYour infrastructure; the enterprise licence adds SSO, audit logs, role-based access and air-gapped deploymentManaged service or private-cloud hosting; open-source gatewayInside that cloudVaries by vendor
What it seesModel calls routed through itModel calls routed through itModel calls in that cloudThe agent: owner, approval, spend and actions
Best fitPlatform teams who want to run and extend their own gatewayTeams who want a managed gateway with guardrails and tracingOrganisations committed to one cloud's modelsOrganisations answering for agents to a board, auditor or regulator

Any of the three gateways can be the path every call takes. None of them records who approved an agent or what it then did in other systems, which is the job a control plane adds above them.

TakeawayAny gateway can carry every call, and none records who approved an agent or what it did in other systems, which is what a control plane adds.

Which one you need.

  • A gateway alone fits an organisation calling models from a few applications its engineers own, where the question is cost and reliability per request.
  • A control plane is needed once agents act on the organisation's behalf, more than one team builds them, and someone has to answer to a board, an auditor or a regulator for what they did.
  • Most organisations running agents in production need both: a path every call takes, and a layer that says who may take it.

The numbers say most are now in the second group. In KPMG's Q2 2026 AI Pulse Survey, 53 percent of large US enterprises had AI agents deployed, and Deloitte's State of AI in the Enterprise 2026 found 21 percent have a mature governance model for them.

KPMG, AI Pulse Survey, Q2 2026; Deloitte, State of AI in the Enterprise 2026.

TakeawayA gateway alone fits a few applications calling models, and most organisations running agents in production need both.

How Novie Control does it.

Novie ControlAvailable now

Governance and accountability.

Give every workload a named owner, a budget and clear boundaries.

1,284

calls governed today.

Sources

  1. KPMG, AI Quarterly Pulse Survey Q2 2026 (24 June 2026)
  2. Deloitte, Business and IT leaders report AI agents are scaling faster than their guardrails (24 April 2026)
  3. Gartner, Gartner Forecasts the Market for Securing AI Will Reach $4.8 Billion in 2027 (26 August 2026)
  4. LiteLLM, documentation
  5. LiteLLM, Enterprise
  6. Portkey, What is Portkey?
  7. Portkey, Budget limits
  8. Palo Alto Networks, Palo Alto Networks Completes Acquisition of Portkey to Secure AI Agents (29 May 2026)

Other articles

Frequently asked questions

What is an AI control plane?

The layer that governs what AI does across an organisation: the inventory of agents and their owners, the approval and scope for each, budgets that hold before spend is committed, and a record of every action taken on whose authority.

What is an AI gateway?

A point between applications and model providers that every model call passes through. It routes requests, holds provider keys, applies rate limits, caches repeats and records tokens and cost per request.

Do we need an AI gateway or an AI control plane?

A gateway alone suits a few engineer-owned applications calling models. Once agents act on the organisation's behalf and someone has to answer for them, a control plane is needed, usually with a gateway as the path every call takes.

Which LLM gateway is best for an enterprise?

It depends on who runs it. LiteLLM suits platform teams who want an open-source gateway in their own infrastructure. Portkey suits teams who want a managed gateway with guardrails and tracing. A cloud provider's gateway suits organisations committed to that cloud's models. Once agents act on the organisation's behalf, add a control plane above the gateway.

Can an AI gateway govern AI agents?

It can limit and record the calls an agent makes to a model. It cannot see what the agent then does in other systems, or who approved it, which is what governance has to answer.

Build the intelligenceyour organisation compounds.