What is an AI control plane, and how does it differ from an AI gateway?
An AI gateway sees the call to the model. An AI control plane governs the agent. Which one you need, and how the enterprise LLM gateways compare.
- Topic
- Control plane
- Product
- Novie Control
- Format
- Guide
- Reading time
- 5 min
- Published
- 2 October 2026
- Updated
- 21 October 2026
An AI gateway sits between applications and model providers and manages the call to the model: routing, keys, rate limits, caching and cost per request. An AI control plane governs what AI does across the organisation: which agents exist, who owns them, what each is approved to reach and spend, and a record of every action. A gateway sees the request to the model. A control plane sees the agent, its permissions and its owner.
Where the terms come from.
Both are borrowed from networking and cloud infrastructure. A gateway is a point traffic passes through. A control plane is the layer that decides policy, as distinct from the data plane that carries the traffic. Applied to AI, the gateway is the path of the call, and the control plane is the set of rules and records about who may make which calls, for what, at what cost.
TakeawayThe gateway is the path a call takes and the control plane is the set of rules and records about who may make it.
What an AI gateway does.
- Routes each request to a model and provider, with failover when one is down.
- Holds the provider keys, so applications do not.
- Applies rate limits and, in some cases, content filters.
- Caches repeated requests.
- Records tokens and cost per request.
TakeawayA gateway routes, keys, limits, caches and prices each request to a model.
What an AI control plane does.
- Keeps the inventory: every agent, its owner, its model and tools, its data and systems.
- Holds the approvals: the scope each agent was approved for, by whom, until when.
- Sets and enforces budgets: caps by organisation, team, project and agent, holding before spend is committed.
- Records actions: what each agent did, in which system, on whose authority.
- Brings approvals back for review when an agent's model, tools or reach change.
TakeawayA control plane keeps the inventory, the approvals, the budgets and the record of what each agent did.
The difference that matters: the enforcement point.
A gateway can see that a call went to a model and what it cost. It cannot see that the answer was then used to change a customer record, approve a refund or send an email, because those actions happen in other systems, through other tools. A control plane is built around the agent rather than the call, so its question is whether this agent was allowed to take this action, and who answers for it.
That is why the two are often used together. The gateway is a good place to enforce a budget, because every call passes through it. The control plane is where the budget, the owner and the approval are defined, and where the record of what the agent did is kept.
TakeawayA gateway sees the call to the model, and only a control plane built around the agent can say whether the action it then took was allowed.
Enterprise LLM gateway comparison.
The gateway market is growing quickly. In August 2026, Gartner forecast spending on AI gateways of $251 million in 2026 and $429 million in 2027, growth of 70.9 percent, and expects major vendors to keep their lead in gateways by adding AI features to the API and data management products they already sell.
Two gateways come up most often in enterprise evaluations: LiteLLM, an open-source proxy, and Portkey, which Palo Alto Networks acquired in May 2026. The table sets them beside the gateway built into a cloud provider's model service and beside a control plane. Details are taken from each vendor's own documentation as of 19 October 2026; check them again before you buy.
| LiteLLM | Portkey | Cloud provider gateway | AI control plane | |
|---|---|---|---|---|
| What it is | Open-source (MIT) proxy and SDK, self-hosted | AI gateway with observability and guardrails; part of Palo Alto Networks since May 2026 | Model access built into one cloud platform | A governance layer over agents, owners, budgets and actions |
| Models | 100+ providers through one API | 250+ models through one API | That cloud's model catalogue | Whichever models the organisation chooses |
| Spend limits | Virtual keys with budgets per key, team and user | Cost or token limits per integration and workspace, on Enterprise and some Pro plans | Quotas and budget alerts by account | Caps by organisation, team, project and agent, with a named approver |
| Deployment | Your infrastructure; the enterprise licence adds SSO, audit logs, role-based access and air-gapped deployment | Managed service or private-cloud hosting; open-source gateway | Inside that cloud | Varies by vendor |
| What it sees | Model calls routed through it | Model calls routed through it | Model calls in that cloud | The agent: owner, approval, spend and actions |
| Best fit | Platform teams who want to run and extend their own gateway | Teams who want a managed gateway with guardrails and tracing | Organisations committed to one cloud's models | Organisations answering for agents to a board, auditor or regulator |
Any of the three gateways can be the path every call takes. None of them records who approved an agent or what it then did in other systems, which is the job a control plane adds above them.
TakeawayAny gateway can carry every call, and none records who approved an agent or what it did in other systems, which is what a control plane adds.
Which one you need.
- A gateway alone fits an organisation calling models from a few applications its engineers own, where the question is cost and reliability per request.
- A control plane is needed once agents act on the organisation's behalf, more than one team builds them, and someone has to answer to a board, an auditor or a regulator for what they did.
- Most organisations running agents in production need both: a path every call takes, and a layer that says who may take it.
The numbers say most are now in the second group. In KPMG's Q2 2026 AI Pulse Survey, 53 percent of large US enterprises had AI agents deployed, and Deloitte's State of AI in the Enterprise 2026 found 21 percent have a mature governance model for them.
TakeawayA gateway alone fits a few applications calling models, and most organisations running agents in production need both.
How Novie Control does it.
Novie ControlAvailable now
Governance and accountability.
Give every workload a named owner, a budget and clear boundaries.
1,284
calls governed today.
Example figure.
Sources
- KPMG, AI Quarterly Pulse Survey Q2 2026 (24 June 2026)
- Deloitte, Business and IT leaders report AI agents are scaling faster than their guardrails (24 April 2026)
- Gartner, Gartner Forecasts the Market for Securing AI Will Reach $4.8 Billion in 2027 (26 August 2026)
- LiteLLM, documentation
- LiteLLM, Enterprise
- Portkey, What is Portkey?
- Portkey, Budget limits
- Palo Alto Networks, Palo Alto Networks Completes Acquisition of Portkey to Secure AI Agents (29 May 2026)