Building an effective AI governance framework.

01Overview

The framework has eight parts, covered in this order:

• Accountability: one senior owner, and a named person for every AI system.
• Inventory: every AI system you build, buy or find embedded in other software.
• Risk levels: controls sized to what each system can do.
• Approval path: how a new system gets from request to live.
• Enforcement: controls that run while AI runs.
• Human control: least agency, and a person who can stop any agent.
• Spend: caps that hold before money is spent.
• Review and reporting: monthly checks and a quarterly report to the board.

Here's where each part comes from:

02Accountability

Australia's guidance puts accountability first. Its opening practice reads "Decide who is accountable", and it asks you to "make a specific person accountable for every AI system your organisation uses". NIST makes the same call, with Govern as the first of its four functions.

Give every system a name, and every name a system

A common mistake is handing AI governance to a committee and stopping there. Committees are good at setting direction. They're less good at answering for one agent's decision at 2am.

Instead, set it up in two layers. One senior leader owns AI governance overall. Then every AI system on your list gets one named person who answers for what it does and what it costs.

1. ACCOUNTABILITY
   Sources: Australia practice 1 · NIST Govern
   AI governance owner: [senior leader, role]
   System owners: one named person for every AI system
   Board oversight: [committee], [frequency]

Ask the 2am question

For every agent, ask: if it makes a mistake at 2am, whose phone rings? If nobody can answer, the agent isn't ready to go live.

03Inventory

Every other part of the framework depends on this list. Gartner calls a centralised AI inventory "foundational", and its June 2026 guidance tells CIOs to start by "discovering and inventorying AI across the enterprise".

Look beyond what IT approved

A common mistake is building the inventory from the systems IT signed off. That list misses the tools teams bought on a card, and the AI that arrived inside software you already pay for.

Australia's guidance asks for a register that covers AI "embedded in other systems, like human resources and customer engagement tools". So we suggest starting with each team lead. Then check expense claims and software invoices, and the AI features inside platforms you already pay for. (Yes, the chatbot on your careers page counts.)

2. INVENTORY
   Sources: Gartner · Australia practice 4 · NIST Map
   Every AI system, built, bought or embedded in other software:
   name, purpose, owner, vendor, models, data it reads and writes,
   risk level, budget, approval date, review date.

04Risk levels

A summarising assistant and an agent that can change customer records carry very different risks. The framework should treat them differently.

Size the gate to what the agent can do

Gartner sorts agents into four autonomy levels: agents that observe, agents that advise, agents that act with approval, and agents that act on their own. Its warning is that applying the same governance to all of them leads to failure. It predicts that by 2027, 40% of enterprises will demote or decommission autonomous agents because of governance gaps found only after production incidents.

Forrester puts the principle in one line: "The higher the consequence of the use case, the stronger the foundation needed beneath it." Our template adds controls as the levels rise:

3. RISK LEVELS
   Sources: Gartner autonomy levels · Australia practice 3
   Level 1  Reads data only            access limits, usage logs
   Level 2  Drafts or recommends       + output quality checks
   Level 3  Acts after human approval  + approval records, audit trail
   Level 4  Acts on its own            + spend caps, stop controls,
                                         rollback, named owner

05Approval path

The approval path is how a new AI system gets from request to live, and onto the inventory on the way.

Keep approval fast enough that people use it

A common mistake is making every request pass the heaviest checks. Gartner names the cost: over-restricting simple agents "slows delivery and drives shadow development". When approval is slow, teams build around it, and the inventory falls out of date.

Instead, let the risk level set the depth of review. A Level 1 assistant can pass in days. A Level 4 agent earns the full review.

4. APPROVAL PATH
   Sources: Australia practices 1 and 3 · NIST Map, Measure
   Request > owner named > risk level set > data and privacy check >
   budget set > sign-off by [role] > added to inventory > review date
   Depth of review set by the risk level.

06Enforcement

This is the part where the analysts have moved furthest in 2026.

Turn policies into controls that run

Gartner's June 2026 article is titled "AI Governance Requires More Than Policies". Policies "establish intent", it says, but they can't make AI systems behave as expected in real time. Its February analysis is blunter: "Point-in-time audits are simply not enough."

Forrester reaches the same place from another direction. For agents, it argues, responsible AI has to become a runtime discipline.

We think this is the requirement organisations most often underweight. A policy reviewed once a year can't keep pace with an agent that acts thousands of times a day. So the template asks for controls that live in software and run alongside your AI:

5. ENFORCEMENT
   Sources: Gartner AI TRiSM · Australia practice 5 · NIST Manage
   Controls enforced in software while systems run:
   access, spend caps, usage limits.
   Exceptions logged and reviewed [monthly].

07Human control

"Maintain human control" is the last of Australia's six practices. For agents, it comes down to two things.

Give each agent the least it needs

Forrester's AEGIS framework is built on "least agency": each agent gets only the permissions its task requires. An agent that drafts replies doesn't need permission to send them.

Make sure someone can stop it

For agents that act on their own, Gartner asks for rapid rollback and clear ownership. It also asks for "circuit breakers that halt agent operation on threshold violations". Write down who can pause each agent before it goes live, and make sure that person knows.

6. HUMAN CONTROL
   Sources: Australia practice 6 · Forrester AEGIS
   Least agency: each agent holds only the permissions its task needs.
   A named person can pause or stop any agent.
   Incidents recorded and reviewed.

08Spend

The published frameworks centre on risk and behaviour. Once agents run at scale, we think cost belongs in the same framework.

About 20% of organisations now say AI operating costs, including token costs, constrain their AI use. Meanwhile, 40% of large organisations report scaling AI agents, up from 27% a year earlier. Both figures come from McKinsey's 2026 state of AI survey of 1,719 people.

Cap spend before it happens

A common mistake is governing AI spend through the monthly invoice. Picture an agent that reconciles supplier invoices. It can run thousands of model calls a day and use up a quarter's budget well before the bill arrives.

Instead, give every system a budget, and every agent a cap that holds before money is spent. A spend cap is also the simplest of the circuit breakers Gartner describes. It's easy to set, and easy to explain to a board.

Set caps from real usage

Start with caps based on what a normal month actually costs, then tighten them as real numbers come in. Name the person who approves an increase, so a cap that's too tight gets raised quickly instead of worked around.

7. SPEND
   Sources: McKinsey 2026 · Gartner circuit breakers
   Organisation ceiling: [amount] a month
   Team allocations: [amounts]
   Per-agent caps: [amounts]
   Increases approved by: [role]

09Review and reporting

ISO/IEC 42001 describes an AI management system you keep "continually improving". That means a regular rhythm, and a clear line to the top.

Bring it to the top

In McKinsey's mid-2024 survey, CEO oversight of AI governance was one of the factors most linked to bottom-line impact from generative AI. Yet only 28% of respondents said their CEO held that role, and 17% said their board did.

We suggest a one-page quarterly report with four numbers your board can track over time:

8. REVIEW AND REPORTING
   Sources: NIST Govern · ISO/IEC 42001 continual improvement
   Inventory checked [monthly]
   Board report [quarterly]: inventory coverage, spend against budget,
   open exceptions, approval lead time
   Framework reviewed [annually], or when the rules change

10Fitting it to New Zealand and Australia

Neither country has a dedicated AI Act, so your framework is how you show responsible use under existing law and voluntary guidance.

Our guides to each country cover the local rules in depth, and there's one for Singapore too.

11Getting started

Our framework comes down to eight rules:

• Give every AI system a named owner, with one senior owner above them all.
• Build the inventory from invoices and team leads, as well as IT's list.
• Size each agent's controls to what it can do.
• Keep approval fast for low-risk systems, so people use it.
• Turn policies into controls that run while AI runs.
• Give each agent the least it needs, and someone who can stop it.
• Cap spend before it happens, and set the caps from real usage.
• Report to the board every quarter, on four numbers.

Your first 30 days

You don't need everything perfect on day one. Getting the order right matters more.

1. Week one: name your AI governance owner, then build the inventory.
2. Week two: put an owner and a risk level against every line.
3. Week three: agree the approval path and the spend caps with finance, and name who can stop an agent.
4. Week four: bring a one-page summary to the executive team, then set the board cycle.

The full template

Copy this into your own document and fill in the brackets. Treat it as a starting point, and adapt it to your own systems and your board.

[ORGANISATION] AI GOVERNANCE FRAMEWORK
Version [x.x] · Owner [name, role] · Approved by [board or committee]
Next review [date]
Aligned to: NIST AI RMF 1.0 · ISO/IEC 42001 ·
[Guidance for AI Adoption (AU) or MBIE guidance (NZ)]

1. ACCOUNTABILITY
   Sources: Australia practice 1 · NIST Govern
   AI governance owner: [senior leader, role]
   System owners: one named person for every AI system
   Board oversight: [committee], [frequency]

2. INVENTORY
   Sources: Gartner · Australia practice 4 · NIST Map
   Every AI system, built, bought or embedded in other software:
   name, purpose, owner, vendor, models, data it reads and writes,
   risk level, budget, approval date, review date.

3. RISK LEVELS
   Sources: Gartner autonomy levels · Australia practice 3
   Level 1  Reads data only            access limits, usage logs
   Level 2  Drafts or recommends       + output quality checks
   Level 3  Acts after human approval  + approval records, audit trail
   Level 4  Acts on its own            + spend caps, stop controls,
                                         rollback, named owner

4. APPROVAL PATH
   Sources: Australia practices 1 and 3 · NIST Map, Measure
   Request > owner named > risk level set > data and privacy check >
   budget set > sign-off by [role] > added to inventory > review date
   Depth of review set by the risk level.

5. ENFORCEMENT
   Sources: Gartner AI TRiSM · Australia practice 5 · NIST Manage
   Controls enforced in software while systems run:
   access, spend caps, usage limits.
   Exceptions logged and reviewed [monthly].

6. HUMAN CONTROL
   Sources: Australia practice 6 · Forrester AEGIS
   Least agency: each agent holds only the permissions its task needs.
   A named person can pause or stop any agent.
   Incidents recorded and reviewed.

7. SPEND
   Sources: McKinsey 2026 · Gartner circuit breakers
   Organisation ceiling: [amount] a month
   Team allocations: [amounts]
   Per-agent caps: [amounts]
   Increases approved by: [role]

8. REVIEW AND REPORTING
   Sources: NIST Govern · ISO/IEC 42001 continual improvement
   Inventory checked [monthly]
   Board report [quarterly]: inventory coverage, spend against budget,
   open exceptions, approval lead time
   Framework reviewed [annually], or when the rules change

This framework is general guidance and not legal advice. Check your obligations with your own advisers.