How to verify your AI policy controls

An approved AI tool can be used through different accounts, devices and input paths. Your policy may permit one combination and prohibit another. Before extending a rollout, record what should happen and check what your controls actually do in the environment you intend to support.

This guide explains how to use Aona’s six-scenario AI Policy Crash Test. Start with a team discussion, then assign authorised checks where evidence is missing.

Open the free worksheet

No email gate. Use synthetic examples only. The worksheet does not connect to your AI tools or inspect your systems.

Start with an expectation you can test

Choose one policy rule and write down its scope: the AI service, account class, device, application, input type and expected outcome.

For example, your team might expect a personal test account to be blocked in a managed browser while an approved work account remains available. That expectation needs evidence for both routes. Knowing that the AI service is approved does not answer the account question.

Keep tabletop discussion and observed results separate. A discussion identifies what to check. An authorised observation provides evidence for the environment tested.

Work through these six scenarios

1. Same AI tool. Different account.

An employee uses an approved business account, then switches to a personal account in the same browser.

Check whether your controls distinguish the account classes. Separately, check whether the resulting decision can be attributed to the correct user, device and tenant.

Record the permitted or blocked route, account class, policy version and a sanitised event reference. An application name alone does not establish account or tenant attribution.

2. The extension is logged out.

A new browser session opens before the security extension signs in. A private window creates a second route to examine.

Record what happens in each state and whether IT can identify a browser outside the expected protection state. Evidence may include an observed control outcome, posture alert, inventory entry or documented exception.

Installation alone does not establish protection across these browser states.

3. A browser pass is not a desktop pass.

Repeat an approved synthetic request through the desktop AI application. Assess mobile and unmanaged-device routes separately.

Record the operating system, application version, device management state and input path. Check each route against documented coverage and available event evidence.

Keep missing evidence as unknown. Where product documentation explicitly excludes a route, record that exclusion.

4. A sensitive file reaches the boundary.

Use a fictional fixture with clearly labelled fake personal and confidential fields. Discuss pasted text and file uploads separately.

In an authorised environment, check whether the expected allow, redact or block outcome occurs for each input path and file type. Compare a harmless synthetic request with a flagged one to examine the configured logging policy.

Record the outcome, retained metadata and retrieval limits through sanitised references. Keep real customer data, credentials and logs out of the worksheet.

5. A plausible answer is still unverified.

An AI-generated supplier summary contains an unsupported fictional certification claim.

Check whether the employee verifies the claim before external use, and whether consequential output has a named human review step.

Record the source check, reviewer decision and responsible role. Input protection and factual review address different parts of the workflow.

6. An employee leaves. A vendor changes.

A test user is offboarded, an AI vendor changes its terms, and IT needs to remove a security agent from a test device.

Check account revocation, policy ownership and vendor approval after the change. On a separately authorised test device, verify the uninstall result, residual permissions and updated inventory.

Record observations and approval ownership. Closing an offboarding ticket leaves the access-revocation question unresolved unless it contains supporting evidence.

Keep evidence with every result

Use one row for each check. The worksheet contains two checks per scenario.

A Pass means the expected result was observed within the recorded authorised test scope. A Fail means the expected control was not observed. Untested / unknown means evidence is missing. Unsupported requires a documented coverage exclusion.

These statuses describe your recorded checks. They do not form a security score.

Choose the next action from the evidence

If the expected control is observed, keep the evidence and define when to recheck it.

If the outcome differs from the expectation, review configuration, policy and documented coverage before assigning remediation.

If evidence is missing, give the check an owner and a review date. Avoid treating an unanswered question as proof of protection or proof of failure.

Revisit the affected checks after account, browser, application, agent, policy or vendor changes.

What this exercise can establish

An authorised check can document an observed result for a specific environment and input. The exercise helps teams keep that result, its scope and its next action together.

It does not independently verify observations, inspect infrastructure, establish provider retention, certify compliance or guarantee coverage across other routes and inputs. Human output review remains a separate responsibility.

Download the blank test plan