Measure false positives against an agreed task set
A detector can match its configuration correctly while enforcing a policy the business does not want. Agree the expected decision before measuring false positives, and keep policy disagreements separate from detection mistakes and compatibility failures.
For Security operations and pilot policy owners
A synthetic prompt set for a support team
A security lead and support manager prepare fictional prompts representing ordinary work. They label expected outcomes together before running the pilot, then review disagreements using the same definitions.
What you are working with
- A set of permitted prompts covering public product descriptions, fictional summaries and approved business tasks.
- A separate restricted set containing vendor-confirmed synthetic fixtures for the detectors under evaluation.
- An evaluator-maintained label sheet with the expected action, rationale and business owner for each example.
A safer approach
- Use varied synthetic contexts rather than repeating a single obvious sensitive string across every sample.
- Agree which interruptions count as false positives, distinguishing warnings from blocks and redactions.
- Reserve unseen examples for a later check so policy tuning is not judged only on the cases used to tune it.
Expected outcome: The team obtains an interpretable account of unnecessary interventions and missed restrictions, with denominators, disagreements and test boundaries recorded.
Work through the procedure
Label before running
Have the business owner and security reviewer agree what should happen to each sample. Resolve ambiguous cases or mark them as disputed. Without this reference, a blocked prompt may be called a false positive merely because the user wanted to send it.
Run a representative mix
Exercise the agreed samples on a fixed product, policy and provider configuration. Keep benign and restricted cases distinct in the results. Record unsupported inputs and technical failures separately; neither demonstrates that the detector made a correct or incorrect content decision.
Calculate transparent measures
Report unnecessary interventions divided by all permitted cases for a false-positive rate. Report missed restrictions against the restricted set separately. Include counts, severity and sample composition alongside percentages; a single headline number can hide the cost of a disruptive block.
Tune and validate independently
Review the rule behind each disagreement and change one policy variable at a time where possible. Retest the original examples, then evaluate the reserved set. Record both results so improvement on familiar samples is not mistaken for broader reliability.
What to check before proceeding
1. Ground truth
- Ready when
- Each scored sample has an agreed expected action and a documented reason.
- If the check fails
- Move disputed examples out of the metric until policy owners resolve the classification.
2. Measurement clarity
- Ready when
- Counts, denominators and intervention types accompany reported rates.
- If the check fails
- Rebuild the result table before comparing products or declaring a threshold satisfied.
3. Generalization
- Ready when
- A reserved set meets the team's agreed criteria after tuning.
- If the check fails
- Narrow the deployment or expand evaluation instead of repeatedly tuning to the same examples.
Common mistakes to avoid
- Treating every disliked block as a detection mistake when the underlying rule intentionally prohibits that content.
- Claiming real-world accuracy from a tiny synthetic set that contains only easy positive examples.
Evaluate this workflow with Aona
Where Aona can help
Ask Aona to review suitable synthetic fixtures and the policy decisions being evaluated, then use your own agreed scoring sheet.
What to confirm
This protocol makes no accuracy claim for Aona and does not imply every business-specific category has an available classifier.
Evaluating a control for your organization?
Bring your target AI tool, device and acceptance criteria. Review the supported control path, the evidence you need and any limitations before deciding on a pilot.