Select your measures

Item definition and operating envelope documented (task, authority, action space, contexts allowed)

Example: One-page description of task, allowed actions, limits and excluded cases, approved by the process owner.

Least-privilege, scoped and non-transferable agent authority

Example: Agent may create payments only to suppliers in the vendor master, in one company code.

Hard action limits enforced outside the model (value caps, rate limits, allow-lists)

Example: Payment API rejects amounts above a cap or more than 50 payments per hour.

Reversibility by design: prefer reversible actions; staging or undo for irreversible ones

Example: Payments are staged and released after two hours unless stopped.

Oversight mode defined per action class (in / on / out of the loop)

Example: Below €1k autonomous; €1k–10k human approval; above €10k never by the agent.

Approval gate for irreversible or above-threshold actions

Example: Payments above the threshold require approval in the ERP workflow.

Oversight sufficiency test (Sufficient / Nominal / Insufficient / Theatrical), incl. measured error-detection rate of reviewers

Example: Quarterly check of review time per item and reviewer workload against the sufficiency criteria.

Oversight metrics monitored (override rate, response time, outlier reviewers)

Example: Dashboard of override rate and response time per reviewer.

Decision context package for reviewers (reasoning summary, evidence, flags)

Example: Approval screen shows invoice, order match, anomalies and the agent's reasoning summary.

Defined safe state and degradation modes

Example: On anomaly the agent pauses payments and hands the queue to the accounts payable team.

Escalation on uncertainty (calibrated uncertainty signal plus escalation rule; no raw confidence %)

Example: Weak match between invoice and order routes the case to a human with the reasons.

Kill switch: halt and lock autonomous action

Example: One-click stop in the operations console, tested monthly.

Behavioral anomaly and drift detection against a baseline

Example: Alert when daily payment volume deviates strongly from the baseline.

Outcome monitoring beyond the target metric (second-order effects)

Example: Track cash position and supplier complaints, not only the on-time payment rate.

Input and context validity check (data quality, staleness, domain match)

Example: Reject invoices with missing order number, stale exchange rate or unknown currency.

Incident and near-miss reporting process

Example: Incident form and monthly review of all stopped or reversed payments.

Decision logging (inputs, actions, model version, rationale)

Example: Log per payment: inputs, decision, model version and approver.

Log retention and audit access defined

Example: Logs kept for the legal retention period; auditors have read access.

Scenario-based evaluation before release (representative, edge and failure cases)

Example: Test set of historical invoices incl. duplicates and known fraud cases.

Known-unsafe scenarios detected and routed to humans

Example: Invoices in foreign currency always go to a human.

Unknown-unsafe exploration (red-teaming, adversarial edge cases)

Example: Red team tries fake invoices and hidden instructions in invoice text.

Staged deployment (shadow → limited → full)

Example: Shadow mode for one month, then 10% of invoices, then all.

Regression re-test after model, prompt or tool change

Example: Rerun the test set after every model, prompt or tool change.

Untrusted content isolation (external content treated as data; provenance marked)

Example: Invoice text is passed as data, never as instructions.

Tool-call validation and safe output handling (schema checks, sandboxed execution)

Example: Payment calls validated against schema and vendor master before execution.

Secrets and credential isolation (no secrets in context, short-lived tokens)

Example: Short-lived tokens from a vault; no keys in prompts.

Supply-chain vetting of models, tools and agent products

Example: Vendor review incl. update policy and security certification.

Data provenance and quality gate

Example: Vendor master data validated and owned by procurement.

Fundamental-rights screening; formal FRIA where Art. 27 applies

Example: Screening checklist completed; formal FRIA where legally required.

Bias and fairness testing on affected groups

Example: Compare payment delays across supplier size classes.

Required by law (LO04 · EU AI Act Art. 50) — this measure contributes to Transparency: persons are informed that they interact with an AI system.

Transparency to affected persons (AI disclosure, explanation, appeal path)

Example: AI notice in supplier emails; named contact for disputes.

Named accountable owner and responsibility matrix (value chain + three lines of defense)

Example: Head of accounts payable is accountable; responsibilities agreed with IT and compliance.

Re-classification triggers defined and monitored

Example: Re-assessment when model, volume or scope changes, or at least yearly.

Independent safety assessment before go-live

Example: Internal audit reviews the Safety Concept before go-live.

Required by law (LO01 · EU AI Act Art. 4) — this measure contributes to AI literacy of staff dealing with the AI system.

Operator and user training; AI literacy; end-user responsibility

Example: Training for clerks on reviewing agent payments.

Agent interaction map and trust boundaries (freedom from interference)

Example: Payment agent accepts only validated orders from the ordering agent.

Authenticated and validated inter-agent messaging

Example: Signed messages between ordering and payment agent.

Network circuit breakers (aggregate thresholds regardless of contributing agent)

Example: Halt when total commitments across agents exceed a threshold within four hours.

Independence check for decomposed or layered controls (no shared base model, context or memory)

Example: Guardrail uses a rule engine, not the same model as the agent.

Answers and statements only from authoritative, versioned sources; otherwise hand over to a human

Example: Agent answers payment-term questions only from the current supplier terms document; anything else goes to accounts payable.

No commitments, offers or exceptions outside the agent's authority; such requests are routed to a human

Example: Agent may not agree to early-payment discounts or deadline extensions; it forwards such requests.

Lawful-basis review of decision logic and data items before go-live and after rule changes

Example: Legal reviews which supplier attributes the agent may use to prioritize payments.

Data minimization in agent context and outputs

Example: Agent receives invoice fields only; no access to full vendor contracts or employee data.

Output and egress control (no auto-rendered external links or images; egress allow-list)

Example: Agent emails are plain text; outbound connections only to the ERP and the bank API.

Environment separation (sandbox or staging; gated production writes) and tested restore

Example: Agent works on a staging copy of the payment file; release to the bank needs a separate gated step; restore tested quarterly.

Audit by sampling of autonomous decisions

Example: Each week 30 random agent payments are re-checked by a clerk; error rate reported.

Unique agent identity registered in an agent catalog

Example: Agent registered with ID, owner and permissions in the IT asset catalog.

Tamper-evident logs

Example: Write-once log storage with integrity protection.

Minimum development process capability

Example: Version control, documented tests and reviews for prompts and configurations.

Cross-firm interface contract (assume / guarantee safety obligations)

Example: Contract with the payment provider defines validation duties and liability.