Worked example · Step 7 Get your Safety Concept

Live draft · changes as you edit

AASIF SAFETY CONCEPT

Supplier invoice payment agent

Pay approved supplier invoices on time with less manual effort

Use case	Pay approved supplier invoices on time with less manual effort
Date	2026-10-10
Document version	live draft
Snapshot	—
Flow AI-SIL	AI-SIL 3
Status	draft (specified)

Legend entries mean "contributes to" a framework or standard — never a statement of conformity.

Test version: generated by the AASIF test tool with an uncalibrated Logic Pack. Not for confidential data and not a certification.

4 open points

0 · Executive summary

0.1 Classification at a glance

Classification: Flow AI-SIL 3 · Safety Integrity Level 3
Driving hazard: The level is driven by H06 Manipulated input or goal hijack (prompt injection, poisoned documents or memory): S3 (organization S3, persons S1), E3, C3 → AI-SIL 3, critical even if rare.
Status: draft (specified)
Go-live readiness: No go-live blockers

0.2 Top risks

ID	Hazard	AI-SIL	Worst credible harm
H06	Manipulated input or goal hijack (prompt injection, poisoned documents or memory) · critical even if rare	AI-SIL 3	Major or existential financial loss; major sanction or license risk; severe reputational crisis
H01	Wrong decision or action from incorrect output (hallucination, misreasoning)	AI-SIL 2	Material financial loss for the business unit; customer-visible incident; regulatory finding
H03	Irreversible commitment executed before review (payment, contract, deletion, external message)	AI-SIL 2	Material financial loss for the business unit; customer-visible incident; regulatory finding

0.3 Decisions and sign-offs needed

GA09 Failure mid-action · owner: Engineering / IT
GA10 Error discovered afterwards · owner: Process owner + legal
GA11 Objection by an affected person · owner: Legal / compliance
GA15 Switch-off and fallback · owner: Process owner
C08 · Flow AI-SIL ≥ 3 → expert review of the classification recommended (independence per T5) · independence I2 (independent of the responsible team)

0.4 Commitments before go-live

Required measures (++): 46
Of which in Phase 0: 46
Recommended measures added (+): 4
Balance over the top hazards: 12 prevent · 6 detect & recover

ID	Top measures
M41	Authenticated and validated inter-agent messaging
M49	Output and egress control (no auto-rendered external links or images; egress allow-list)
M13	Kill switch: halt and lock autonomous action
M18	Decision logging (inputs, actions, model version, rationale)
M26	Untrusted content isolation (external content treated as data; provenance marked)

0.5 Legal obligations and regulatory flags

ID	Obligation	Required measures
LO01	Art. 4 · AI literacy of staff dealing with the AI system	M39
LO04	Art. 50 · Transparency: persons are informed that they interact with an AI system	M34

T02 · Do decisions affect persons outside the organization (customers, applicants, citizens)? · yes
T13 · Does the agent communicate directly with humans (chat, email, voice)? · yes

Contributes to: EU AI Act Art. 10, Art. 10, 15, Art. 10, 26, 27, Art. 12, 15, Art. 12, 19, 26, Art. 13, 14 …; ISO/IEC 42001 A.10, A.3, A.4, A.5, A.5, A.7, A.6 …; NIST AI RMF GOVERN, MANAGE, MAP, MEASURE; Singapore MGF D1, D2, D3, D4 (see 5.1).

0.6 Review and approval

Activity	Required independence
Review of the AI-SIL classification	I2 · independent of the responsible team
Review of the AASIF Safety Concept	I2 · independent of the responsible team

C08 · Flow AI-SIL ≥ 3 → expert review of the classification recommended (independence per T5)

Sign-off	Name and role	Date	Signature
Accountable owner
Reviewer

Bow-tie view of top hazards · 3.1

PreventM26Untrusted content isolation (external content treated as data; provenance marked)M23Unknown-unsafe exploration (red-teaming, adversarial edge cases)M41Authenticated and validated inter-agent messagingM49Output and egress control (no auto-rendered external links or images; egress allow-list)

H06Manipulated input or goal hijack (prompt injection, poisoned documents or memory)AI-SIL 3S3 · critical even if rare

Detect & recoverM14Behavioral anomaly and drift detection against a baselineM18Decision logging (inputs, actions, model version, rationale)M13Kill switch: halt and lock autonomous action

PreventM16Input and context validity check (data quality, staleness, domain match)M21Scenario-based evaluation before release (representative, edge and failure cases)M45Answers and statements only from authoritative, versioned sources; otherwise hand over to a humanM31Data provenance and quality gateM22Known-unsafe scenarios detected and routed to humans

H01Wrong decision or action from incorrect output (hallucination, misreasoning)AI-SIL 2

Detect & recoverM12Escalation on uncertainty (calibrated uncertainty signal plus escalation rule; no raw confidence %)M14Behavioral anomaly and drift detection against a baselineM11Defined safe state and degradation modesM18Decision logging (inputs, actions, model version, rationale)

PreventM05Reversibility by design: prefer reversible actions; staging or undo for irreversible onesM07Approval gate for irreversible or above-threshold actionsM50Environment separation (sandbox or staging; gated production writes) and tested restore

H03Irreversible commitment executed before review (payment, contract, deletion, external message)AI-SIL 2

Detect & recoverM10Decision context package for reviewers (reasoning summary, evidence, flags)M11Defined safe state and degradation modesM13Kill switch: halt and lock autonomous action

1 · Use case and scope

1.1 Purpose and scope

What should the agent achieve for the business?: Pay approved supplier invoices on time with less manual effort
Where does it sit in the business process? What happens before and after?: After invoice receipt and order matching in the ERP; before the daily bank payment run
What must the agent never do? Which cases are excluded?: Never change bank details; no payments to suppliers not in the vendor master; no splitting of invoices to stay below limits

1.2 Item definition and agent actions

How autonomous is the agent?: acts and informs
Who is affected by its outcomes?: internal teams, suppliers
Organization context: private · Manufacturing, about 5,000 employees, private company
Which data does the agent use?: confidential, untrusted external · Supplier invoices received by email (external), vendor master, bank details
Which systems and tools can the agent use?: ERP read/write, payment API, email
Does it work with other agents or other organizations' systems?: yes · Ordering agent (internal) sends purchase orders; House bank payment API (external system)
How do humans oversee it today?: Accounts payable clerks review exceptions and all payments above €10k · 40 per hour
How many decisions does the agent make?: 800 per day
Which model or product powers it, and who controls updates?: vendor-managed · Vendor LLM, updates by the vendor
Does the agent's output inform decisions about people, money or regulated controls, even if a person makes the final call?: yes
Where is the agent used? (jurisdictions): EU / EEA

ID	Action (verb + object)	What does the action change in the world?	Can the effect be undone?	Typical and maximum size of the effect	How often does the agent take this action?	Does a human check before the effect happens?	Which inputs does the action rely on?	Is the maximum effect of this action material for the organization?	Does the result reach people outside the organization, now or later (e.g. in a client deliverable, a publication or a letter)?	If a person checks before the effect: does this check meet all four criteria of a qualified human gate?
ACT-01	Matches invoice to purchase order	internal record	fully and instantly	One invoice record; typical €2k, maximum €250k	many times per day or continuous	on exceptions only	Invoice PDF from supplier email (untrusted), purchase order from the ordering agent	material	no	–
ACT-02	Pays supplier invoice	money movement	with effort or cost	Typical €2k, maximum €250k	many times per day or continuous	on exceptions only	Matched invoice, vendor master bank details	material	yes	–
ACT-03	Sends payment advice email to supplier	external message	not at all	One supplier contact per payment	many times per day or continuous	never	Payment record, supplier contact data	immaterial	yes	–

1.3 Roles and accountability

Responsibility	Role or group	Owns
Accountable role (GA13)	Head of accounts payable is accountable; deputy: AP team lead	Agent decisions · M35 Named accountable owner and responsibility matrix (value chain + three lines of defense)
Expert group	Process owner	Scope & authority · Human oversight · Monitoring · open point GA15
Expert group	IT architecture	Scope & authority · Multi-agent
Expert group	HR	Human oversight
Expert group	works council (DACH)	Human oversight
Expert group	Engineering / IT operations	Safe state · Monitoring
Expert group	Engineering	Logging
Expert group	internal audit	Logging
Expert group	Engineering / QA	Validation
Expert group	CISO	Security-for-safety · Multi-agent
Expert group	Data protection officer	Data & rights
Expert group	legal / compliance	Data & rights · open point GA11
Expert group	Risk / compliance	Governance
Expert group	management	Governance
Expert group	Engineering / IT	open point GA09
Expert group	Process owner + legal	open point GA10

2 · Risk assessment

2.1 Grey areas and decisions

ID	Situation	Scope	Design decision	Owner
GA01	Missing data	whole flow	Stop and escalate to accounts payable; never guess missing order numbers	Process owner
GA02	Conflicting information	whole flow	Escalate when invoice and order amounts differ by more than 2%	Process owner
GA03	Limit reached	whole flow	Block and escalate; never split an invoice to stay below the €10k limit	Process owner + risk
GA04	Uncertainty	whole flow	Escalate with context; never proceed with a flag for payments	Process owner
GA05	Out-of-scope request	whole flow	Refuse, log and route to a human	Process owner + CISO
GA06	First-time or unusual case	whole flow	First invoice from a new country goes to a human	Process owner
GA07	Human unavailable	whole flow	Safe default: do not pay; notify the deputy approver	Process owner
GA08	Suspected manipulation	whole flow	Freeze the payment and verify bank details by phone through a known contact; alert security	CISO + process owner
GA12	Changed rules or context	whole flow	Head of accounts payable owns rule updates; regulation changes trigger re-classification	Process owner + compliance
GA13	Accountability	whole flow	Head of accounts payable is accountable; deputy: AP team lead	Management
GA14	Data boundaries	whole flow	Never use or store employee bank details; supplier bank details only from the vendor master	Data protection officer

4 open points — see 4.1.

2.2 Hazards and ratings

ID	Hazard	Source	S org	S person	E	C	AI-SIL	S3	Reasons	Overrides	Note
H01	Wrong decision or action from incorrect output (hallucination, misreasoning)	guide_word G02	S2	S1	E4	C2	AI-SIL 2	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Matches invoice to purchase order' runs many times per day; for 'Wrong output' the hazard situation usually occurs about as often as that (E4). · RH05: Starting point: in the 60 reference cases the harm of 'Wrong output' could typically be corrected only with effort or delay (C2). Confirm it for your case.	–	–
H02	Action beyond authorized scope (wrong tool, wrong amount, wrong recipient)	trigger T07	S2	S1	E3	C2	AI-SIL 1	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Pays supplier invoice' runs many times per day; for 'Beyond authority' the hazard situation usually occurs somewhat less often (E3). · RH05: Starting point: in the 60 reference cases the harm of 'Beyond authority' could typically be corrected only with effort or delay (C2). Confirm it for your case.	–	–
H03	Irreversible commitment executed before review (payment, contract, deletion, external message)	trigger T01	S2	S1	E4	C2	AI-SIL 2	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Pays supplier invoice' runs many times per day; for 'Irreversible before review' the hazard situation usually occurs about as often as that (E4). · RH05: Starting point: in the 60 reference cases the harm of 'Irreversible before review' typically could not be corrected (C3). Confirm it for your case.	–	–
H04	Correct behavior causes harm: misspecified objective or shortcut (reward hacking)	trigger T14 (unsure)	S2	S1	E3	C1	AI-SIL 0	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Matches invoice to purchase order' runs many times per day; for 'Right but harmful' the hazard situation usually occurs about as often as that (E4). · RH05: Starting point: in the 60 reference cases the harm of 'Right but harmful' could typically be corrected only with effort or delay (C2). Confirm it for your case. · RH22: Once step 'Sends payment advice email to supplier' has taken effect, it cannot be undone. Ask whether the harm can still be corrected before it causes real damage.	–	–
H05	Out-of-envelope context (new domain, market, population or data shift)	guide_word G11	S2	S1	E2	C2	AI-SIL 0	–	RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Matches invoice to purchase order' runs many times per day; for 'Outside the design envelope' the hazard situation usually occurs somewhat less often (E3). · RH05: Starting point: in the 60 reference cases the harm of 'Outside the design envelope' could typically be corrected only with effort or delay (C2). Confirm it for your case.	–	–
H06	Manipulated input or goal hijack (prompt injection, poisoned documents or memory)	trigger T06	S3	S1	E3	C3	AI-SIL 3	critical even if rare	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Matches invoice to purchase order' runs many times per day; for 'Manipulated input' the hazard situation usually occurs somewhat less often (E3). · RH05: Starting point: in the 60 reference cases the harm of 'Manipulated input' could typically be corrected only with effort or delay (C2). Confirm it for your case.	–	–
H08	Unfair or discriminatory outcome for affected persons	trigger T02	S1	S2	E3	C2	AI-SIL 1	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Pays supplier invoice' runs many times per day; for 'Unfair outcome' the hazard situation usually occurs somewhat less often (E3). · RH05: Starting point: in the 60 reference cases the harm of 'Unfair outcome' could typically be corrected only with effort or delay (C2). Confirm it for your case.	–	–
H09	Cascading or emergent failure across agents	trigger T09	S2	S1	E4	C2	AI-SIL 2	–	RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Matches invoice to purchase order' runs many times per day; for 'Cascade' the hazard situation usually occurs somewhat less often (E3). · RH05: Starting point: in the 60 reference cases the harm of 'Cascade' could typically be corrected only with effort or delay (C2). Confirm it for your case.	–	–
H10	Ineffective human oversight (automation bias, rubber-stamping)	guide_word G14	S2	S1	E3	C2	AI-SIL 1	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Pays supplier invoice' runs many times per day; for 'Ineffective oversight' the hazard situation usually occurs about as often as that (E4). · RH05: Starting point: in the 60 reference cases the harm of 'Ineffective oversight' could typically be corrected only with effort or delay (C2). Confirm it for your case.	–	–
H12	Untraceable decision (no evidence for audit or incident analysis)	guide_word G13	S2	S1	E2	C3	AI-SIL 1	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Matches invoice to purchase order' runs many times per day; for 'Untraceable decision' the hazard situation usually occurs about as often as that (E4). · RH05: Starting point: in the 60 reference cases the harm of 'Untraceable decision' could typically be corrected only with effort or delay (C2). Confirm it for your case. · RH22: Once step 'Sends payment advice email to supplier' has taken effect, it cannot be undone. Ask whether the harm can still be corrected before it causes real damage.	–	–
H13	Runaway consumption or loop (cost, rate, resources)	guide_word G07	S2	S1	E4	C2	AI-SIL 2	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Pays supplier invoice' runs many times per day; for 'Runaway loop' the hazard situation usually occurs somewhat less often (E3). · RH05: Starting point: in the 60 reference cases the harm of 'Runaway loop' could typically be corrected at once (C1). Confirm it for your case.	–	–
H14	Third-party component failure or compromise (model, tool, plugin, agent product)	trigger T11	S2	S1	E2	C2	AI-SIL 0	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Matches invoice to purchase order' runs many times per day; for 'Third-party failure' the hazard situation usually occurs much less often (E2). · RH05: Starting point: in the 60 reference cases the harm of 'Third-party failure' could typically be corrected only with effort or delay (C2). Confirm it for your case. · RH22: Once step 'Sends payment advice email to supplier' has taken effect, it cannot be undone. Ask whether the harm can still be corrected before it causes real damage.	–	–
H15	Misleading communication to humans (undisclosed AI, overtrust, impersonation)	trigger T13	S1	S1	E4	C1	AI-SIL 0	–	RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Sends payment advice email to supplier' runs many times per day; for 'Misleading communication' the hazard situation usually occurs about as often as that (E4). · RH05: Starting point: in the 60 reference cases the harm of 'Misleading communication' could typically be corrected only with effort or delay (C2). Confirm it for your case. · RH22: Once step 'Sends payment advice email to supplier' has taken effect, it cannot be undone. Ask whether the harm can still be corrected before it causes real damage.	–	–
H16	Required action not performed (silent omission)	guide_word G01	S2	S1	E3	C1	AI-SIL 0	–	RH12: Step 'Pays supplier invoice' moves money and its maximum size is material (A08): at least S2 for the organization or for the affected persons (R-S-FLOOR). · RH14: The agent decides about, or informs decisions about, suppliers: consider at least S2 for them (73% of such hazards in the 60 cases were rated S2 or higher). · RH21: Step 'Pays supplier invoice' runs many times per day; for 'Silent omission' the hazard situation usually occurs somewhat less often (E3). · RH05: Starting point: in the 60 reference cases the harm of 'Silent omission' could typically be corrected at once (C1). Confirm it for your case.	–	–

Flow AI-SIL (highest): AI-SIL 3

Hazards marked not relevant

ID	Hazard	Reason
H07	Sensitive data disclosure or privacy breach	Personal data limited to sole-trader bank details from the vendor master; covered by GA14
H11	Silent degradation after model, prompt or context change	Covered by H14 (vendor-managed model) for this flow

2.3 Rights, regulatory flags and legal obligations

ID	Question	Answer	Flag
T02	Do decisions affect persons outside the organization (customers, applicants, citizens)?	yes	Runs fundamental-rights screening
T03	Is the deployer a public body, a provider of public services or using AI for credit scoring or life/health-insurance pricing?	no	Flags a legally mandatory FRIA (EU AI Act Art. 27)
T04	Is the use case possibly in an EU AI Act Annex III area (e.g. employment, credit, education, essential services)?	no	Flags high-risk obligations; recommends expert review
T13	Does the agent communicate directly with humans (chat, email, voice)?	yes	AI disclosure (EU AI Act Art. 50)

Legal obligations (t18) · the listed measures contribute to these obligations

ID	Regime	Reference	Obligation	Measures (contribute to)
LO01	EU AI Act	Art. 4	AI literacy of staff dealing with the AI system	M39
LO04	EU AI Act	Art. 50	Transparency: persons are informed that they interact with an AI system	M34

3 · Measures and verification

3.1 Bow-tie view of top hazards

H06Manipulated input or goal hijack (prompt injection, poisoned documents or memory)AI-SIL 3S3 · critical even if rare

Detect & recoverM14Behavioral anomaly and drift detection against a baselineM18Decision logging (inputs, actions, model version, rationale)M13Kill switch: halt and lock autonomous action

H01Wrong decision or action from incorrect output (hallucination, misreasoning)AI-SIL 2

H03Irreversible commitment executed before review (payment, contract, deletion, external message)AI-SIL 2

Detect & recoverM10Decision context package for reviewers (reasoning summary, evidence, flags)M11Defined safe state and degradation modesM13Kill switch: halt and lock autonomous action

3.2 Measures by phase

Phase 0 · Before go-live

All ++ measures (prevent and detect & recover) · all decided grey areas implemented · undecided grey areas linked to S3 hazards resolved

ID	Measure	Grade	Barrier	Control type	How you'll do it	Verification	Reviewed by	Status
M01	Item definition and operating envelope documented (task, authority, action space, contexts allowed)	++	Prevent	Process	–	V1 Design review	–	specified
M02	Least-privilege, scoped and non-transferable agent authority	++	Prevent	Structural	–	V1 Design review; V2 Control audit	–	specified
M04	Hard action limits enforced outside the model (value caps, rate limits, allow-lists)	++	Prevent	Rule-based	–	V1 Design review; V3 Scenario-based evaluation (evals)	–	specified
M05	Reversibility by design: prefer reversible actions; staging or undo for irreversible ones	++	Prevent	Structural	–	V1 Design review; V3 Scenario-based evaluation (evals)	–	specified
M06	Oversight mode defined per action class (in / on / out of the loop)	++	Prevent	Process	–	V1 Design review	–	specified
M07	Approval gate for irreversible or above-threshold actions	++	Prevent	Structural	–	V1 Design review; V3 Scenario-based evaluation (evals)	–	specified
M08	Oversight sufficiency test (Sufficient / Nominal / Insufficient / Theatrical), incl. measured error-detection rate of reviewers	++	Detect & recover	Process	–	V2 Control audit; V5 Runtime monitoring and log review	–	specified
M09	Oversight metrics monitored (override rate, response time, outlier reviewers)	++	Detect & recover	Process	–	V5 Runtime monitoring and log review	–	specified
M10	Decision context package for reviewers (reasoning summary, evidence, flags)	++	Detect & recover	Model-based	–	V3 Scenario-based evaluation (evals)	–	specified
M11	Defined safe state and degradation modes	++	Detect & recover	Structural	–	V1 Design review; V3 Scenario-based evaluation (evals)	–	specified
M12	Escalation on uncertainty (calibrated uncertainty signal plus escalation rule; no raw confidence %)	++	Detect & recover	Model-based	–	V3 Scenario-based evaluation (evals)	–	specified
M13	Kill switch: halt and lock autonomous action	++	Detect & recover	Structural	–	V3 Scenario-based evaluation (evals)	–	specified
M14	Behavioral anomaly and drift detection against a baseline	++	Detect & recover	Model-based	–	V5 Runtime monitoring and log review	–	specified
M15	Outcome monitoring beyond the target metric (second-order effects)	++	Detect & recover	Process	–	V5 Runtime monitoring and log review	–	specified
M16	Input and context validity check (data quality, staleness, domain match)	++	Prevent	Rule-based	–	V3 Scenario-based evaluation (evals)	–	specified
M17	Incident and near-miss reporting process	++	Detect & recover	Process	–	V2 Control audit	–	specified
M18	Decision logging (inputs, actions, model version, rationale)	++	Detect & recover	Structural	–	V2 Control audit; V5 Runtime monitoring and log review	–	specified
M20	Log retention and audit access defined	++	Detect & recover	Governance	–	V2 Control audit	–	specified
M21	Scenario-based evaluation before release (representative, edge and failure cases)	++	Prevent	Process	–	V3 Scenario-based evaluation (evals)	–	specified
M22	Known-unsafe scenarios detected and routed to humans	++	Prevent	Rule-based	–	V3 Scenario-based evaluation (evals)	–	specified
M23	Unknown-unsafe exploration (red-teaming, adversarial edge cases)	++	Prevent	Process	–	V4 Adversarial testing (red-teaming)	–	specified
M24	Staged deployment (shadow → limited → full)	++	Prevent	Process	–	V2 Control audit	–	specified
M25	Regression re-test after model, prompt or tool change	++	Detect & recover	Process	–	V6 Periodic re-testing	–	specified
M26	Untrusted content isolation (external content treated as data; provenance marked)	++	Prevent	Prompt-layer / Structural	–	V4 Adversarial testing (red-teaming)	–	specified
M27	Tool-call validation and safe output handling (schema checks, sandboxed execution)	++	Prevent	Rule-based	–	V3 Scenario-based evaluation (evals); V4 Adversarial testing (red-teaming)	–	specified
M29	Secrets and credential isolation (no secrets in context, short-lived tokens)	++	Prevent	Structural	–	V2 Control audit	–	specified
M30	Supply-chain vetting of models, tools and agent products	++	Prevent	Governance	–	V2 Control audit; V7 Independent assessment / certification	–	specified
M31	Data provenance and quality gate	++	Prevent	Process	–	V2 Control audit	–	specified
M32	Fundamental-rights screening; formal FRIA where Art. 27 applies	++	Prevent	Process	–	V2 Control audit	–	specified
M33	Bias and fairness testing on affected groups	++	Prevent	Process	–	V3 Scenario-based evaluation (evals)	–	specified
M34	Transparency to affected persons (AI disclosure, explanation, appeal path)	++	Both	Process	–	V2 Control audit	–	specified
M35	Named accountable owner and responsibility matrix (value chain + three lines of defense)	++	Prevent	Governance	–	V2 Control audit	–	specified
M36	Re-classification triggers defined and monitored	++	Detect & recover	Process	–	V2 Control audit; V6 Periodic re-testing	–	specified
M38	Independent safety assessment before go-live	++	Prevent	Process	–	V7 Independent assessment / certification	–	specified
M39	Operator and user training; AI literacy; end-user responsibility	++	Prevent	Process	–	V2 Control audit	–	specified
M40	Agent interaction map and trust boundaries (freedom from interference)	++	Prevent	Structural	–	V1 Design review	–	specified
M41	Authenticated and validated inter-agent messaging	++	Prevent	Structural	–	V4 Adversarial testing (red-teaming)	–	specified
M42	Network circuit breakers (aggregate thresholds regardless of contributing agent)	++	Detect & recover	Rule-based	–	V3 Scenario-based evaluation (evals)	–	specified
M43	Independence check for decomposed or layered controls (no shared base model, context or memory)	++	Prevent	Process	–	V1 Design review; V7 Independent assessment / certification	–	specified
M45	Answers and statements only from authoritative, versioned sources; otherwise hand over to a human	++	Prevent	Rule-based	–	V3 Scenario-based evaluation (evals)	–	specified
M46	No commitments, offers or exceptions outside the agent's authority; such requests are routed to a human	++	Prevent	Rule-based	–	V1 Design review; V3 Scenario-based evaluation (evals)	–	specified
M47	Lawful-basis review of decision logic and data items before go-live and after rule changes	++	Prevent	Process	–	V2 Control audit	–	specified
M48	Data minimization in agent context and outputs	++	Prevent	Structural	–	V2 Control audit	–	specified
M49	Output and egress control (no auto-rendered external links or images; egress allow-list)	++	Prevent	Structural	–	V1 Design review; V4 Adversarial testing (red-teaming)	–	specified
M50	Environment separation (sandbox or staging; gated production writes) and tested restore	++	Prevent	Structural	–	V1 Design review; V3 Scenario-based evaluation (evals)	–	specified
M51	Audit by sampling of autonomous decisions	++	Detect & recover	Process	–	V5 Runtime monitoring and log review	–	specified

Phase 1 · Within 3 months

All + measures · open points without S3 link

ID	Measure	Grade	Barrier	Control type	How you'll do it	Verification	Reviewed by	Status
M03	Unique agent identity registered in an agent catalog	+	Prevent	Structural	–	V2 Control audit	–	specified
M19	Tamper-evident logs	+	Detect & recover	Structural	–	V2 Control audit	–	specified
M37	Minimum development process capability	+	Prevent	Process	–	V2 Control audit	–	specified
M44	Cross-firm interface contract (assume / guarantee safety obligations)	+	Prevent	Governance	–	V2 Control audit	–	specified

Phase 2 · Continuous

Runtime monitoring (M14, M15), oversight metrics (M09), re-testing (M25), re-classification (M36), periodic audits (T5)

None.

3.3 Deviations

No deviations.

4 · Open points and lifecycle

4.1 Open points by expert group

Engineering / IT

ID	Open point	Linked hazards	Go-live blocker
GA09	Failure mid-action	H16, H01	no

Process owner + legal

ID	Open point	Linked hazards	Go-live blocker
GA10	Error discovered afterwards	H03, H12	no

Legal / compliance

ID	Open point	Linked hazards	Go-live blocker
GA11	Objection by an affected person	H08, H15	no

Process owner

ID	Open point	Linked hazards	Go-live blocker
GA15	Switch-off and fallback	H16, H11	no

Expert review

ID	Open point	Linked hazards	Go-live blocker
T14	Marked unsure	H04	no

4.2 Re-classification triggers

M36: Re-classification triggers defined and monitored
Can the model or provider change outside the deployer's control?: yes

The classification must follow changes in use and technology.

5 · Compliance legend

5.1 Contribution table

Legend entries mean "contributes to" a framework or standard — never a statement of conformity. Status: indicative, to verify.

ID	Measure	Contributes to: EU AI Act	ISO/IEC 42001	NIST AI RMF	Singapore MGF	Verification schemes
M01	Item definition and operating envelope documented (task, authority, action space, contexts allowed)	Art. 9, 11	Cl. 6, 8; A.6	MAP	D1 Assess and bound risks upfront	Design review
M02	Least-privilege, scoped and non-transferable agent authority	Art. 15, 26	A.6, A.9	MANAGE	D1 Assess and bound risks upfront; D3 Implement technical controls and processes	ISO 27001, AIUC-1
M03	Unique agent identity registered in an agent catalog	Art. 26	A.6	GOVERN	D1 Assess and bound risks upfront	–
M04	Hard action limits enforced outside the model (value caps, rate limits, allow-lists)	Art. 9, 15	A.6	MANAGE	D3 Implement technical controls and processes	AIUC-1
M05	Reversibility by design: prefer reversible actions; staging or undo for irreversible ones	Art. 9, 14	A.6	MANAGE	D3 Implement technical controls and processes	–
M06	Oversight mode defined per action class (in / on / out of the loop)	Art. 14, 26	A.9	GOVERN	D2 Make humans meaningfully accountable	–
M07	Approval gate for irreversible or above-threshold actions	Art. 14	A.9	MANAGE	D2 Make humans meaningfully accountable; D3 Implement technical controls and processes	–
M08	Oversight sufficiency test (Sufficient / Nominal / Insufficient / Theatrical), incl. measured error-detection rate of reviewers	Art. 14	A.9	MEASURE	D2 Make humans meaningfully accountable	–
M09	Oversight metrics monitored (override rate, response time, outlier reviewers)	Art. 14, 26	A.9	MEASURE	D2 Make humans meaningfully accountable	–
M10	Decision context package for reviewers (reasoning summary, evidence, flags)	Art. 13, 14	A.8, A.9	MANAGE	D2 Make humans meaningfully accountable	–
M11	Defined safe state and degradation modes	Art. 9, 15	A.6	MANAGE	D3 Implement technical controls and processes	–
M12	Escalation on uncertainty (calibrated uncertainty signal plus escalation rule; no raw confidence %)	Art. 14, 15	A.6	MANAGE	D3 Implement technical controls and processes	–
M13	Kill switch: halt and lock autonomous action	Art. 14	A.6, A.9	MANAGE	D3 Implement technical controls and processes	–
M14	Behavioral anomaly and drift detection against a baseline	Art. 15, 72	Cl. 9; A.6	MEASURE	D3 Implement technical controls and processes	–
M15	Outcome monitoring beyond the target metric (second-order effects)	Art. 9, 72	Cl. 9	MEASURE	D3 Implement technical controls and processes	–
M16	Input and context validity check (data quality, staleness, domain match)	Art. 10, 15	A.7	MEASURE	D3 Implement technical controls and processes	–
M17	Incident and near-miss reporting process	Art. 73	Cl. 10	MANAGE	D3 Implement technical controls and processes	–
M18	Decision logging (inputs, actions, model version, rationale)	Art. 12, 19, 26	A.6	MEASURE	D3 Implement technical controls and processes	AIUC-1
M19	Tamper-evident logs	Art. 12, 15	A.6	MANAGE	D3 Implement technical controls and processes	ISO 27001
M20	Log retention and audit access defined	Art. 19, 26	Cl. 7.5	GOVERN	D3 Implement technical controls and processes	ISO 27001
M21	Scenario-based evaluation before release (representative, edge and failure cases)	Art. 9, 15	A.6	MEASURE	D3 Implement technical controls and processes	AIUC-1
M22	Known-unsafe scenarios detected and routed to humans	Art. 9	A.6	MEASURE	D3 Implement technical controls and processes	–
M23	Unknown-unsafe exploration (red-teaming, adversarial edge cases)	Art. 9, 15	A.6	MEASURE	D3 Implement technical controls and processes	AIUC-1
M24	Staged deployment (shadow → limited → full)	Art. 9	A.6	MANAGE	D1 Assess and bound risks upfront; D3 Implement technical controls and processes	–
M25	Regression re-test after model, prompt or tool change	Art. 9, 15	A.6	MEASURE	D3 Implement technical controls and processes	AIUC-1
M26	Untrusted content isolation (external content treated as data; provenance marked)	Art. 15	A.6	MANAGE	D3 Implement technical controls and processes	AIUC-1, ISO 27001
M27	Tool-call validation and safe output handling (schema checks, sandboxed execution)	Art. 15	A.6	MANAGE	D3 Implement technical controls and processes	AIUC-1
M29	Secrets and credential isolation (no secrets in context, short-lived tokens)	Art. 15	A.6	MANAGE	D3 Implement technical controls and processes	ISO 27001, SOC 2
M30	Supply-chain vetting of models, tools and agent products	Art. 25, 26	A.10	GOVERN	D1 Assess and bound risks upfront	AIUC-1, ISO 27001
M31	Data provenance and quality gate	Art. 10	A.7	MAP	D1 Assess and bound risks upfront	–
M32	Fundamental-rights screening; formal FRIA where Art. 27 applies	Art. 27	A.5	MAP	D1 Assess and bound risks upfront	–
M33	Bias and fairness testing on affected groups	Art. 10	A.5, A.7	MEASURE	D1 Assess and bound risks upfront; D3 Implement technical controls and processes	–
M34	Transparency to affected persons (AI disclosure, explanation, appeal path)	Art. 26, 50, 86	A.8	GOVERN	D4 Enable end-user responsibility	–
M35	Named accountable owner and responsibility matrix (value chain + three lines of defense)	Art. 17, 26	Cl. 5; A.3	GOVERN	D2 Make humans meaningfully accountable	ISO 42001
M36	Re-classification triggers defined and monitored	Art. 9, 72	Cl. 9, 10	MANAGE	D1 Assess and bound risks upfront	–
M37	Minimum development process capability	Art. 17	Cl. 8; A.6	GOVERN	D3 Implement technical controls and processes	ISO 42001
M38	Independent safety assessment before go-live	Art. 43	Cl. 9	MEASURE	D2 Make humans meaningfully accountable	Art. 43 conformity assessment, AIUC-1
M39	Operator and user training; AI literacy; end-user responsibility	Art. 4, 26	Cl. 7.2; A.4	GOVERN	D4 Enable end-user responsibility	–
M40	Agent interaction map and trust boundaries (freedom from interference)	Art. 15	A.6	MAP	D3 Implement technical controls and processes	–
M41	Authenticated and validated inter-agent messaging	Art. 15	A.6	MANAGE	D3 Implement technical controls and processes	AIUC-1
M42	Network circuit breakers (aggregate thresholds regardless of contributing agent)	Art. 9, 15	A.6	MANAGE	D3 Implement technical controls and processes	–
M43	Independence check for decomposed or layered controls (no shared base model, context or memory)	Art. 15	A.6	MEASURE	D3 Implement technical controls and processes	–
M44	Cross-firm interface contract (assume / guarantee safety obligations)	Art. 25	A.10	GOVERN	D1 Assess and bound risks upfront	–
M45	Answers and statements only from authoritative, versioned sources; otherwise hand over to a human	Art. 13, 15	A.6, A.8	MANAGE	D3 Implement technical controls and processes	–
M46	No commitments, offers or exceptions outside the agent's authority; such requests are routed to a human	Art. 14, 26	A.6	GOVERN	D1 Assess and bound risks upfront; D3 Implement technical controls and processes	–
M47	Lawful-basis review of decision logic and data items before go-live and after rule changes	Art. 10, 26, 27	A.5, A.7	MAP	D1 Assess and bound risks upfront	–
M48	Data minimization in agent context and outputs	Art. 10	A.7	MANAGE	D3 Implement technical controls and processes	–
M49	Output and egress control (no auto-rendered external links or images; egress allow-list)	Art. 15	A.6	MANAGE	D3 Implement technical controls and processes	–
M50	Environment separation (sandbox or staging; gated production writes) and tested restore	Art. 9, 15	A.6	MANAGE	D3 Implement technical controls and processes	–
M51	Audit by sampling of autonomous decisions	Art. 14, 26, 72	A.9; Cl. 9	MEASURE	D2 Make humans meaningfully accountable; D3 Implement technical controls and processes	–

Annex A · Implementation binding

To be filled by engineering: how each requirement is implemented (tool, configuration, process).

ID	Requirement	Implemented by (tool, configuration, process)
M01	Item definition and operating envelope documented (task, authority, action space, contexts allowed)
M02	Least-privilege, scoped and non-transferable agent authority
M03	Unique agent identity registered in an agent catalog
M04	Hard action limits enforced outside the model (value caps, rate limits, allow-lists)
M05	Reversibility by design: prefer reversible actions; staging or undo for irreversible ones
M06	Oversight mode defined per action class (in / on / out of the loop)
M07	Approval gate for irreversible or above-threshold actions
M08	Oversight sufficiency test (Sufficient / Nominal / Insufficient / Theatrical), incl. measured error-detection rate of reviewers
M09	Oversight metrics monitored (override rate, response time, outlier reviewers)
M10	Decision context package for reviewers (reasoning summary, evidence, flags)
M11	Defined safe state and degradation modes
M12	Escalation on uncertainty (calibrated uncertainty signal plus escalation rule; no raw confidence %)
M13	Kill switch: halt and lock autonomous action
M14	Behavioral anomaly and drift detection against a baseline
M15	Outcome monitoring beyond the target metric (second-order effects)
M16	Input and context validity check (data quality, staleness, domain match)
M17	Incident and near-miss reporting process
M18	Decision logging (inputs, actions, model version, rationale)
M19	Tamper-evident logs
M20	Log retention and audit access defined
M21	Scenario-based evaluation before release (representative, edge and failure cases)
M22	Known-unsafe scenarios detected and routed to humans
M23	Unknown-unsafe exploration (red-teaming, adversarial edge cases)
M24	Staged deployment (shadow → limited → full)
M25	Regression re-test after model, prompt or tool change
M26	Untrusted content isolation (external content treated as data; provenance marked)
M27	Tool-call validation and safe output handling (schema checks, sandboxed execution)
M29	Secrets and credential isolation (no secrets in context, short-lived tokens)
M30	Supply-chain vetting of models, tools and agent products
M31	Data provenance and quality gate
M32	Fundamental-rights screening; formal FRIA where Art. 27 applies
M33	Bias and fairness testing on affected groups
M34	Transparency to affected persons (AI disclosure, explanation, appeal path)
M35	Named accountable owner and responsibility matrix (value chain + three lines of defense)
M36	Re-classification triggers defined and monitored
M37	Minimum development process capability
M38	Independent safety assessment before go-live
M39	Operator and user training; AI literacy; end-user responsibility
M40	Agent interaction map and trust boundaries (freedom from interference)
M41	Authenticated and validated inter-agent messaging
M42	Network circuit breakers (aggregate thresholds regardless of contributing agent)
M43	Independence check for decomposed or layered controls (no shared base model, context or memory)
M44	Cross-firm interface contract (assume / guarantee safety obligations)
M45	Answers and statements only from authoritative, versioned sources; otherwise hand over to a human
M46	No commitments, offers or exceptions outside the agent's authority; such requests are routed to a human
M47	Lawful-basis review of decision logic and data items before go-live and after rule changes
M48	Data minimization in agent context and outputs
M49	Output and egress control (no auto-rendered external links or images; egress allow-list)
M50	Environment separation (sandbox or staging; gated production writes) and tested restore
M51	Audit by sampling of autonomous decisions

Annex B · Change log and self-check

Version history

Live draft — not saved as a version yet.

Five-point self-check

Check	Result
Understanding: every agent action is described (C01)	yes
Grey areas: every grey area is decided or an open point	yes
Side effects: every relevant hazard is rated	yes
Countermeasures: no ++ deviation without rationale and owner (C10)	yes
Reasoning: no open point linked to a critical-even-if-rare hazard (C09)	yes

Annex C · Glossary and assumptions

Glossary

Term	Definition
Agent / agentic AI	An AI system that takes actions toward a goal with some autonomy, e.g. calling tools, sending messages or making payments.
Action	Something the agent does that changes the world, e.g. "pays supplier invoice".
Hazard	A way the agent could cause harm, e.g. paying the wrong recipient.
Harm	Damage to the organization or to affected persons: financial, reputational, regulatory, operational, rights, health or safety.
Severity (S)	How bad the harm would be: the higher of harm to the organization and harm to affected persons.
AI-SIL	AI Safety Integrity Level 0–4: how much engineering rigor the flow needs. Derived from S, E and C.
Critical-even-if-rare	A hazard with the highest severity (S3), always shown even if its AI-SIL is low.
Grey area	Agent behavior that has not been decided yet, e.g. what happens when data is missing.
Guide word	A short prompt like "too much" or "too late" used to think through what could go wrong with an action.
Measure	A requirement that reduces risk: a design rule, control, process or check.
Safe state	The predefined behavior the agent switches to when something is wrong, e.g. pause and hand over to a human.
Human oversight (in / on / out of the loop)	Human approves before the action / monitors and can intervene / is not involved in the individual action.
Oversight sufficiency	Whether human review is real: enough time, information and authority to intervene. Otherwise it is "theatrical".
Decomposition	Splitting a safety goal across independent elements, e.g. the agent and a deterministic guard, so that each carries part of the rigor. Written X(Y): rigor X, goal Y.
Independence	Two layers are independent only if they do not share the same model, data or blind spots.
Verification	Proof that a measure is implemented and works: review, audit, evals, red-teaming, monitoring or assessment.
FRIA	Fundamental rights impact assessment under EU AI Act Art. 27; legally required for certain deployers.
AASIF Safety Concept	The living document per business flow with scope, risks, AI-SIL, measures, roadmap and contributions to regulations.
Contributes to	How AASIF describes its link to regulations and standards: a measure supports an obligation; it does not certify compliance.
Hazard situation	The situation in which a hazard can occur. Exposure counts how often it occurs, not how often the agent runs.
Element	A part of the architecture that carries part of a safety goal: the agent, a guard, a human gate, a separation or a diverse model.
Qualified human gate	A human check that meets four criteria (independent source check; competence, time and authority; measured detection rate; logged and auditable). Since v0.6 it can be used as decomposition pattern DP4; it never lowers C by itself.

Assumptions

This Safety Concept docks into the organization's existing management system (e.g. ISO/IEC 42001); it does not replace it.

Ratings, decisions and measures reflect the answers given at the date of this document and must be re-opened when a re-classification trigger occurs.

Test version: generated by the AASIF test tool with an uncalibrated Logic Pack. Not for confidential data and not a certification.

Annex D · Quality checks and traceability

Quality checks · 15 of 18 passed

ID	Check	Result
C01	At least one agent action is defined	passed
C02	Each action has been run through all guide words (T8), or a guide word is marked not applicable	open: ACT-01:G01, ACT-01:G02, ACT-01:G03, ACT-01:G04, ACT-01:G06, ACT-01:G08, ACT-01:G09, ACT-01:G10, ACT-01:G11, ACT-01:G12, ACT-01:G13, ACT-01:G14, ACT-01:G15, ACT-01:G16, ACT-02:G01, ACT-02:G02, ACT-02:G03, ACT-02:G04, ACT-02:G05, ACT-02:G06, ACT-02:G07, ACT-02:G08, ACT-02:G09, ACT-02:G10, ACT-03:G01, ACT-03:G02, ACT-03:G03, ACT-03:G05, ACT-03:G06, ACT-03:G09, ACT-03:G10
C03	Each hazard type (malfunction, SOTIF, security, rights, oversight, accountability, lifecycle; multi-agent if T09 = yes) has at least one hazard selected or marked not relevant	passed
C04	A pre-selected hazard marked "not relevant" needs a one-line reason	passed
C05	"Unsure" counts as relevant and adds an expert-review flag	noted: T14
C06	Action marked 'not at all' undoable (A03) but harm rated C1	open: H15
C07	E0 proposed although I14 = yes or T19 = yes (not allowed)	passed
C08	Flow AI-SIL ≥ 3 → expert review of the classification recommended (independence per T5)	open: Expert review of the classification recommended
C09	Undecided grey area linked to an S3 hazard → listed as go-live blocker	passed
C10	A deselected ++ measure needs a rationale and an owner	passed
C11	T09 = yes → network assessment measures (M40–M43) included	noted: M40, M41, M42, M43
C12	T7 T03 = yes → formal FRIA shown as legally required	passed
C13	Every S3 hazard appears in section 2.4 and is considered for the bow-tie view	noted: H06
C14	Five-point self-check offered at the end (understanding, grey areas, side effects, countermeasures, reasoning)	not evaluated
C15	T04 = yes (possible EU AI Act Annex III use) and flow AI-SIL < 2 → review the ratings; the AI-SIL is not overridden	passed
C16	An action with A02 = physical actuation → scope notice (t21): the domain functional-safety standard governs that action and its safety function; AASIF rates the remaining hazards of the flow and states the exclusion in the Safety Concept	passed
C17	DP4 (qualified human gate) needs evidence text for each of the four A10 criteria on every linked action	not evaluated
C18	Short path chosen but full rating gives flow AI-SIL ≥ 2 or any S3 → prompt to complete the full analysis	passed
C19	Applicable legal obligations (t18) are listed in section 2.5 and their measures marked 'required by law'	noted: LO01, LO04
C20	Internal action with A09 = yes but every linked hazard rated S1 → review severity	passed
C21	Split not allowed for the hazard's AI-SIL, element above its cap, or pattern does not cover the hazard → decomposition rejected with reason	not evaluated
C22	Any independence question IQ1–IQ7 answered yes or unsure, or evidence missing → decomposition rejected with reason	not evaluated
C23	Valid decomposition → M43 required and section 3.5 generated	not evaluated
C24	Decomposed hazard without safe-state confirmation for the non-agent element (D-R5)	not evaluated

Traceability

Logic Pack: v0.6.5 (0.6.4 + human-review round 2 presentation data (Safety Concept structure, explainer, step texts); classification, selection and grading unchanged)
SHA-256 checksum: 5e5f6cb7250bfcc70d113a8a3e776cbeb6bdf18562467a5086be5025133dd8d6
Snapshot: live draft
AI suggestions: on