AASIFWorked example · read-only
STEP 3 OF 7

Identify the hazards

Check which hazards could occur in your use case. The examples come from your own actions.

  1. 1Answer the trigger and guide-word questions
  2. 2Mark each proposed hazard relevant, not relevant (with a one-line reason) or unsure
  3. 3Add hazards of your own
16 of 16 hazards decided

Facts about your use case

Edit in step 1

Yes or unsure: T01, T02, T05, T06, T07, T09, T10, T11, T13, T14, T15

Guide words per action

Your hazards

16 of 16 decided

Wrong decision or action from incorrect output (hallucination, misreasoning)

Could the agent act on a wrong fact or a wrong conclusion?

In your use case, for example:

  • In step 'Pays supplier invoice', the agent pays the wrong amount or the wrong party because it misreads its inputs (Matched invoice, vendor master bank details).
  • In step 'Matches invoice to purchase order', the agent draws a wrong conclusion and acts on it.

Type: Malfunction · STPA: Provided wrongly

guide_word G02
Linked actions:

Action beyond authorized scope (wrong tool, wrong amount, wrong recipient)

Could the agent do something it was never meant to be allowed to do?

In your use case, for example:

  • In step 'Pays supplier invoice', the agent pays the wrong amount or the wrong party.
  • In step 'Pays supplier invoice', the agent uses a tool, a recipient or a scope it was never meant to use.

Type: Malfunction · STPA: Provided when unsafe

trigger T07
Linked actions:

Irreversible commitment executed before review (payment, contract, deletion, external message)

Could the agent commit something you can't take back, like a payment, contract or email, before anyone checks?

In your use case, for example:

  • In step 'Pays supplier invoice', the payment becomes final before anyone reviews it; afterwards the money can only be recalled with effort, and recalls often fail.
  • Once the money has left the account, the payment from step 'Pays supplier invoice' cannot be taken back.

Type: Controllability · STPA: Provided too early (before review)

trigger T01
Linked actions:

Correct behavior causes harm: misspecified objective or shortcut (reward hacking)

Could the agent do exactly what it was told and still hurt the business?

In your use case, for example:

  • In step 'Pays supplier invoice', the agent meets its target but harms suppliers or the business along the way.
  • The goal set for step 'Sends payment advice email to supplier' rewards a shortcut nobody wanted.
  • In step 'Matches invoice to purchase order', the agent meets its target but harms suppliers or the business along the way.

Type: SOTIF · STPA: Provided correctly, unsafe in context

trigger T14 (unsure)
Flow-level (no action linked)

Out-of-envelope context (new domain, market, population or data shift)

Could the agent meet a situation it was never designed for and not notice?

In your use case, for example:

  • In step 'Matches invoice to purchase order', the agent meets a case outside what it was designed for and does not notice.
  • The situation of suppliers changes, and the agent keeps applying old assumptions in step 'Matches invoice to purchase order'.

Type: SOTIF · STPA: Provided wrongly (wrong process model)

guide_word G11
Linked actions:

Manipulated input or goal hijack (prompt injection, poisoned documents or memory)

Could someone trick the agent through an email, a document or a website?

In your use case, for example:

  • Someone manipulates an input of step 'Pays supplier invoice' (Matched invoice, vendor master bank details) so that the agent acts against you.
  • A hidden instruction in a document or message takes over the agent in step 'Matches invoice to purchase order'.

Type: Security · STPA: Provided when unsafe (manipulated controller)

trigger T06
Linked actions:

Unfair or discriminatory outcome for affected persons

Could the agent treat some people or groups worse than others?

In your use case, for example:

  • In step 'Pays supplier invoice', the agent treats some suppliers worse than others.
  • Past data makes the agent repeat an old bias against a group in step 'Pays supplier invoice'.

Type: Rights · STPA: Provided when unsafe

trigger T02
Linked actions:

Cascading or emergent failure across agents

Could one agent's mistake trigger a chain reaction in other agents or systems?

In your use case, for example:

  • An error in another system or agent makes the agent act wrongly in step 'Matches invoice to purchase order'.
  • A mistake in step 'Matches invoice to purchase order' triggers a chain reaction further down the process.

Type: Multi-agent · STPA: Wrong timing or order across controllers

trigger T09
Linked actions:

Ineffective human oversight (automation bias, rubber-stamping)

Could the people checking the agent stop really checking?

In your use case, for example:

  • The people who check step 'Pays supplier invoice' approve without really checking.
  • Reviewers of step 'Pays supplier invoice' face more cases than they can check properly.

Type: Oversight · STPA: Not provided (by the human controller)

guide_word G14
Linked actions:

Untraceable decision (no evidence for audit or incident analysis)

Could you be unable to explain afterwards why the agent did something?

In your use case, for example:

  • Nobody can explain afterwards why the agent acted as it did in step 'Pays supplier invoice'.
  • When an incident needs analysis, the logs for step 'Sends payment advice email to supplier' are missing.
  • Nobody can explain afterwards why the agent acted as it did in step 'Matches invoice to purchase order'.

Type: Accountability · STPA: Missing feedback (control loop)

guide_word G13
Flow-level (no action linked)

Runaway consumption or loop (cost, rate, resources)

Could the agent get stuck in a loop and burn money or resources?

In your use case, for example:

  • In step 'Pays supplier invoice', the agent gets stuck in a loop and pays the same item twice.
  • Step 'Pays supplier invoice' uses far more budget or resources than planned.

Type: Malfunction · STPA: Applied too long

guide_word G07
Linked actions:

Third-party component failure or compromise (model, tool, plugin, agent product)

Could a vendor's model, tool or agent change or fail and take your process down with it?

In your use case, for example:

  • The vendor behind step 'Pays supplier invoice' changes or withdraws its model or tool.
  • A tool or plugin used in step 'Sends payment advice email to supplier' is compromised.
  • The vendor behind step 'Matches invoice to purchase order' changes or withdraws its model or tool.

Type: Supply chain · STPA: Provided wrongly (component)

trigger T11
Flow-level (no action linked)

Misleading communication to humans (undisclosed AI, overtrust, impersonation)

Could people not realize they are dealing with an AI, or trust it too much?

In your use case, for example:

  • Suppliers do not realize they are dealing with an AI in step 'Sends payment advice email to supplier', or trust it too much.
  • In step 'Sends payment advice email to supplier', the agent sounds certain where it is not, and people rely on it.

Type: Rights / Trust · STPA: Provided when unsafe

trigger T13
Linked actions:

Required action not performed (silent omission)

Could the agent silently stop doing something that must happen?

In your use case, for example:

  • Step 'Pays supplier invoice' silently stops, and something that must happen does not.
  • In step 'Pays supplier invoice', the agent skips cases without telling anyone.

Type: Malfunction · STPA: Not provided

guide_word G01
Linked actions:

Sensitive data disclosure or privacy breach

Could the agent reveal confidential or personal data to the wrong party?

In your use case, for example:

  • In step 'Pays supplier invoice', the agent reveals personal or confidential data to the wrong party.
  • Details from the inputs of step 'Sends payment advice email to supplier' (Payment record, supplier contact data) end up where they should not.
  • In step 'Matches invoice to purchase order', the agent reveals personal or confidential data to the wrong party.

Type: Security / Data · STPA: Provided when unsafe

Flow-level (no action linked)

Silent degradation after model, prompt or context change

Could the agent get worse after an update without anyone noticing?

In your use case, for example:

  • After a model or prompt update, the agent gets worse in step 'Pays supplier invoice' and nobody notices.
  • The agent slowly drifts in step 'Sends payment advice email to supplier' as its inputs (Payment record, supplier contact data) change over time.
  • After a model or prompt update, the agent gets worse in step 'Matches invoice to purchase order' and nobody notices.

Type: Lifecycle · STPA: Provided wrongly (drifted process model)

Flow-level (no action linked)

Step 3 complete — every hazard has a decision.

Each action has been run through all guide words (T8), or a guide word is marked not applicable — still open: ACT-01:G01, ACT-01:G02, ACT-01:G03, ACT-01:G04, ACT-01:G06, ACT-01:G08, ACT-01:G09, ACT-01:G10 +23

Back

Classification is deterministic — AI only adapts wording and suggests, you confirm.