Operation risk assessment
Before a query reaches the model or an AI digital worker performs an action, AiTrust assesses it according to the rules set by the organization and assigns a risk zone. The rest of the workflow depends on the zone: in the examples shown, restricting the content or a requirement for human approval.
- Intent
- Input control
- Authorization (applies)
- AI
- Result validation
- Human
- Evidence
Highlighted: the points of the flow that this description refers to. Orange marker: a human decision at this point.
What the assessment consists of
Risk assessment in AiTrust means applying the organization’s rules to a specific operation. The organization sets the criteria for assigning zones, and what follows from each zone, during deployment.
- Operation
- A single query to the model or an action of an AI digital worker, e.g. preparing a draft reply to a complaint.
- Rule and match
- A rule recognizes a designated data category in the content; a match means it has detected something. In the example: an account number (pii.iban) and a phone number (pii.phone_pl). The abbreviation pii stands for personally identifiable information.
- Zone
- A risk class marked with a letter (A, B and D in the examples shown). The organization sets the rules for assigning zones as part of the Human oversight of AI methodology.
- Decision
- The next step that follows from the zone and the policy. In the example: Redact, that is, restricting the content by replacing detected data with tags.
Limitations of the assessment
Rules may not recognize all data. That is why the user sees the version prepared for sending and can correct it before it reaches the model.
The record of the assessment is not a data protection impact assessment (DPIA) or a legal assessment. What the run record can contribute to an audit is described on the AI Governance page, in the part on audits.
Conditions, transitions and decisions
Diagram of the controlled run
- Step 1: OperationA query or a planned action together with its context, e.g. the selected AI digital worker.
- Step 2: RulesRules check whether the content contains data categories designated by the organization.
- Step 3: ZoneBased on the rules and the matches, the operation is given a risk zone.
- Step 4: DecisionThe policy for the zone sets the next step, e.g. restricting the content or human approval before execution.
- Step 5: RecordThe zone and the decision go into the evidence trail of the operation in the controlled path.
| Condition | Outcome in the example shown |
|---|---|
| The query contains designated data, and the zone policy provides for restriction | Detected data is replaced with tags, and the user decides whether to send the restricted version (recorded run, see Boundaries in the workflow). |
| The AI digital worker’s action falls within the zone allowed in its agent passport | The AI digital worker acts within the limits granted to it. |
| The AI digital worker’s action goes beyond the designated zone | Approval by an authorized person is required before execution (a separate example: a passport that requires approval above zone B, see Accountability for decisions). |
The assessment visible before sending
The screenshot comes from one recorded run in AiTrust on synthetic data: preparing a reply to a customer complaint.
Zone, rule matches and decision
The check panel has four parts:
- A: the instruction after the filters,
- B: the version sent to AI,
- C: what was removed or masked (e-mail, phone and account number, 3 redactions in total),
- D: risk and policy (zone D, rule matches and the Redact decision).
We show the whole run, from instruction to result, in the Boundaries in the workflow example.
What the screenshots show
- The assessment of one query in this configuration: the zone, the rule matches and the decision.
- The choices available to the user before sending.
What the screenshots do not prove
- How zones are assigned in other deployments (the organization sets the criteria).
- The effectiveness of detection in other queries and data categories.
- Approval of an operation by an authorized person.
- A legal assessment or a DPIA.
Related areas
Methodologies and compliance
In the platform description
Let’s discuss risk zones for your processes
In the demo, we will show the assessment of a query on synthetic data and discuss how to set zones and decisions for operations in your organization.