Back
Control example

Operation risk assessment

Before a query reaches the model or an AI digital worker performs an action, AiTrust assesses it according to the rules set by the organization and assigns a risk zone. The rest of the workflow depends on the zone: in the examples shown, restricting the content or a requirement for human approval.

The points of the flow this example concernsSee the whole flow
  1. Intent
  2. Input control
  3. Authorization (applies)
  4. AI
  5. Result validation
  6. Human
  7. Evidence

Highlighted: the points of the flow that this description refers to. Orange marker: a human decision at this point.

Explanation

What the assessment consists of

Risk assessment in AiTrust means applying the organization’s rules to a specific operation. The organization sets the criteria for assigning zones, and what follows from each zone, during deployment.

Operation
A single query to the model or an action of an AI digital worker, e.g. preparing a draft reply to a complaint.
Rule and match
A rule recognizes a designated data category in the content; a match means it has detected something. In the example: an account number (pii.iban) and a phone number (pii.phone_pl). The abbreviation pii stands for personally identifiable information.
Zone
A risk class marked with a letter (A, B and D in the examples shown). The organization sets the rules for assigning zones as part of the Human oversight of AI methodology.
Decision
The next step that follows from the zone and the policy. In the example: Redact, that is, restricting the content by replacing detected data with tags.

Limitations of the assessment

Rules may not recognize all data. That is why the user sees the version prepared for sending and can correct it before it reaches the model.

The record of the assessment is not a data protection impact assessment (DPIA) or a legal assessment. What the run record can contribute to an audit is described on the AI Governance page, in the part on audits.

Diagram

Conditions, transitions and decisions

Diagram of the controlled run

  1. Step 1: OperationA query or a planned action together with its context, e.g. the selected AI digital worker.
  2. Step 2: RulesRules check whether the content contains data categories designated by the organization.
  3. Step 3: ZoneBased on the rules and the matches, the operation is given a risk zone.
  4. Step 4: DecisionThe policy for the zone sets the next step, e.g. restricting the content or human approval before execution.
  5. Step 5: RecordThe zone and the decision go into the evidence trail of the operation in the controlled path.
Illustrative diagram: the conditions and decisions of risk assessment, not a view of a running system or the list of rules of a specific deployment.
Conditions and outcomes in the examples shown
ConditionOutcome in the example shown
The query contains designated data, and the zone policy provides for restrictionDetected data is replaced with tags, and the user decides whether to send the restricted version (recorded run, see Boundaries in the workflow).
The AI digital worker’s action falls within the zone allowed in its agent passportThe AI digital worker acts within the limits granted to it.
The AI digital worker’s action goes beyond the designated zoneApproval by an authorized person is required before execution (a separate example: a passport that requires approval above zone B, see Accountability for decisions).
Recorded run

The assessment visible before sending

The screenshot comes from one recorded run in AiTrust on synthetic data: preparing a reply to a customer complaint.

Recorded run, step 2 of 3

Zone, rule matches and decision

The check panel has four parts:

  • A: the instruction after the filters,
  • B: the version sent to AI,
  • C: what was removed or masked (e-mail, phone and account number, 3 redactions in total),
  • D: risk and policy (zone D, rule matches and the Redact decision).

We show the whole run, from instruction to result, in the Boundaries in the workflow example.

The check before sending in the recorded example: zone D, 3 redactions, Redact decision (synthetic data).

The check before sending in the recorded example: zone D, 3 redactions, Redact decision (synthetic data).

The “Security check before sending” panel with zone D: the version sent to AI with the [EMAIL], [PHONE] and [IBAN] tags, 3 redactions, the Redact decision and the Cancel, Edit prompt, Check again and Send restricted version buttons.

What the screenshots show

  • The assessment of one query in this configuration: the zone, the rule matches and the decision.
  • The choices available to the user before sending.

What the screenshots do not prove

  • How zones are assigned in other deployments (the organization sets the criteria).
  • The effectiveness of detection in other queries and data categories.
  • Approval of an operation by an authorized person.
  • A legal assessment or a DPIA.

Let’s discuss risk zones for your processes

In the demo, we will show the assessment of a query on synthetic data and discuss how to set zones and decisions for operations in your organization.

Request an AiTrust demo

Leave your contact details. We’ll agree on the scope of the demo and show how AiTrust can support your organization’s processes.

The controller of the data provided in this form is Polski Instytut AI Sp. z o.o. We will use the data to answer your request and arrange the demo. More information, including your rights, is in the documents GDPR information notice and Privacy policy.

* required field

Want to add your company, phone or a description of the task? Open the full form