Agent Reliability for Nondevelopers

You will produce an Agent Reliability Card and use it to run one bounded synthetic workflow through success, rejection, and recovery.

Operators, strategists, and team leads who direct AI agents but do not maintain production software systems.

• One recurring agent task
• A synthetic example
• Knowledge of who owns the outcome
• A list of possible external actions
• No coding knowledge required

1. 1. Define reliability in plain language

   Reliability means the workflow produces a useful inspectable artifact for the cases claimed, fails visibly outside them, and does not exceed authority. It does not mean the model never makes mistakes.

2. 2. Write the job contract

   Name outcome, deliverable, evidence, success criteria, boundaries, stop conditions, and owner. If the agent cannot know one of these, mark it for human input.

3. 3. Map authority

   Separate reading, transforming, judging, proposing, approving, and acting. Put human approval before messages, publishing, deletion, payment, deployment, permissions, and sensitive data movement.

4. 4. Design evidence

   Require the artifact, source trail, calculation, test, rendered output, or execution receipt that proves each stage. A green scheduler status proves only that something ran.

5. 5. Make failure visible

   Specify what happens with missing input, ambiguous scope, tool failure, stale approval, duplicate triggers, and partial external action. Default to no action at consequential edges.

6. 6. Run three synthetic paths

   Test a normal case, a rejected approval, and a tool failure with recovery. Confirm the agent neither improvises authority nor hides incomplete work.

7. 7. Set monitoring and reversal

   Choose what a human inspects, how often, what change triggers reevaluation, and what evidence suspends the workflow.

Synthetic example

Scenario

A small editorial team uses an agent to prepare a weekly public-content candidate list for a human editor.

Input

Public sources, a topic boundary, no publication authority, and a synthetic source with a conflicting date.

Result

The agent produces candidate cards with claim statuses and flags the conflict. It proposes but does not publish. A rejected card stays rejected; a browser error produces an incomplete receipt and no fabricated source.

• Equating autonomous behavior with reliable behavior
• Giving broad standing authority because one prior run succeeded
• Checking logs without inspecting artifacts
• Letting no response count as approval
• Retrying a partial external action without reconciliation

• □The outcome and owner are explicit
• □Read, propose, approve, and act permissions are separated
• □Every consequential action has a prior human gate
• □Normal, rejection, and recovery paths are demonstrated
• □Evidence proves the artifact rather than only execution
• □Monitoring and suspension conditions are recorded

agent-reliability-card.md

# Agent Reliability Card

## Job contract
- Outcome:
- Deliverable:
- Owner:
- Evidence:
- Success criteria:

## Authority
- May read:
- May transform:
- May judge:
- May propose:
- Requires approval before:
- May never:

## Failure behavior
- Missing input:
- Ambiguous scope:
- Tool failure:
- No approval response:
- Duplicate trigger:
- Partial external action:

## Demonstration cases
- [ ] Normal case
- [ ] Rejected approval
- [ ] Tool failure and recovery

## Monitoring
- Artifact inspected:
- Review frequency:
- Re-evaluation trigger:
- Suspension condition:

## Current status
[Draft / Demonstrated for bounded cases / Suspended]

1. 0:00 Show a reliable artifact and an unsafe action side by side
2. 0:40 Define reliability without technical jargon
3. 1:15 Fill the job contract
4. 2:00 Map read, propose, approve, and act authority
5. 2:50 Trigger a rejected approval
6. 3:35 Trigger a tool failure and show the receipt
7. 4:20 Set monitoring and suspension conditions