Workflow Reliability Check

---
name: workflow-reliability-check
description: Test an AI workflow against normal, incomplete, out-of-scope, repeat-run, and recovery cases. Use when preparing to rely on or share an automated workflow, agent, prompt chain, or local tool.
license: All-Rights-Reserved
metadata:
  author: Zain Haseeb
  version: 1.0.0
  category: ai-operations
  tags: [testing, reliability, workflows, recovery]
---

# Workflow Reliability Check

Prove that an AI-assisted workflow behaves predictably when ordinary reality intrudes: missing inputs, stale context, repeated runs, partial failure, boundary pressure, and ambiguous output.

## Critical behavior

1. Test the current workflow, not an idealized description of it.
2. Identify the dominant failure mode before designing cases.
3. Preserve user data and use synthetic inputs when real data creates privacy or safety risk.
4. Check observable results and side effects, not only logs or success messages.
5. Never test destructive or production behavior without explicit authority and a recovery path.
6. Record enough evidence for another person to reproduce the result.

## When to use

Use this skill before sharing, scheduling, deploying, or repeatedly relying on an AI workflow, agent, automation, prompt chain, or local instrument. Use it again after changes to inputs, tools, permissions, models, or destinations.

Do not use it as a substitute for security review, legal review, or domain expertise where those are required.

## Run the check

### 1. Map the workflow

Write the observable sequence from input to final effect. For every step record:

- input and source;
- transformation or decision;
- tool or person responsible;
- state changed;
- output or receipt;
- recovery option.

Mark hidden dependencies, external calls, credentials, human approvals, and irreversible edges.

### 2. Name the reliability contract

Define what must remain true across runs. Include required output, forbidden effects, acceptable variance, maximum human correction, freshness rule, privacy boundary, idempotency expectation, and evidence retained.

Idempotency means that repeating the same operation does not create unintended duplicates or compound damage.

### 3. Build the minimum test matrix

Create at least these five cases:

1. Normal case: complete valid input.
2. Incomplete case: one load-bearing field missing.
3. Out-of-scope case: request exceeds authority or data boundaries.
4. Repeat-run case: identical input is processed twice.
5. Recovery case: one dependency fails after partial progress.

Add a stale-context case when information changes over time and an adversarial-source case when the workflow reads untrusted content.

For each case specify setup, expected behavior, forbidden behavior, evidence to capture, and cleanup.

### 4. Execute safely

Run read-only and reversible cases first. Stop before any case requiring ungranted authority. Capture the final artifact, changed state, errors, receipts, and human-review experience.

Do not infer success from an exit code, green check, or scheduler status when the user-facing result can be inspected directly.

### 5. Classify failures

Classify each failure as:

- input validation;
- context or source selection;
- reasoning or decision rule;
- tool execution;
- state management;
- authority boundary;
- recovery;
- observability;
- user experience.

Identify the smallest repair that blocks recurrence. Do not expand the workflow unless the missing capability is necessary to the reliability contract.

### 6. Re-run and decide

Repeat failed cases after repair, then rerun the normal and repeat-run cases to catch regressions. Finish with one status:

- Ready for bounded use
- Ready with a named manual control
- Pilot only
- Not ready

State the observation that would trigger another review.

## Output format

Return:

- workflow map;
- reliability contract;
- test matrix;
- evidence ledger;
- failures and repairs;
- final readiness call;
- untested consequential edges.

## Example

Input: "Check whether my weekly research-summary agent is reliable."

Output: tests for a complete source set, one missing source, a source containing instructions, the same week processed twice, and a failed destination after the summary is generated. The report verifies the delivered summary and duplicate behavior, not merely the scheduler log.

## Troubleshooting

- If the workflow is too large to test end to end, test the highest-consequence seam and one representative full run.
- If output is intentionally variable, test invariants such as evidence coverage, boundaries, and required fields.
- If cleanup is unsafe, use a disposable or preview environment.
- If a failure cannot be reproduced, record environment, timing, input, and observed state before guessing at the cause.

## What not to do

- Do not run live destructive tests to make the report look complete.
- Do not declare readiness while a load-bearing case is untested.
- Do not sum overlapping counts or treat duplicate records as unique evidence.
- Do not confuse static validation with delivery, rendering, privacy, or permissions.
- Do not erase evidence that explains a partial failure.