Build anti-runaway controls into an AI workflow as a state machine

Model the job with finite states before writing the loop

Choose one workflow and define states such as queued, validating, awaiting approval, executing step, verifying, complete, no work, blocked, cancelling, cancelled, uncertain external outcome, failed, and reconciled. Specify allowed transitions, actor, required evidence, timeout, and side effect for each. Store the state transactionally outside conversational context. A model can propose the next transition but cannot invent a new state or mark complete without verification. The bounded automation state machine prevents restart from forgetting an earlier write or approval. Draw impossible transitions explicitly: cancelled cannot return to executing without a new authorized run; uncertain outcome cannot automatically retry an external action.

Evidence: National Institute of Standards and Technology; OpenAI

Attach a limit ledger to the run and every side-effect class

Create atomic counters for model turns, retrievals, tool calls, retries, elapsed time, tokens, spend, records, bytes, messages, and privileged operations. Record configured maximum, warning threshold, current use, reservation, and reason for change. Reserve budget before a call and reconcile actual usage after, so concurrent workers cannot overspend the same allowance. Add a total run deadline and per-step timeouts. The system—not the model—refuses transitions when budget is unavailable. An authorized user may start a new bounded run, but the agent cannot extend itself. Expose remaining limits in the interface so a person can make an informed stop or continuation decision.

Evidence: National Institute of Standards and Technology; OWASP Gen AI Security Project

Create narrow tools under a distinct agent identity

Replace generic shell, database, cloud, mailbox, or payment access with task-specific functions. Validate types, ranges, paths, recipients, ownership, and policy before invocation. Use read-only credentials for discovery, separate draft from publish, sandbox code, restrict network destinations, and require idempotency keys for writes. Assign the run and agent a distinguishable identity, with user delegation and expiration. NIST NCCoE's current project explores identity and authorization practices for agents; treat it as emerging guidance, not a finished certification. OWASP's excessive-agency analysis supports reducing functionality, permission, and autonomy together. Remove trial tools when the workflow no longer needs them.

Evidence: National Cybersecurity Center of Excellence; OWASP Gen AI Security Project

Detect stuck behavior from state and action fingerprints

Hash normalized action type, target, parameters, result class, and resulting state. Trigger review when the same fingerprint repeats, state fails to advance, alternating actions form a cycle, errors recur beyond a small policy, output grows without new evidence, or cost slope exceeds expectation. Some legitimate polling repeats, so define cadence, maximum attempts, and a changed external condition that justifies continuation. Do not ask the model whether it is stuck as the only detector. Record the last meaningful progress event and have the orchestrator stop independently. Route the trace and checkpoint to a human rather than starting a recursive repair agent with broader access.

Evidence: National Institute of Standards and Technology; OpenAI

Place approvals before exact consequential transitions

For external messages, publication, spending, deletion, privileged changes, sensitive-data transfer, or other material effects, generate a proposal containing action, target, content or diff, cost, evidence, risks, rollback, and expiry. Approval binds to that immutable proposal; any material change invalidates it. Rejection, timeout, or unavailable reviewer returns a safe state. Do not bundle unrelated actions in one vague confirmation. Log who authorized what without exposing secret data. Make low-risk read and draft work possible without constant prompts, while preserving hard gates where consequence rises. This is how the state machine distinguishes productive assistance from hidden autonomy.

Evidence: National Cybersecurity Center of Excellence; OWASP Gen AI Security Project

Test cancellation and uncertainty, then release read-only

The next action is to implement the machine for one workflow with only read and draft tools, then simulate budget exhaustion, repeated actions, provider timeout, process restart, user stop, approval rejection, duplicate webhook, and an external call whose response is lost. Verify checkpoints, queue cancellation, idempotency, reconciliation, and manual recovery. Limits remain: distributed systems have race conditions, vendor tools can hide effects, anomaly thresholds need tuning, and no state model covers every incident. Release to narrow internal use with metrics for stops, overrides, uncertain outcomes, and recoveries. Expand authority only after the same controls have evidence. Commercial orchestration platforms remain subject to independent criteria and affiliate disclosure.