AI systems
Designing AI automation that is allowed to be uncertain
A practical pattern for using language models in business workflows without pretending every generated answer is correct.
Automate a decision-shaped task
‘Add AI’ is not a product requirement. Choose a bounded task with a recognizable input, output, reviewer, and cost of error: classifying an inquiry, drafting a response from approved material, extracting fields from a document, or identifying records that need attention.
Write down what happens when the model is unsure or unavailable. If the workflow cannot tolerate a wrong answer, the model should recommend or prepare work rather than commit the final action.
Build an evidence path
Generated text is easier to review when the interface shows where important statements came from. Retrieval results should preserve source identifiers and useful excerpts, and the final output should link claims back to them. This turns review from intuition into inspection.
Structured outputs help, but a valid JSON object can still contain a bad decision. Validate types and business rules separately, then apply confidence thresholds or human review according to risk.
- Keep approved source material versioned.
- Separate system instructions from untrusted user content.
- Log model, prompt version, latency, cost, and outcome—not unnecessary sensitive input.
- Create a non-AI path for outages and rejected outputs.
Evaluate the failure you care about
A generic benchmark rarely describes a specific business workflow. Build a small evaluation set from real, permissioned examples, including ambiguous and adversarial cases. Define unacceptable errors before comparing models or prompts.
Measure the complete system: extraction accuracy, unsupported claims, reviewer corrections, time saved, abandonment, and downstream mistakes. A faster generation that creates more review work is not an improvement.
Earn more autonomy gradually
Start in shadow mode, where the system produces an answer without affecting the workflow. Next, let people approve suggestions. Only automate low-risk actions after results are stable and reversible. Keep monitoring after release because inputs, source material, and model behavior change.
Evidence from our work
Unavailable and uncertain are different states
In systems we operate, a generated suggestion is not allowed to silently become a payment, deployment, access-control change, or outbound message. We separate preparation from commitment and preserve the evidence a reviewer needs to decide. A timeout produces an unavailable state; weak or conflicting evidence produces an uncertain state. Those states need different recovery paths.
The same pattern applies outside language models. When an ad SDK reports no fill, retrying layout code is not a valid substitute for recording the provider error. When authentication fails, the application must distinguish user cancellation, certificate configuration, network failure, and server rejection.
Limit: Examples describe control patterns. They do not claim that a confidence score alone makes an automated action safe.