Define the job and the limit
A convincing demonstration establishes that a system can produce a useful result for a particular input. It does not establish how often the system fails, what those failures cost, or whether anyone will notice them. Production readiness begins with a bounded task and an explicit definition of unacceptable behavior.
Choose a narrow responsibility such as drafting a summary from an approved document set. State what the system must not do: invent missing facts, treat retrieved instructions as authority, disclose restricted information, or take consequential actions on its own. Define an abstention path when the evidence is insufficient.
Separate output quality from authorization. A plausible answer is not permission to update a record, contact someone, or affect equipment. Keep any action pathway outside the model response and enforce permissions in the surrounding application.
Evaluate failures that matter
Build an evaluation set that reflects the intended task, including incomplete inputs, contradictory sources, unsupported questions, and malicious instructions embedded in retrieved text. Use synthetic or appropriately approved data, and protect evaluation material that contains sensitive information. Preserve a held-out set rather than tuning repeatedly to every example.
Score dimensions separately: factual support, completeness, appropriate abstention, access control, latency, and cost. An average quality score can conceal a small number of unacceptable disclosures or unauthorized actions. Define release criteria with the people accountable for those consequences.
Use human review to calibrate automated scoring, particularly for ambiguous cases. Model-based judges can help organize evaluation, but their judgments also need checking. Record the model, prompt, retrieval configuration, and application version so comparisons have a meaningful reference point.
Contain the runtime
Treat retrieved content and model output as untrusted input. Validate structured responses against the expected schema, constrain input and output sizes, and bound execution time and retries. Schema validity only establishes structure; it does not establish truth, relevance, or permission.
Start with an isolated evaluation environment and no external side effects. A conceptual review record might look like the example below. It is not a security configuration: the application must actually enforce access restrictions and prevent tool execution. Where sensitive or safety-related decisions are involved, use qualified human review and the established change process.
{
"purpose": "draft summary for review",
"environment": "isolated evaluation",
"externalActions": false,
"humanApprovalRequired": true
}Operate with a fallback
Decide what users see when the model is unavailable, a response fails validation, or the source material is insufficient. A clear failure message and access to the underlying documents may be more useful than repeated attempts to produce an answer. Never silently turn an abstention into a guessed result.
Monitor task-level outcomes alongside service health. Track validation failures, user corrections, abstentions, latency, and spending without indiscriminately retaining prompts or sensitive source text. Set a deliberate retention policy and make access to diagnostic records appropriately restricted.
Reevaluate changes to models, prompts, retrieval, and permissions before release. A successful previous version does not certify a new combination. Maintain a way to disable the AI feature while preserving a usable manual workflow. Reliability includes making failure understandable and keeping the rest of the product useful when the model cannot help.