Turn evaluation into repeatable test sets
Evaluation should not rely on ad hoc questions. Collect real tasks with expected answers, required sources, acceptable variance and mandatory refusals, then rerun them after changes to models, prompts, knowledge or tools.
- Answer correctness and key-fact consistency
- Source recall and citation accessibility
- Correct refusal, clarification and escalation
- Latency, usage cost and failure rate
Test authorization boundaries and prompt injection
Testers should attempt to override rules, access another department, expose system instructions or invoke unauthorized tools. Security cannot live only in prompts; identity, retrieval filters and tool authorization must enforce it.
Operate continuous evaluation after launch
After launch, monitor adoption, human edits, unanswered requests, error categories, tool failures and abnormal cost. Incidents should be traceable to model, knowledge, prompt and tool-call versions.
- Require human confirmation for high-risk tasks
- Retain necessary logs while limiting sensitive data
- Provide kill switches, quotas and anomaly alerts