Start with a small, representative eval set for each core task: Q&A, summarization, extraction. Define clear acceptance criteria and sample expected outputs.
Instrument your app to capture explicit ratings and implicit signals (edits, abandonment). Combine with offline metrics like BLEU/ROUGE for trend tracking, but prioritize human-verified outcomes.
Set up a weekly eval pipeline. Compare prompts, models, temperature, and retrieval variants. Keep experiments scoped and document changes to avoid regression.
When issues recur, enrich retrieval sources, refine prompts, and add safety filters. Share learnings in a changelog to help teammates avoid repeating mistakes.