Quality rarely falls because a model got worse overnight. It falls because the world moved: a new product line, a changed form, a policy update, a supplier sending a different file format. The system keeps doing exactly what it was built to do, and the work it was built around has quietly changed underneath it. Nothing in the code changed and nothing in the report changed either, which is why a drop is often noticed late.
The first defence is a fixed test set that is never used for tuning. Collect real examples, agree the correct answer for each one with the people who do the work, and keep them in a file that nobody edits while improving the system. If the test set is adjusted whenever results look poor, it stops measuring anything. Review it once a quarter and add new cases as the work changes.
The second is a sample of live output, read by a person who knows the work. A fixed number of items each week, chosen at random, with the disagreements written down. This is not a full audit and is not meant to be: it is an early warning, because the first signs of drift usually appear as a handful of odd cases that a reviewer notices before any dashboard moves. Keep the sample size constant so the numbers stay comparable. Write the batch date next to each sample, so a later reviewer can tell when a case first failed.
Watch the inputs as carefully as the outputs. Missing fields, longer documents, new languages and sudden changes in volume show up in the pipeline before they show up in quality. Treat those conditions as incidents. An input change that nobody noticed is the most common explanation for a quality drop, and it is also the cheapest one to fix.
Write down what good enough means before you need the answer. Decide, per error type, how often a mistake is acceptable and who is allowed to pause the system. When quality discussions happen without that agreement in advance, they turn into arguments about anecdotes, and the loudest recent example wins. A written tolerance turns the same discussion into a decision about whether the numbers are inside the line.
Keep a version history and a way back. Record what changed, when and on what evidence, and keep the previous version runnable for a stated period, so a rollback is a decision rather than a project. Change one thing at a time where you can: a change to the prompt, the retrieval step and the review rules at once makes it impossible to say which one helped. The same rule applies to the test set, where adding cases is fine and quietly replacing old ones is not.
Then set a rhythm: a short monthly check against the test set and the sample, and a deeper review each quarter covering drift, cost and the cases that keep failing. Retrain or re-prompt only with a reason written down first. A change made without a recorded reason cannot be evaluated later, and it usually gets reverted by whoever holds the system next. The goal is not a model that is always right. It is a system whose quality you can describe with evidence, and defend when someone asks.