Evaluate Models and Apps Quiz

What you'll be asked

1. Which of the following metrics is most commonly used to evaluate the factual accuracy of a generative AI application's output against a ground-truth reference?
2. You are building a RAG (Retrieval-Augmented Generation) application and notice the model frequently hallucinates when the retrieved context is irrelevant. Which evaluation approach best addresses this?
3. When evaluating an agentic workflow, why is 'Model-based Evaluation' (using an LLM to grade another LLM) often preferred over static unit tests?
4. In the context of evaluating a production-grade generative application, what is the primary purpose of 'A/B Testing'?
5. You are performing a 'Golden Dataset' evaluation. If your model achieves high performance on the golden dataset but performs poorly in production, what is the most likely cause?

Answers and explanations come after you submit.