LLM-as-Judge Evaluation Quiz

5 questions Pass: 70% +25 pts (50 with Pro)

Quiz covering Advanced Evaluation Techniques

What you'll be asked

  1. What is the primary purpose of using an 'LLM-as-a-Judge' in a QA pipeline?
  2. When using an LLM-as-a-Judge, what is the 'positional bias' phenomenon?
  3. Which strategy is most effective for reducing 'verbosity bias' when an LLM acts as a judge?
  4. Why is it considered a best practice to use a stronger model (e.g., GPT-4o) to evaluate the outputs of a smaller model (e.g., GPT-3.5 or Llama 3 8B)?
  5. In a sophisticated RAG evaluation framework, why might you use 'Reference-Guided' evaluation instead of 'Reference-Free' evaluation?

Answers and explanations come after you submit.