A/B Testing Quiz
5 questions Pass: 70% +25 pts (50 with Pro)
Quiz covering Model Evaluation and AI Safety
What you'll be asked
- What is the primary purpose of conducting an A/B test when deploying a new version of a language model?
- When performing A/B testing on an AI chatbot, which of the following is considered a 'North Star' metric for user satisfaction?
- You are A/B testing a model's safety guardrails. If Version A has higher safety scores but significantly lower user engagement than Version B, what is the best next step?
- Which of the following scenarios describes a 'Novelty Effect' that could bias your A/B test results?
- In a sophisticated A/B testing setup for LLMs, why is 'statistical significance' (p-value) insufficient on its own to declare a winner?
Answers and explanations come after you submit.
Course Content
- Automated Machine Learning for Tabular Data
- AutoML Tabular Data Quiz5q
- Automated Machine Learning for Computer Vision
- AutoML Computer Vision Quiz5q
- Automated Machine Learning for NLP
- AutoML NLP Quiz5q
- Training Options Preprocessing and Algorithms
- Training Options Quiz5q
- Evaluating AutoML Runs with Responsible AI
- AutoML Responsible AI Quiz5q
- Configuring a Compute Instance Terminal
- Compute Instance Terminal Quiz5q
- Accessing and Wrangling Data in Notebooks
- Data Wrangling in Notebooks Quiz5q
- Using Synapse Spark Pools and Serverless Spark
- Spark Pools Quiz5q
- Retrieving Features from a Feature Store
- Feature Store Quiz5q
- Tracking Model Training with MLflow
- MLflow Tracking Quiz5q
- Evaluating Models with Responsible AI Guidelines
- Responsible AI Model Evaluation Quiz5q
- Consuming Data in a Job
- Consuming Data in Jobs Quiz5q
- Configuring Compute for a Job Run
- Job Compute Configuration Quiz5q
- Configuring an Environment for a Job Run
- Job Environment Configuration Quiz5q
- Tracking Model Training with MLflow in Jobs
- MLflow in Jobs Quiz5q
- Defining Parameters for a Job
- Job Parameters Quiz5q
- Running a Script as a Job
- Running Scripts as Jobs Quiz5q
- Using Logs to Troubleshoot Job Errors
- Job Troubleshooting Quiz5q
- Configuring Settings for Online Deployment
- Online Deployment Settings Quiz5q
- Deploying a Model to an Online Endpoint
- Online Endpoint Deployment Quiz5q
- Testing an Online Deployed Service
- Testing Online Services Quiz5q
- Configuring Compute for Batch Deployment
- Batch Compute Configuration Quiz5q
- Deploying a Model to a Batch Endpoint
- Batch Endpoint Deployment Quiz5q
- Invoking Batch Endpoint for Scoring Jobs
- Batch Scoring Jobs Quiz5q
- Preparing Data for Fine-Tuning
- Fine-Tuning Data Preparation Quiz5q
- Selecting an Appropriate Base Model
- Base Model Selection Quiz5q
- Running a Fine-Tuning Job
- Fine-Tuning Jobs Quiz5q
- Evaluating Your Fine-Tuned Model
- Fine-Tuned Model Evaluation Quiz5q
- Deploying Fine-Tuned Models
- Deploying Fine-Tuned Models Quiz5q
- Understanding AI Model Evaluation Metrics
- AI Model Evaluation Metrics Quiz5q
- Implementing Content Safety Filters
- Content Safety Filters Quiz5q
- Managing Model Bias and Fairness
- Model Bias and Fairness Quiz5q
- Model Interpretability and Explainability
- Model Interpretability Quiz5q
- Monitoring Model Performance in Production
- Model Performance Monitoring Quiz5q
- Data Drift Detection and Handling
- Data Drift Detection Quiz5q
- Azure AI Content Safety Integration
- Azure AI Content Safety Quiz5q
- Model Versioning and Lineage Tracking
- Model Versioning Quiz5q
- A/B Testing for Model Deployment
- A/B Testing Quiz5q
- Cost Optimization for ML Workloads
- Cost Optimization Quiz5q
- Security Best Practices for ML Models
- Security Best Practices Quiz5q
Godlike Exams
No DP-100 practice exam yet
Your points already work on 21 Amazon and Microsoft exams.
See the exams