A/B Testing Course
A/B Testing is used in applications, websites, and research to split the target audience between two variations and statistically determine the better-performing version based on predefined metrics.
One of the most common methods in data analysis and research impact assessment is A/B Testing. The central idea of this methodology is to examine and compare two variations of specific factors on different groups and determine which one yields the best outcome.
Throughout the course, students learn a statistical introduction to test construction and execution, using statistical tools for result analysis, and understanding the implications of these outcomes. Within the business context, the course provides tools for measuring the impact of changes and business decisions, such as choosing the appropriate graphical representation for the most effective result analysis.
Course Structure
During the course, we cover the statistical fundamentals of testing, methods to present and analyze results, and the implications of these findings. We also introduce two approaches: the classical approach based on multiple parameters and the modern approach based on Bayesian logic (Frequentist vs. Bayesian). We examine these approaches in various scenarios with different success metrics relevant to real-world industry scenarios.
In the course, you will learn:
- Fundamental concepts in probability and statistics
- A/B test fundamentals – definitions, approaches
- Parametric approach – Model, T-test, sample size estimation
- Bayesian approach – Discrete model, Continuous model
- Real-world applications: Learning periods, Control groups, Simulations, Simulators, Online testing
