A/B Test like a Pro! (online course) – Data36 – Data Science Online Video Courses

Current Status

Not Enrolled

Price

Closed

Get Started

This course is currently closed

Course Content

Expand All

#0 – WELCOME 3 Lessons

Expand

Module Content

0% Complete 0/3 Steps

Welcome to the course

About me

What’s in the course? How to get the most out of it.

#1 – INTRODUCTION 6 Lessons

Expand

Module Content

0% Complete 0/6 Steps

What is A/B testing exactly?

The correlation vs. causation issue

Why do you need A/B testing? (a story and the real reason)

What can you A/B test?

EXTRA: Hand-picked A/B Testing case studies

ACTION: The A/B tests you’ve run so far — and the A/B tests you want to run in the future

#2 – IMPORTANT INITIAL BUSINESS QUESTIONS 7 Lessons

Expand

Module Content

0% Complete 0/7 Steps

Important A/B Testing mindset: Conversion Rate Optimization or Research?

Two key questions before starting an A/B Test. (And when is A/B testing a really bad idea?)

When you don’t need to run experiments

When you need to run experiments

A/B test only when you need it

EXTRA: the “can I run this A/B test” checklist

ACTION: Filter out the bad A/B test ideas from your backlog!

#3 – STATISTICAL BACKGROUND 8 Lessons

Expand

Module Content

0% Complete 0/8 Steps

Statistics is important

Randomization (we all are terrible “random-generators”)

Statistical significance and the certainty of your results

An intuitive way of interpreting the importance of statistical significance

80%? 95%? 99%? What’s the right confidence level?

Calculating the required sample size before the test

EXTRA: online calculators (statistical significance and sample size)

ACTION: Review your A/B testing backlog!

VAULT: Q&A 1 Lesson

Expand

Module Content

0% Complete 0/1 Steps

Your questions answered.

#4 – EXECUTION – STEP 1: PREPARING YOUR A/B TEST(S) 6 Lessons

Expand

Module Content

0% Complete 0/6 Steps

The four steps of executing an A/B test

The Research Framework

Your success metric(s) to evaluate your test

A few words about setting a hypothesis

EXTRA: A/B Testing Hypothesis Form

ACTION: Run your first research round!

#4 – EXECUTION – STEP 2: IMPLEMENTATION 7 Lessons

Expand

Module Content

0% Complete 0/7 Steps

My favorite A/B testing tools

Tool demo (Google Optimize)

Avoiding website flickering in an A/B test

The change-one-thing-at-a-time myth, multivariate tests, A/B/n tests and unusual audience splits

A few typical mistakes at the implementation phase

EXTRA: The “can I start the A/B test?” checklist

ACTION: Try out the simplest ever A/B test!

#4 – EXECUTION – STEP 3: RUNNING A TEST 3 Lessons

Expand

Module Content

0% Complete 0/3 Steps

How long should an A/B test run?

Which part of the year/week shouldn’t you run an A/B test?

Typical mistakes while running an A/B test

#4 – EXECUTION – STEP 4: EVALUATING YOUR EXPERIMENTS 4 Lessons

Expand

Module Content

0% Complete 0/4 Steps

Don’t lie to yourself!

What to do when a test didn’t perform the way you expected

What to do after evaluating your experiment

ACTION: Start your own knowledge base!

#5 – A FEW SMALLER ADDITIONS 4 Lessons

Expand

Module Content

0% Complete 0/4 Steps

The limitations of A/B testing

The art and science of A/B testing

Segmentation

EXTRA: Further reads and videos

#6 – SUMMARY 2 Lessons

Expand

Module Content

0% Complete 0/2 Steps

Summary & The right mindset to be successful with A/B testing

Where to go next

VAULT: Q&A 1 Lesson

Expand

Module Content

0% Complete 0/1 Steps

Your questions answered

VAULT #2: Running a redirect A/B test without 3rd party tools — by writing your own code. 5 Lessons

Expand

Module Content

0% Complete 0/5 Steps

Why?

1) Traffic allocation: sending visitors to the old/new landing pages (with JavaScript)

2) Data collection and visualization (Bash + SQL + Google Data Studio)

3) Evaluation: significance calculations (using Python)

Conclusion