Instrucțiuni pas cu pas

Gather Your Inputs

First, identify your conversion rates and sample sizes for both the control and treatment groups. For example, let's say the control group has a conversion rate of 5% and a sample size of 1000, while the treatment group has a conversion rate of 6% and a sample size of 1000.

Calculate the Standard Error

Next, calculate the standard error using the formula: sqrt((p1 \* (1-p1) / n1) + (p2 \* (1-p2) / n2)). Using the example from step 1, the standard error would be: sqrt((0.05 \* (1-0.05) / 1000) + (0.06 \* (1-0.06) / 1000)) = sqrt((0.05 \* 0.95 / 1000) + (0.06 \* 0.94 / 1000)) = sqrt(0.0000475 + 0.0000564) = sqrt(0.0001039) = 0.0102

Calculate the Z-Score

Then, calculate the z-score using the formula: (p1 - p2) / standard error. Using the example from step 1, the z-score would be: (0.06 - 0.05) / 0.0102 = 0.01 / 0.0102 = 0.98

Calculate the P-Value

Next, use a standard normal distribution table or calculator to find the p-value. The p-value is the probability of observing a result at least as extreme as the one you observed, assuming that the null hypothesis is true. Using the example from step 3, the p-value would be: 2 \* (1 - normsinv(0.98)) = 2 \* (1 - 0.8365) = 2 \* 0.1635 = 0.327

Determine the Confidence Level

Finally, determine the confidence level by subtracting the p-value from 1. Using the example from step 4, the confidence level would be: 1 - 0.327 = 0.673. This means that you can be 67.3% confident that the results are due to the changes made and not due to chance.

Common Mistakes to Avoid and When to Use the Calculator

Common mistakes to avoid include using the wrong formula, not accounting for the sample sizes, and not using a standard normal distribution table or calculator. To avoid these mistakes and for convenience, you can use an A/B test calculator. The calculator will do the calculations for you and provide the p-value and confidence level. It is especially useful when dealing with large sample sizes or complex calculations.

Introduction to A/B Test Calculator

The A/B test calculator is a tool used to determine the statistical significance of the results from an A/B split test. In this guide, we will walk you through the steps to calculate the statistical significance manually.

What is Statistical Significance?

Statistical significance is a measure of whether the results of an A/B test are due to chance or if they are a result of the changes made. It is calculated using the p-value and confidence level.

Prerequisites

Before we dive into the calculation, you need to have the following:

Conversion rates for both the control and treatment groups
Sample sizes for both the control and treatment groups

Formula

The formula to calculate the p-value is: p = 2 * (1 - (normsinv((p1 - p2) / sqrt((p1 * (1-p1) / n1) + (p2 * (1-p2) / n2)))))

where:

p1 is the conversion rate of the control group
p2 is the conversion rate of the treatment group
n1 is the sample size of the control group
n2 is the sample size of the treatment group
normsinv is the inverse of the standard normal distribution

A/B Test Calculator: Step-by-Step Guide