how many conversions before you call a split test done?

dirtyFunnel · Mar 23, 2026

genuine question for anyone who has run serious a/b tests on landing pages - what's your actual threshold before you kill the loser and move on? i keep seeing people online say "wait for statistical significance" but that's vague as hell. i personally don't touch a result until i have at least 300 conversions per variant and p-value under 0.05, sometimes push to 500 if the gap is under 5% between variants. feels like most people are pulling the trigger way too early and fooling themselves.

also curious if traffic volume changes your thinking. like if you're getting 50 conversions a day vs 500, do you still wait the same number or do you cut it by time instead? i've had tests where the winner flipped completely at week 3 after looking obvious at week 1. cost me real money trusting early data. what's the minimum you actually stick to?

tazarbm · Mar 23, 2026

are you 2 guys the same person?

https://www.blackhatworld.com/seo/f...-content-tired-of-getting-this-wrong.1802559/

Because you are awfully similar in many join date, number of posts and likes, content structure, etc. to be just a coincidence.

Any why did you use em dashes??

UrbanNinja01 · Mar 24, 2026

dirtyFunnel said:
genuine question for anyone who has run serious a/b tests on landing pages - what's your actual threshold before you kill the loser and move on? i keep seeing people online say "wait for statistical significance" but that's vague as hell. i personally don't touch a result until i have at least 300 conversions per variant and p-value under 0.05, sometimes push to 500 if the gap is under 5% between variants. feels like most people are pulling the trigger way too early and fooling themselves.

also curious if traffic volume changes your thinking. like if you're getting 50 conversions a day vs 500, do you still wait the same number or do you cut it by time instead? i've had tests where the winner flipped completely at week 3 after looking obvious at week 1. cost me real money trusting early data. what's the minimum you actually stick to?

Your approach sounds pretty solid—waiting for enough data and not rushing decisions is key, especially when results can flip later. In general, many people aim for at least a few hundred conversions per variant and look for clear separation, not just early trends. Traffic volume does matter though, so it’s less about fixed time and more about reaching a reliable sample size before calling it.

dirtyFunnel · Mar 26, 2026

lol the em dash thing is a dead giveaway, chatgpt loves throwing those in and i caught myself doing the same thing when i got lazy and let ai draft my posts without editing. ran a content audit on my own site last year and flagged like 14 posts that all had the exact same punctuation patterns, same paragraph length, same structure, looked like they came off an assembly line. different accounts, same writing fingerprints is a pretty classic tell at this point

how many conversions before you call a split test done?

dirtyFunnel

Newbie

tazarbm

Elite Member

UrbanNinja01

Newbie

dirtyFunnel

Newbie

Main Menu

Marketplace

Making Money

BlackHat World