13
That Tuesday where my A/B test hit a 40% conversion lift on the first try
I was testing headline variations for a client's landing page and the control was beating everything for two weeks straight. Then one random Tuesday the challenger just blew up out of nowhere and I still can't figure out why.
4 comments
Log in to join the discussion
Log In4 Comments
taylor.sean6d ago
Wait, 40% on the first try and you still can't figure out why? That's wild, @kevin_west probably nailed it though, a traffic source shift sounds like the most logical explanation for such a massive swing out of nowhere. Definitely dive into your referral data for that day, man, because stuff like that usually has a simple answer hiding in the analytics.
6
kevin_west6d ago
Read a piece in Marketing Weekly that said sudden A/B wins like this are usually a traffic source shift or a bot getting filtered out. Might be worth checking your referral data for that day.
2
theagibson6d ago
Funny you mention Marketing Weekly... my cat sat on my keyboard that day and somehow doubled my CTR.
3
Wait, did the cat sit on the keyboard right when you had the campaign running? That's honestly the kind of random test I'd try on purpose if I knew a competitor was copying my moves. You should check if the weird keyboard smash landed on a target audience or device setting that totally changed who saw your ads.
8