An A/B test costs more than it looks. Someone designs the new version, someone builds it, and then you wait weeks for enough traffic to get an answer. If the new version loses, all of that is gone.
Many tests lose for reasons nobody needed traffic to find. The new version had a button that didn't work on a phone. The price moved below the first screen. The form asked for one more thing than the old one. The test didn't measure your idea. It measured a bug.
The boring reasons tests lose
Here are the usual ones. None of them are about your idea — they are about how it was built, and each can be found in an afternoon, not a month.
- A button or link that works on desktop and not on a phone
- A price, fee or shipping cost that moved or disappeared
- A form with a new box, a stricter rule, or an error that clears what was typed
- Words that promise something the next page doesn't deliver
- A layout that pushes the main button below the first screen
Check both versions before you split traffic
Before the test goes live, go through the old version and the new one the same way: same device, same task, every button and form. Look for anything the new version breaks that the old one didn't.
If the new version has a problem the old one doesn't, fix it first. Otherwise the test will tell you the idea lost, when really the build did.
When to skip the test
Sometimes a side-by-side check makes the answer obvious. If one version hides the price and the other shows it, you don't need three weeks of traffic to learn which is clearer.
Save your traffic for questions a check can't answer, like which offer makes more people buy. Those are the tests worth waiting for.
How a Run fits in
Paste the link to each version, or to a Figma or Framer prototype if it isn't built yet. A Run goes through the whole page on desktop or a phone and hands back a Run Sheet with every problem pinned to the spot and the change to make.
Compare puts two pages side by side in one Run, so you see what each version gets wrong. Then launch the test with two versions that both work.
— FAQ
Questions people ask
- Why do A/B tests lose?
- Often for boring reasons: the new version had a broken button on mobile, a hidden price or a harder form. The test measured the bug, not the idea.
- Can I check an A/B test variant before it goes live?
- Yes. Go through both versions the same way — same device, same task, every button and form — and fix anything the new one breaks. Compare in Prior.Run does this side by side.
- Does Prior.Run replace A/B testing?
- No. A Run finds what's broken so both versions work. An A/B test then tells you which one sells more.
- How long does it take?
- Under an hour — usually about 45 to 55 minutes.