Shopify A/B testing: how to run a test you can trust

Which Shopify A/B testing route fits your store?

Before and after proves little, because seasons, ads and stock all change between the two periods.

How do you run an A/B test on Shopify without an app?

Shopify's own A/B test is called an Experiment, and it lives in Markets > Rollouts on the Grow plan and up. On Basic you can schedule a change in Rollouts, but you can't run an Experiment. In the US, Grow costs $105 a month and Basic $39, billed monthly. Our Shopify review lists every plan's price and card rate.

An Experiment can change your theme, your checkout and accounts pages, your discounts or your catalogs, which hold prices. Discounts arrived on 2 October and catalogs on 23 October 2026.

Each visitor gets a group at random before the page loads, so the page doesn't flicker. You can't aim a test at phones or at a customer group.

Experiments run on your online store only. They can't carry a developer's change to your theme or run on vintage themes, Shopify's older kind.

Plan for step 6, because a rollout that reaches its end date rolls back and leaves nothing to apply.

1. Pick one change and the one number that decides it.
2. Go to Markets, then Rollouts, and click "Create rollout". Name it and select "Experiment" in the type menu.
3. Under "Changes", click "Add changes". Pick "Online store", "Checkout & accounts", "Catalogs" or "Discounts", then click "Apply". To edit your main theme, save first, then click "Edit theme".
4. In "Audience", keep the 50/50 "Traffic split", or move it. "Traffic allocation" sets how much of your store takes part.
5. In "Scheduling", set the dates. The end date defaults to 90 days out. Read any conflict message before you save, because overlapping rollouts can hide your change.
6. To keep the change, open the rollout before its end date and click "End", then "Apply changes", then "End". An applied change can't be undone. To drop it, click "Roll back changes" instead, or let the end date pass, when it rolls back by itself.

What do Rollouts A/B test results show, and what's missing?

Expect rates, not revenue. A theme Experiment reports conversion rate over time, plus bounce, reached checkout and add to cart rates. A checkout Experiment reports checkout conversion rate.

Shopify's help page lists no revenue figure for an Experiment. So a version can win on conversion and still lower your average order value. You also can't add your own metrics, or filter results by device or country.

A discount Experiment shows no revenue or orders for the discount, so it can't say if a sale earned money or gave away margin. It also counts visitors who reached the discount without qualifying for it.

Shopify's help pages describe the rates, not a verdict, so the call is yours. Only visitors who consent to tracking are counted.

How long should a Shopify A/B test run?

Two to four weeks is the usual answer, if your traffic can carry it. Shopify's A/B testing guide asks for more than 5,000 visitors to a page and at least one, ideally two, full business cycles. Run it in full weeks. Stop early and you may be reading luck. Run too long and cleared cookies and switched devices skew the result, the guide adds.

Treat that 5,000 as a floor. At a 2% conversion rate, spotting a 20% lift to 2.4% takes about 21,000 visitors per version, or about 42,000 in all. Those are our own sums at the usual 95% confidence and 80% power.

At 500 visits a week, a product page needs about 84 weeks to get there. If you can't reach your sample in two to four weeks, try other fixes until traffic grows, Shopify's guide says. So test big changes on your busiest page, or fix the leaks first.

Two Experiments at once thin your traffic further, and overlapping promotions can muddy a result too.

A test started now may not finish before 27 November, so your Black Friday checklist comes first.

How do you A/B test prices on Shopify?

A price test shows two groups of visitors two prices at the same time. Judge it on profit, not conversion, because a lower price usually converts better and can still earn less.

You have four routes. If Shopify offers your store Smart Pricing, start there.

• A catalog Experiment. In Markets, then Catalogs, build a catalog with a "Price adjustment" percentage or set prices, and assign it to a market. Keep it on "Draft", because an "Active" catalog is already live. Add it to an Experiment and choose "Set to active". It's included on Grow and up and runs only in that market. Where another catalog already covers those products, the lower price can show. The route is weeks old and Shopify lists no metrics for it, so profit is yours to work out.
• Smart Pricing. Shopify's free app runs A/B pricing experiments in early access, for select US stores on Grow or higher. It's the only one of Shopify's own routes that says "Conclusive", at 95% confidence, and shows profit. You import the winning prices with a spreadsheet file (CSV).
• A test app. The next section prices six.
• Two product pages at two prices. It works on any plan, if you add a test tool or pay a developer to split traffic. It can split your reviews and search visibility, and you may track the same stock twice.

Watch three things.

• Checkout. Changing only the price your theme shows can leave a shopper charged a different price at checkout.
• Ads. In a catalog Experiment or a Smart Pricing test, ads, listings and feeds keep the old price. Shopify warns that Google and Meta can delist products or suspend an account over mismatched offers. Pause those ads or remove those products from listing channels for the test.
• Fairness and the law. A price test can feel unfair and hurt your brand's reputation, HubSpot's pricing guide warns. Rules on showing different prices differ by country, so check yours first.

If you don't want two sticker prices live, test the deal instead. A discount Experiment does this, and so does a bundle at 12% off against 15% off. Neither changes a product's listed price, so your ads should still match, unless an ad promises that discount. Run both versions together, because 10% off one week against 20% off the next tells you about the weeks, not the offers.

Which apps run A/B tests on Shopify?

A Shopify test app can add what an Experiment leaves out: revenue per visitor, profit with your own costs, tests aimed at a device or a traffic source, and winners applied for you. These six are sorted by the cheapest plan that lists price tests.

Prices are as of November 2026.

Work out next month's bill from your own orders or visitors before you start. Ask each app who sets up your first test.

How do you A/B test offers, sort orders and badges?

You test an app's offers, sort rules and badges inside that app, because "which bundle sold more" needs revenue per offer. On some plans, Rollouts can switch an app on or off, but it can't compare two settings inside one.

Sledge is a Shopify search and filter app that also A/B tests what it puts on your pages: offers, collection sort rules, badges, pre-orders, banners and countdowns. Each shopper stays on one version in their browser.

A test takes four steps.

1. Set the item's status to "Enable", then press "Create A/B" on its row to open versions A and B.
2. Change version B, and add up to two more versions if you want them.
3. Pick "Even allocation" or "Custom allocation" in "A/B Test settings", then save, or the versions stay a draft.
4. To finish, press "Stop A/B", then "Choose" on the version you want, then "Choose winner".

The winner then gets all your traffic, and the test moves to the "A/B Test Archive", where "Start over" can run it again. The "Choose winner" dialog says the other versions will be deleted, but the archive keeps them. Back in Stock has no archive tab, so read that warning before you press.

Sledge shows the numbers and makes no winner call, so the traffic and timing advice above still applies. It can't test themes, checkout, product prices, recommendations or filters. Back in Stock needs Accelerate or higher and shows no per-version numbers yet. Revenue here is what Sledge counts inside your store, so it can differ from your ad platforms.

The 14-day free trial runs three active tests per tool, with no Back in Stock.

Which ecommerce A/B tests should you run first?

The best first test changes one thing on a page that already gets visitors and sells.

• Product page layout, on your busiest product (Rollouts, theme edit).
• Your checkout setup (Rollouts, checkout and accounts).
• A sale discount against your usual offer (Rollouts, discount change).
• 12% off a bundle against 15% (Sledge, Fix Bundles).
• A $75 gift goal against $85 (Sledge, Free Gift).
• Boosting in-stock products against boosting best sellers on one collection (Sledge, Strategy).

Reach your sample with no clear winner, and the change probably didn't matter much.