More often than the folklore suggests — but on the thinnest evidence here. Button-colour tests win 16% of the time. 81% change nothing measurable.
Cheap to run. It carries the highest win rate on this page, but on the fewest tests of any change type: the range around that number is wide enough to overlap every other category, so do not read it as the best bet available.
Small sample. Few of these tests cleared the bar. The 95% interval on that win rate runs 7% to 32%, so read it as directional rather than settled.
| Tests analysed | 32 tests |
| Beat control | 16% |
| No measurable difference | 81% |
| Lost to control | 3% |
| Median lift when it won | too few winners to report |
All 5 winners fell between +5.2% and +75.7%. Median traffic per variant was 2079 visitors. Most ran on homepages (11), carts (9), category pages (6).
Only the button's fill, border or contrast changes. The label is identical, the size is identical, the position is identical.
Adjacent categories, and how often they win: CTA copy (12%), styling (11%), layout (15%).
The button changed from a blue background and border to a different colour scheme, and a hover colour and shadow effect was added without changing the text.
The order button's background color changed from the original color to orange.
The submit and checkout buttons, plus the mini-cart link, changed from their previous styling to a green background with white text, and the mini-cart button lost its border.
The Add to Cart button’s background, border, and hover state changed, and the shipping feature pill’s background color also changed.
The button links had their background color changed to magenta or pink.
The primary button background and border changed from the original color to green.
The button changed from its original styling to a teal background, white text, and no border.
These are individual tests, not rules. Each ran on one site, with one audience, against one page we are not showing you. They are picked to be illustrative rather than sampled at random, and a change that won here can lose on your page for reasons none of this captures. Read them as prompts for what to test, not as findings to copy.
Button colour is the most-repeated tip in conversion optimisation, and it survives because the few wins are memorable while the flat results are not. A flat test gets abandoned rather than written up.
| change type | win rate | tests |
|---|---|---|
| CTA colour (this page) | 16% | 32 |
| Layout | 15% | 351 |
| Form | 12% | 81 |
| CTA copy | 12% | 315 |
| Hero image | 12% | 160 |
| Price framing | 11% | 117 |
| Styling | 11% | 1414 |
| Split URL | 10% | 1165 |
| Headline | 8% | 918 |
| Social proof | 8% | 196 |
| Body copy | 6% | 337 |
These rates are not like-for-like. How often a test wins depends partly on what it chose to measure: a goal that records a soft signal — a click, a scroll — clears the bar more often than one that records a purchase. Across this data, tests measured against a proxy goal win 13%, against 9% for tests measured against a business outcome.
Categories differ a lot in that mix — proxy goals account for anywhere from 2% to 41% of a category's tests.
Hold the metric fixed and the spread narrows sharply. Comparing only tests measured against a business outcome, the gap between the highest and lowest category falls from 9 points to 6 — at which point the intervals overlap and the categories are not statistically distinguishable.
So read the table as a description of what happened, not as a ranking of what works. If you want the comparison to mean something, compare categories that were measured the same way. The methodology page has the full breakdown.
Our data does not support a universal answer. The winning tests moved in different directions on different pages.
Only if it is cheap and you have traffic to spare. With so few winners in this category, this page cannot tell you much either way.
This is every Mida account that ran a readable test — in-house marketers, founders, product teams, and agencies working on client sites. Nothing here is filtered by who ran the experiment or how experienced they are.
Low win rates are normal in experimentation, including at the top end. Microsoft's experimentation team, reporting on its own platform, found that only about one third of ideas improve the metric they were designed to improve — and that roughly another third actively hurt it. That is a dedicated experimentation organisation with research, prioritisation and review behind every test.
A mixed population like this one runs below that. The gap is roughly what disciplined practice buys you: ideas grounded in research rather than opinion, one variable at a time, and tests built so the result can actually be read.
So treat these as a general base rate, not a target. If you would rather not close that gap the slow way, find a CRO agency in the United States, the United Kingdom, the Netherlands or Australia — or browse every region.
A win is a variant that beat its control on that test's primary goal with a statistically significant result. Tests that never got enough traffic to say anything either way are excluded. The methodology has the full detail, including what these numbers cannot tell you.
Mida.so is a super lightweight A/B testing tool to help you experiment, analyze and implement conversion strategies in minutes.
Still deciding?
A quick screen share on your actual site — no slides, no generic tour. Just your questions answered.
30 min · no commitment · no sales pressure