Less often than most things we track. Headline tests win 8% of the time, one of the lower published rates, and 81% produce no measurable difference at all. They are also among the most-run categories, which is the gap worth noticing.
The most-recommended tactic in conversion optimisation returns less than most of the alternatives. Not worthless, just badly over-recommended relative to what it gives back.
| Tests analysed | 500+ |
| Beat control | 8% |
| No measurable difference | 81% |
| Lost to control | 11% |
| Median lift when it won | +29.1% |
Typical winning range: +16.3% to +58.7% for the middle half of winners. Median traffic per variant was 1103 visitors. Most ran on homepages (395), landing pages (159), product pages (153).
The main heading is rewritten. Structure, styling and every other element stay put.
Adjacent categories, and how often they win: layout (15%), hero image (12%), styling (11%).
| page type | win rate | tests |
|---|---|---|
| homepages | 8% | 250+ |
| landing pages | 3% | 100+ |
| product pages | 18% | 100+ |
| lead capture pages | 7% | 50+ |
| content pages | 2% | 50+ |
Only page types with at least 30 headline tests appear here.
The headline changed from online master's programs to online postgraduate programs with official recognition.
The hero headline and subhead copy were changed, and the hero image of a person with an app screenshot was replaced with a phone and laptop mockup collage.
The main heading text was rewritten in steps, and the navigation buttons were restyled and rewired while the navbar, footer, badges, and bullet sections were hidden.
The left-aligned headline and subhead were replaced with a centered mind-mapping headline, eyebrow text, and a new illustrated hero graphic with converging lines and avatars.
The plain text headline and static screenshot hero were replaced with a translated headline inside an illustrated mind-map graphic, and the CTA button was restyled as a pill shape.
It hid the existing content block and added a new centered H1 heading at the top of the section.
The modal headline was rewritten to emphasize the expiring dollar credit, and a new subline was added warning that the credit expired after submission.
The payment option label changed to a secure card payment message, the wallet icon was replaced with a card icon, and a yellow note box was added recommending card payment for box shipping options.
The headline text changed from a general message to a question about revenue from SEO, and one phrase was highlighted in red.
These are individual tests, not rules. Each ran on one site, with one audience, against one page we are not showing you. They are picked to be illustrative rather than sampled at random, and a change that won here can lose on your page for reasons none of this captures. Read them as prompts for what to test, not as findings to copy.
A headline rewrite is a small intervention wearing important clothes. It feels central because it is at the top, but it changes only what one line says to people who were going to read that line anyway. Position and visual changes alter what people see at all, which is a bigger lever.
| change type | win rate | tests |
|---|---|---|
| CTA colour | 16% | 32 |
| Layout | 15% | 351 |
| Form | 12% | 81 |
| CTA copy | 12% | 315 |
| Hero image | 12% | 160 |
| Price framing | 11% | 117 |
| Styling | 11% | 1414 |
| Split URL | 10% | 1165 |
| Headline (this page) | 8% | 918 |
| Social proof | 8% | 196 |
| Body copy | 6% | 337 |
A win rate only means as much as the thing being counted, so this is what these tests were actually optimising for.
| measured outcome | share of tests |
|---|---|
| Purchase / order complete | 29% |
| Signup / registration | 27% |
| Lead / form submission | 11% |
| On-page click / engagement | 9% |
Outcomes below 5% of the set are not listed, so these do not sum to 100. How the outcome was measured moves the win rate a lot — see the methodology.
These rates are not like-for-like. How often a test wins depends partly on what it chose to measure: a goal that records a soft signal — a click, a scroll — clears the bar more often than one that records a purchase. Across this data, tests measured against a proxy goal win 13%, against 9% for tests measured against a business outcome.
Categories differ a lot in that mix — proxy goals account for anywhere from 2% to 41% of a category's tests, and headline tests sit at 27%.
Hold the metric fixed and the spread narrows sharply. Comparing only tests measured against a business outcome, headline wins 10% (7–15%) rather than 8%, and the gap between the highest and lowest category falls from 9 points to 6 — at which point the intervals overlap and the categories are not statistically distinguishable.
So read the table as a description of what happened, not as a ranking of what works. If you want the comparison to mean something, compare categories that were measured the same way. The methodology page has the full breakdown.
It is cheap to produce, easy to explain, and the occasional big win is memorable while 81% flat results are not. Aggregate data is the only way to see the denominator.
Neither, on this data. Winners and failures both contain long and short rewrites.
Layout and hero image carry higher published rates on the same page. Be careful reading much into the gap: headline tests are more often measured against a funnel step than a purchase, and the difference largely disappears once you compare like for like. See the note on comparing categories.
This is every Mida account that ran a readable test — in-house marketers, founders, product teams, and agencies working on client sites. Nothing here is filtered by who ran the experiment or how experienced they are.
Low win rates are normal in experimentation, including at the top end. Microsoft's experimentation team, reporting on its own platform, found that only about one third of ideas improve the metric they were designed to improve — and that roughly another third actively hurt it. That is a dedicated experimentation organisation with research, prioritisation and review behind every test.
A mixed population like this one runs below that. The gap is roughly what disciplined practice buys you: ideas grounded in research rather than opinion, one variable at a time, and tests built so the result can actually be read.
So treat these as a general base rate, not a target. If you would rather not close that gap the slow way, find a CRO agency in the United States, the United Kingdom, the Netherlands or Australia — or browse every region.
A win is a variant that beat its control on that test's primary goal with a statistically significant result. Tests that never got enough traffic to say anything either way are excluded. The methodology has the full detail, including what these numbers cannot tell you.
Mida.so is a super lightweight A/B testing tool to help you experiment, analyze and implement conversion strategies in minutes.
Still deciding?
A quick screen share on your actual site — no slides, no generic tour. Just your questions answered.
30 min · no commitment · no sales pressure