Do headline A/B tests work in SaaS / Software?

Weak on both counts. Headline tests win 6% in SaaS / Software, below the 10% these tests manage across all industries and below the 9% that SaaS / Software manages across all change types. There is usually something better to spend a test slot on.

The strongest change type we can measure in SaaS / Software is Styling at 15% on 100+ tests.

The numbers

	SaaS / Software	all industries
Tests analysed	100+	500+
Beat control	6%	10%
No measurable difference	85%	—
Lost to control	9%	—
Median lift when it won	+16.8%	—

Median traffic per variant was 828 visitors. Most ran on homepages (166), landing pages (26), lead capture pages (16).

Every change type we can measure in SaaS / Software

change type	win rate in SaaS / Software	tests
Styling	15%	100+
Split URL	12%	100+
Body copy	9%	33 tests
Headline (this page)	6%	100+
CTA copy	2%	50+

The industry overview is at SaaS / Software benchmarks; rates across all industries are on the headline benchmark.

What counts as a headline test

The main heading is rewritten. Structure, styling and every other element stay put.

Control and variant wireframe for a headline A/B test — Control on the left, variant on the right. Only the changed element is highlighted.

Headline tests that won in SaaS / Software

Numbered step describing learning outcome in 6 weeks for complete beginners, followed by a Continue action item
short call-to-action style headline inviting users to join Raintree
headline stating media storage that just works better
Changed the hero headline and subhead copy, and replaced the hero image of a person with an app screenshot by a phone/laptop mockup collage.

Headline tests that did not

Headline touts automatic vendor management that eliminates manual spreadsheet tracking, followed by a subhead describing how the product syncs accounting systems, extracts contract terms with AI, and answers questions in seconds, ending with a live demo prompt.
Changed hero layout from centered single-column to left-aligned with new headline text, added avatar social proof badge and customer logo strip, removed original body copy
headline stating quick 5-minute setup for A/B testing, followed by subhead listing GA4 integration, free forever pricing, drag-and-drop editor, and no coding required

These are individual tests, not rules. Each ran on one site, with one audience, against one page we are not showing you. They are picked to be illustrative rather than sampled at random, and a change that won here can lose on your page for reasons none of this captures. Read them as prompts for what to test, not as findings to copy.

Who ran these tests

This is every Mida account that ran a readable test — in-house marketers, founders, product teams, and agencies working on client sites. Nothing here is filtered by who ran the experiment or how experienced they are.

Low win rates are normal in experimentation, including at the top end. Microsoft's experimentation team, reporting on its own platform, found that only about one third of ideas improve the metric they were designed to improve — and that roughly another third actively hurt it. That is a dedicated experimentation organisation with research, prioritisation and review behind every test.

A mixed population like this one runs below that. The gap is roughly what disciplined practice buys you: ideas grounded in research rather than opinion, one variable at a time, and tests built so the result can actually be read.

So treat these as a general base rate, not a target. If you would rather not close that gap the slow way, find a CRO agency in the United States, the United Kingdom, the Netherlands or Australia — or browse every region.

A win is a variant that beat its control on that test's primary goal with a statistically significant result. Tests that never got enough traffic to say anything either way are excluded. The methodology has the full detail, including what these numbers cannot tell you.