How to A/B Test TV, Radio and Billboard Creative

Digital marketers test creative constantly. Offline advertisers often pick one ad and hope. But you can test ad creative on radio, TV and billboards too. It just takes more planning. Here's how.

This guide is part of our guide to measuring billboard, TV and radio advertising.

What to test

Test big differences first. Small tweaks rarely produce measurable offline results.

• The offer: 20% off vs. free consultation
• The message: price vs. quality vs. speed vs. local pride
• The call to action: URL vs. phone vs. "visit us on…"
• The tone: humour vs. straightforward
• The format: jingle-led vs. voice-led radio; 15s vs. 30s

Test one variable at a time, or you won't know what made the difference.

Step 1: Pre-test before you buy media

Cheap ways to eliminate weak ideas early:

• Audience surveys: show or play two to four concepts to your target audience online and ask about recall, clarity and likelihood to act.
• Social ads as a proxy: run the same headline or message as small-budget social ads. Differences in click-through or engagement are a rough signal of which message resonates.
• Customer feedback: play radio drafts to a few real customers.

Pre-tests aren't perfect predictors, but they catch confusing or forgettable creative before it costs you.

Step 2: Design a live test

Radio

Rotate version A and B across comparable stations or alternate weeks, with the same schedule weight. Give each a unique vanity URL, code or phone number. See using vanity URLs, codes and call tracking.

TV

Split by market (version A in one region, B in a comparable one) or alternate flights. Watch web and search response in the minutes after each airing for fast feedback.

Billboards and transit

Place versions on comparable boards (similar traffic, visibility and audience), or split by area. Digital boards let you rotate creative by day or time of day on the same screen, which is the cleanest comparison. See digital out-of-home.

Step 3: Measure fairly

• Compare responses per 1,000 impressions, not raw totals, when weights differ.
• Run long enough for a meaningful number of responses per version.
• Watch for outside factors: weather, holidays, a competitor's campaign.
• Include a quick recall survey if the goal is awareness rather than response.

Step 4: Roll out and keep testing

Shift weight toward the winner, then test the next variable against it. Over a year, a steady testing habit compounds into much stronger creative. It also helps you catch creative wear-out early.

A simple test log

Test	Version A	Version B	Channel / markets	Dates	Metric	Result	Decision
Offer	20% off	Free consult	Radio, 2 stations	6 weeks	Code uses per 1,000 impressions

Keep a running log. After a few tests, you'll have a playbook of what works for your audience, which is far more valuable than any single winner.

What good offline creative usually has in common

Tests vary by brand, but winners often share:

• Brand shown or said early and often
• One clear message
• A simple, memorable action
• Distinctive brand assets (a colour, voice, character or sound logo)

For billboard specifics, see billboard design for maximum recall. And if every version underperforms, the issue may not be creative at all. Check our diagnostic for ads that don't convert.