CallRail's 60-Day Holdout Test Shows How to Prove Ad Demand

A 60-day holdout test turned CallRail's encouraging connected TV numbers into evidence strong enough to justify a broader rollout. Shaun Bruno, Senior Growth Marketing Manager at CallRail, and Colin Saunders of Matter Made laid out that progression in the webinar "How To Create Demand You Can Actually Convert (And Prove)," covering channel selection, budget approval and measurement before scaling spend.

Their core warning: a promising traffic report is not the same as evidence that a campaign generated additional demand. Adding another B2B marketing channel only helps if the team can capture the interest it creates and evaluate whether the investment works.

Build conversion before reach

Bruno described how CallRail simplified its marketing mix around paid search and the landing pages needed to convert existing demand. The team then strengthened measurement, clarified its ideal customer profile, and built consideration and retargeting programs before investing further in awareness.

The sequence mattered more than copying CallRail's channel mix. "But the important takeaway is that you don't need to run all these channels," Bruno said. The bottom of the funnel had to support the demand created at the top. More reach would not repair a landing page, offer, or follow-up process that failed to convert interested buyers.

What does channel choice depend on?

Saunders distinguished between buying patterns among smaller businesses and enterprise accounts. For SMB and mid-market programs with existing search demand, paid search reached people actively researching a solution and established acquisition benchmarks.

Enterprise programs often needed more education and time to engage a buying group, so his team leaned more heavily on paid social. He presented these as starting approaches, not a universal channel split.

The decision also depended on what the buyer encountered after the click. A demo request, an educational resource, and a product video served different needs. Saunders recommended testing creative and landing-page content against the audience's stage rather than expecting one asset to do every job.

Get agreement before requesting budget

The presenters emphasized defining a test's business purpose before pitching a new platform. Saunders outlined three requirements:

• Identify the problem.
• Agree on the measurement and reporting cadence.
• Set a timeframe with a decision date.

For a lean budget, he recommended narrowing the audience or geography instead of spreading spend across a national market. A smaller test could investigate a specific segment, provided leadership understood what it could reveal and how long it needed to run.

He also stressed connecting advertising data with CRM outcomes. Form submissions provided an early signal, but the team needed visibility into whether those leads became opportunities and customers. That distinction turned a request for more traffic into a business case for further investment.

Why the holdout mattered

CallRail initially saw encouraging website visits and trial activity from its CTV pilot through LinkedIn. Bruno cautioned that an ad exposure followed by a conversion did not establish that the ad caused it. The team then ran a 60-day holdout test: one audience eligible for CTV exposure, one control group that was not, both entering the same downstream campaigns. Bruno described using the comparison to support a broader rollout, then monitoring acquisition efficiency as spend increased.

The lesson was the progression itself. Early signals justified a more rigorous experiment, and that experiment informed the next investment decision. The pilot was a case study, not a guarantee another advertiser would see the same result.

Keep attribution signals separate

Bruno separated three ways of examining marketing's contribution:

• Direct attribution: a campaign or interaction tied to a conversion, without necessarily explaining what first motivated the buyer.
• Influence attribution: earlier video, social, or website interactions that describe the path before a conversion.
• Correlated signals: changes in branded search, direct traffic, or acquisition volume as directional evidence to investigate alongside a channel test.

These views answered different questions. A rise in branded search could strengthen the case for further investigation; it was not interchangeable with a directly attributed conversion or proof of incremental revenue.

On emerging LLM advertising, Saunders described limited reporting and the difficulty of evaluating indirect effects. The presenters also treated self-reported attribution as another source of context, not a complete story.

Prospecting versus retargeting

Asked in Q&A whether starting enterprise programs with paid social meant LinkedIn, Saunders said LinkedIn was typically the starting point, while paid search still had a role. Bruno added a distinction between finding new prospects and re-engaging past site visitors: the likely location of the buyer guided prospecting; retargeting could reach that same person elsewhere. Saunders cited YouTube retargeting as one way to begin testing the channel before expanding into prospecting.

For the next proposal, the presenters' test brief covers the buyer segment and business problem, the offer and follow-up needed to capture interest, the primary outcome and comparison that informs the decision, and the budget, test window, reporting cadence, and decision date to expand, change, or stop.