How to Measure Marketing Effectiveness: A Scoreboards vs Film Room Framework

Why Most Marketing Measurement Fails

Marketing measurement today suffers from a fundamental confusion between two different activities. We've taken tools designed for learning and improvement—Marketing Mix Modeling, attribution, and incrementality testing—and turned them into courtrooms where teams fight over credit and blame. As Patrick Gilbert argues in Never Always, Never Never, this transforms powerful analytical instruments into sources of dysfunction.

The core problem isn't the tools themselves, but how we use them. When an agency's survival depends on producing short-term, attributable ROAS, their incentives drift toward whatever is easiest to measure and claim credit for, even when that's not what's best for the business long-term. This creates a measurement system that looks precise but optimizes for the wrong outcomes.

The Scoreboards vs Film Room Framework

The solution requires separating two distinct activities. Film room work is about reviewing performance to learn and optimize. You're analyzing what worked, what didn't, and what to change next. The mindset is collaborative and analytical, focused on improvement rather than judgment.

Scoreboard work evaluates teams and partners based on business outcomes over time: revenue growth, profit, market share, retention, and brand health. This includes qualitative factors that don't fit in dashboards but determine success: strategic thinking, reliability, decision-making quality, and trust.

When you treat film room tools like scoreboards, the game gets distorted. Teams start optimizing for measurement rather than results. The focus shifts from building something durable to padding statistics that look good in reports.

The Three Pillars of Marketing Measurement

Effective measurement relies on three complementary tools, each designed for different questions and time horizons. Marketing Mix Modeling (MMM) provides the strategic, top-down view for budget allocation across major channels. It's your planning tool for quarters and years, not daily optimization.

Attribution works at the tactical level, helping optimize within channels you've already chosen. Which creative performs best? Which audiences convert most efficiently? Which keywords drive quality traffic? Attribution excels at these execution questions but struggles with strategic ones.

Incrementality testing answers the scientific question attribution can't: 'What would have happened without this campaign?' Through controlled experiments, it separates correlation from causation and validates whether marketing truly drives incremental behavior.

Each of these three tools gives you a different view of reality, answers a different kind of question, and becomes dangerous when you force it to answer questions it was never meant to answer.

Patrick Gilbert, Never Always, Never Never

Why All Models Are Wrong (But Some Are Useful)

The statistician George Box famously observed that 'all models are wrong, but some are useful.' No model can fully capture the complexity of human behavior, market dynamics, competition, and timing that influence marketing outcomes. Every measurement approach is a simplification—a useful fiction that approximates reality.

This matters because modern marketing measurement tools are probabilistic, not deterministic. They estimate likelihoods and work with confidence intervals rather than providing perfect answers. MMM, attribution, and incrementality testing all fall into this category. They're decision aids, not crystal balls.

The right question is never 'Is this model true?' but 'Is this model useful for the decision I'm trying to make?' This perspective prevents organizations from demanding false precision and allows them to make better decisions with imperfect information.

Building Trust Through Measurement

True accountability requires trust, not just data. The most effective marketing organizations aren't those that can explain every outcome in a spreadsheet—they're the ones that can operate with some uncertainty and still move forward together. This means creating measurement systems that encourage collaboration rather than competition.

Trust is built through transparent communication about strategy, clear definition of shared goals, and honest acknowledgment of what can and can't be measured perfectly. When teams and vendors share objectives rather than fighting over attribution, marketing becomes more effective and measurement becomes more meaningful.

Steps