Scoreboards vs. Film Room: A Framework for Marketing Measurement

When the Film Room Becomes the Scoreboard

Marketing organizations are obsessed with measurement. Patrick Gilbert argues in Never Always, Never Never that the problem is not measurement itself but what we think measurement is for. Marketing Mix Modeling, attribution, and incrementality testing are powerful tools for learning. They help you review the tape. They help you understand what happened, why it happened, and what you should change next. Used correctly, they make your marketing program smarter over time. But these tools break down when you treat them like a final score: a clean, definitive verdict on whether your team or agency is good. When the film room becomes the scoreboard, the game gets distorted. The incentives change from winning games to padding your own statistics. This is where so much dysfunction in the industry comes from. When an agency's survival depends on producing short-term, attributable ROAS, their incentives drift. The work starts to tilt toward whatever is easiest to measure and easiest to claim credit for, even if that is not what is best for the business over the long run. This is not a failure of intelligence. It is a failure of framing. We took tools designed for analysis and improvement, and we turned them into a courtroom. The scoreboards vs. film room framework puts those tools back in the right place.

Enjoying this? Never Always, Never Never goes much deeper into the mental models and decision frameworks that shape how we think.

Get the Book on AmazonLearn more about the book

The Marketing Measurement Triangle

Each of the three measurement tools gives you a different view of reality, answers a different kind of question, and becomes dangerous when forced to answer questions it was never meant to answer. Marketing Mix Modeling is the 30,000-foot view. It looks at your business from high altitude and asks: when spend goes up in certain places, does the business tend to grow? MMM helps you decide how to allocate a finite pool of money across major channels and levers over a quarter or a year. But it moves slowly. It cannot tell you which specific creative is driving results, and it cannot act as a real-time optimization dashboard. Attribution is the on-the-ground view. It lives at the touchpoint level, assigning credit for a conversion across digital interactions: clicks, impressions, site visits, retargeting. That makes it an execution tool, useful for optimizing within the system you have already chosen. The problem is that attribution overvalues channels that are easiest to track, especially bottom-funnel tactics like Google Search and retargeting. Incrementality testing is the scientific view. It answers the question attribution cannot: what would have happened if we did not do this? You hold out a control group, compare outcomes, and measure the difference. It gives leadership confidence that marketing is driving behavior that would not have happened anyway. But incrementality is still probabilistic, not deterministic. Running the same test again could produce a materially different outcome.

Think Like an NFL General Manager

Patrick Gilbert draws a detailed analogy between marketing measurement and NFL roster construction to make the framework tangible. In the 2024 season, Bills running back James Cook scored 16 rushing touchdowns, tied for the most in franchise history. If you only looked at the box score, you would assume he was the most valuable offensive player. This is exactly how attribution works in marketing. The channel that finishes the drive gets the credit. But James Cook's contract represented just 1.2% of the salary cap, while quarterback Josh Allen accounted for more than 13%. Allen shaped outcomes long before the ball crossed the goal line. And sitting between Allen and Cook were three offensive linemen whose individual cap hits were more than double Cook's. They never scored touchdowns. Most fans would not recognize their names. But without them, the offense would not function. Attribution has the same blind spot. It struggles to value the work that creates conditions for success rather than claiming success outright. Brand campaigns, upper-funnel media, and creative that changes perception rarely score in attribution models. But they quietly shape outcomes everywhere else. The Bills also let their leading touchdown receiver, Mack Hollins, walk in free agency while extending Khalil Shakir, who led the team in targets and receptions. The offense was roughly twice as productive when Shakir was on the field. That is an incrementality judgment: measuring what changes because a player is active, not just who scored.

How It Works