(Almost) Everybody Hates MMMs
But is the ad industry doing this to itself?
In media circles, you hear it all the time. These clients, god they’re in love with their MMMs. They use them for everything. They can’t make a decision without them.
But you rarely hear any media sellers asking the question - maybe this is partially our fault?
To clarify, I’m talking about marketing mix modeling, which has actually been around for decades, but has been reinvigorated over the past few years. I remember at my first agency job in the late 1990s when brands like Johnson & Johnson would hire companies such as Millward Brown to help them evaluate their media spending from a very macro level on something like an annual basis.
These systems soon became a lot less practical for vehicles like, say, the internet, social media, or the world of always-on media planning. But thanks to recent innovations in technology and research, MMMs have become revitalized - and marketers use them far more frequently.
As Dan Larkman, CEO of Keynes, explained it, there are really now two kinds of MMMs: traditional high-level models, which are “usually run by blue-chip companies…they tend to take about six months to run, and there are a lot of assumptions made within them.”
Micro MMMs, on the other hand, “allow for shorter tests, and they give you more data points faster.”
Either way, not everyone loves them. (maybe in part because brands work with vendors like Haus and LiftLab directly - or the MMMs run by Meta and Google).
Last December, at the Modern Media Summit produced by Sabio, former PHD executive Carol Castillo-Frucher had some unkind words to say about MMMs, which she viewed as a brand crutch. “It’s always annoying to deal with the MMMs,” she said. “I feel like it does what they want it to say…it flows to the narrative leadership wants.”
Even as they pledge to offer more speed than the old days, they are still viewed by many as slow and blunt. Alison Levin, president of advertising at NBCU, implied that MMMs may be even holding TV advertising back from operating at the metabolism of digital platforms.
“It depends on what you’re trying to to prove,” she said earlier this week on the Next in Media podcast. “I think the missing piece has been getting that data feedback loop and not relying on MMM, which could take six months or a year to understand to get smarter and to make those decisions,” she said.
As part of my ongoing Next in Media series leading up to Cannes with my partners at Vizio, I had Javier Bustillos, VP, Integrated Media & Brand Analytics, Georgia-Pacific on to talk about all things data-driven TV. I wondered why a marketer who sells products that are seemingly aimed at everybody (toilet paper, paper towels) would need to lean into such refined analytics.
Bustillos indirectly indicated that maybe the increased use of MMMs in the industry is the result of media companies forcing brands’ hand, by walling up all their data, just like the duopoly they’ve long complained about online.
“We’re talking about the challenges of truly having a cross-channel measurement,” he said. “In the absence of having kind of that unified measurement approach, then marketing-mix modeling is still kind of a robust way of measuring different marketing tactics from national media to retail media to coupons and consumer promotions. So we do rely on MMM as probably that tool that gives us kind of the broadest view of all of the marketing activity and the marketing investment that we’re doing and the ability to compare between different tactics.”
That generally makes sense. The problems come when companies use MMMs to make all of their on-the-ground media buying decisions.
“MMMs are great from a macro view. They are not great from a micro-optimization view,” said Larkman.
On the plus side, “they are a great way of looking at how each channel is performing on incrementality.” Larkman added that the right MMM helps brands and their CFOs to not over-index toward mobile apps like Instagram and Google. “They are great for getting people out of that click world,” he said. “Being able to look across different devices is crucial. Being able to look across different channels is where MMMs really make a difference.”
Yet you hear all the time that MMMs aren’t good at capturing the ‘fuzzier’ metrics - such as the brand halo from being on a sporting event on TV, or the cache from working with the right creator. Even retail media networks complain -even though these companies seemingly have the best attribution data.
“MMM is a statistical model, right? So it’s not perfect,” said Bustillos, who noted that his company also employs a variety of other research and data sources. Still, the plan is to go from annual MMM reports to quarterly. “We are incorporating more channels and more tactics that we can model in MMM, and we’re doing it more granular as well. So we have kind of data breakouts by audience, by device, by publishers or partners, by ad formats. So kind of all of those changes, more frequent, faster, and more granular allow us to make better and faster decisions using MMM data.”
Thus far, that hasn’t been the strength of these tools, said Larkman. “The con for CTV is that there is a lack of granularity. An MMM is going to show you whether CTV was successful as a channel: yes or no.” That’s regardless of whether a buyer is running a campaign on a few hundred TV networks, platforms like Instagram and YouTube, through various audience providers, devices, etc.
“For CTV, MMMs should be a calibration tool,” said Larkman. “Right now, the problem is that brands are using it as the be-all and end-all tool.”
And that is unlikely to stop any time soon, as outcomes rule (or an outcomes-driven mentality does).
“This industry has exploded,” Larkman added. “You can run the same tests across two different MMMs and have completely different outcomes. Then it’s like, “Which one do I trust?”
In a business that is trying to turn every tactic into an exact science, most likely you trust the one that makes your campaign looks the best - or the one the CFO likes.