What Happens When You Buy a Super Bowl Ad?

Posted in: In the News, marketing

It's hardly breaking news that buying airtime during the Super Bowl is expensive. Really expensive. But here's the five million-dollar-question: is a Super Bowl ad actually worth it? DataFox crunched the numbers to figure out companies actually see a benefit from advertising during the biggest media event of the year.

What happens when you buy a Super Bowl ad?

Whether you're selling diamonds or candy-coated chocolate, you're going to expect a return on your investment. But the timelines on those returns are different: you may grab a bag of M&M's during your halftime beer run, but most people don't impulse-buy engagement rings during commercial breaks. You could reasonably argue (and many do) that given the delayed impact, growth of global markets, and numerous marketing initiatives going on at any one time, you can't judge a Super Bowl ad just based on a company's reported profits that quarter. That's fair.

All advertisers, though, are chasing the holy grail: going viral. So we took a look at companies' web traffic and Twitter followers to see whether spending millions on airtime - not to mention producing the ad itself - actually translates to a bigger marketing presence.

DataFox analyzed the January, February and March 2015 web traffic and Twitter followers of 21 companies that bought Super Bowl ads last year (we excluded companies like Procter and Gamble, where the product advertised had a different website than the parent company, and companies where no traffic data was available.)

See the companies that bought ads during the Super Bowl in 2015

We figured that if a Super Bowl ad was worth it, you'd see an uptick in web traffic, Twitter followers or both in February. And if the benefit was due to the ad, you'd probably see web traffic settle back down in March, and for Twitter follower growth to subside.

As it turns out, that didn't happen: the companies who bought ads saw their web traffic actually fall in February, by 4% on average, and decrease by an additional 3% in March. Twitter follower growth was more encouraging, though, growing by 4% in February and 3% in March.

Change in February Change in March
Web Traffic -4% -5%
Twitter Followers 4% 3%

Maybe those ads get you a better social media presence, but as for web traffic, it doesn't look very promising.

But what happens when you don't buy an ad?

Looking at those companies' web and Twitter numbers doesn't answer our question in itself. February's a short month, which can skew traffic numbers; on the other hand, maybe Twitter followers are up across the board. So to really decide if a Super Bowl ad is worth it, we compared the companies buying ads to their competitors, as identified by the DataFox similar companies algorithm. For example, GoDaddy, SquareSpace and McDonald's bought ads; we compared them to Domain.com, Weebly and Burger King, among others.

What happened to those companies' web and Twitter traffic during that time frame? They saw their web traffic drop by a lot: down 13% on average in February, but up by 6% in March. And like those that bought ads, our comparison group saw modest growth in Twitter followers: up 5% in February and 2% in March.

Change in February Change in March
Web Traffic -13% 6%
Twitter Followers 5% 2%

Maybe we were premature saying that a drop in web traffic means Super Bowl ads aren't worth the money. If spending $5 million turns a 13% drop into a 4% one, most marketers would be pretty happy. As it turns out, a drop in web traffic isn't the end of the story.

Conclusion: maybe Super Bowl ads are worth it?

When you dig a little deeper, it turns out Super Bowl ads do stave off a big drop in traffic. Companies that buy Super Bowl ads have a statistically significantly better February (p=0.047) for traffic than their non-buying counterparts. It's hard to determine with publicly available information whether the traffic increase actually leads to more sales, but it's promising to know that companies buying airtime are getting at least some return.

As for social media, though, the results are less promising. There was no statistically significant difference in either February or March in Twitter follower growth, indicating that for all the money these companies spend, they aren't quite getting the social boost they hope for. That doesn't mean TV networks will be biting their nails, worried that ad revenues will dry up. The effect on web traffic is real, and might well justify an ad buy that goes for over $150,000 a minute.

It's worth mentioning, though, that it's possible to go viral during the Super Bowl without spending millions of dollars on ad spending. Years later, marketers still tip their caps to the Oreo tweet for the ages:

Power out? No problem.

Methodology

DataFox compared 21 companies that bought Super Bowl ads in 2015 (excluding companies marketing a product with a different website than the parent company, and ones where no traffic data was available) to 19 competitors identified by our Similar Companies algorithm. We analyzed the difference in web traffic and Twitter followers between January and February 2015, and between February and March 2015, using data from SimilarWeb.

Finally, we used an unpaired Student's t-test (courtesy of the College of St. Benedict's and St. John's) to see whether the average changes in web traffic or Twitter followers were different between the two groups.

Web traffic Twitter followers
% change - February % change - March % change - February % change - March
Ads -4% -5% 4% 3%
No ads -13% 6% 5% 2%
t-statistic 2.05 -1.5 0.895 -1.78
p-value 0.047 0.14 0.38 0.086