What Is The Perfect PPM Sample Size?

May 23, 2017

That’s just one of the questions we asked Nielsen Audio Managing Director Brad Kelly in his May 22 Radio Ink cover story interview. A lot of industry revenue, and a lot of industry jobs, ride on the ratings. In the top 50 markets those ratings are determined by the Portable People Meter.

In a perfect world, there would be an infinite number of meters floating around the markets you operate in. In a perfect world, all the readings would be 100% accurate. And, in a perfect world, you wouldn’t have to pay a penny more for many more meters and 100% accuracy. We’ll never live in a perfect world, of course, but Brad Kelly is working hard to edge us closer. Here is part one of a three-part series previewing our cover story interview with one of the most important employees, certainly for radio, at Nielsen.

Radio Ink: Are you happy with where the sample size is today?
Brad Kelly: Nielsen has committed to an additional 10 percent sample increase in PPM markets that will begin to roll out in June of this year and will continue over 18 months. All markets will receive that 10 percent bump, and it will be proportionate, placed across all demographics. This was my “honeymoon ask” as the new managing director. Sample size has been and continues to be a top industry priority, and it’s for a good reason. Audiences are fragmenting, and the data is being scrutinized at increasingly discrete levels. This sample increase helps with stability and data-mining granularity.

This was no small ask. It was an extraordinary investment. It’s not a one-and-done proposition, where we write the check and move on. I’ve committed the company for the foreseeable future to continue to fund and underwrite this sample increase. This is the second PPM sample increase that Nielsen has funded, the third since PPM was launched, so, collectively, with the initial 12 percent bump, the 7 percent bump after the Arbitron acquisition, and now this additional 10 percent, we are north of 30 percent additional sample since PPM was debuted. By the time this newest rollout is done, we will have 80,000 PPM panelists walking around the country wearing the meters. So it’s a big deal.

Radio Ink: What is the right number? Is there a goal number? Yes, it comes with a cost, but in a perfect world, what would every market have?
Kelly: I don’t think there is an answer to that. It’s an economic equation. That curve of reliability vs. cost, and you’re trying to find the optimal point on that curve where you have the maximum reliability at a cost that makes sense for everybody.

Is more sample better? Sure. Would it be great to double the sample, triple it? Sure. Is anybody going to pay for that? Of course not. We realize the solution is not these incremental sample bumps, although they certainly help. In the end, what we believe in is the combination of high-quality representable projectable samples with an overlay of big data, which we are also working on.

Between our newly announced relationship with NextRadio and our $560 million acquisition of Gracenote, with its 100 million car infotainment systems, we believe that combination can be extraordinarily powerful. What that gives you is what amounts to settop-box-type data for radio. When you get into sample sizes in the millions, that makes a difference.

Now, big data by itself is not the answer, and the folks at GraceNote will be the first to tell you that. I met with them, and they popped up a dashboard. We were looking at a near-real-time drive-data feed from 1.5 million car infotainment systems, which was telling me the top-played song in the last 15 minutes was the new Adele song, with 4,000 spins.

I said, “This is amazing,” and they said, “Yes, but take it for what it is.” It’s coming from higher-end vehicles, from newer vehicles. It’s a skewed look, so you can’t project it out and say it represents the population. However, when you start getting into tens of millions of data points, you start to blend that with a representative projectable panel like what we have now — and we work hard on the PPM side — the two collectively will give us the smoothing and the granularity the industry seeks.

Radio Ink: Do you have any issues with AM stations? A PD told us he believes that his older listeners do not want to carry a PPM around.
Kelly: The data doesn’t support that. PPM carry by demographic is solid, particularly in the 55-plus, 65-plus age group. They’re not doing us a favor — we’re paying them to do it. A lot of folks appreciate the opportunity to have their opinion count in the process.

The younger demographics are a little harder. My daughter is a great example — the 18-year-olds. It’s hard to get her to do anything. It doesn’t really matter how much you beg. That situation is improving and getting closer to where we need it to be for those tougher younger ethnic demos.

Radio Ink: Do you see a world where radio is eventually continuously measured like other media?
Kelly: I do. An approach that I am personally advocating is the idea of taking all markets to continuous measurement and monthly data delivery. Not just the PPM markets, but all diary markets as well. So having diary-based market research in the field 12 months out of the year, and each month we would deliver a new data set.

The question is, what would we be delivering? I am not suggesting a single month’s worth of audience data. That sample would be too small and not particularly usable or reliable. But rather, in the smallest markets, the currency ratings would be a 12-month rolling average. So each month a new data set comes out; you pick up a new fresh month and you drop off the 13th month. And a month later you do the same thing, and the cycle is continuous.

In the slightly larger diary markets, it would be the same basic idea, with monthly measurement and delivery, except with these markets it would be delivered as a six-month rolling average. A tighter snapshot. In the largest diary markets — that’s roughly markets 51-110 — the currency ratings data would essentially be what Arbitrends is now, a three-month rolling average. You pick up the new month of fresh data and you drop off the fourth month.

Now, taken collectively, what that gives you is continuous measurement in all markets. But more importantly, it will give you uniform monthly data delivery across all radio markets. Fresher data and greater stability are huge benefits to this approach. But the real key to this whole idea, and the reason I am pursuing it, is that the diary-measured markets will then sync up with the delivery cadence of the PPM.

Why would we want to do that? If the industry adopts it, now all of a sudden all radio data, not just PPM data, but nationally, all radio data will feed more seamlessly into the Marketing Mix Models. These ROI analysis models are so critically important to the big CPG brands and major advertisers; they’re telling us that radio audience data the way it is now doesn’t flow through their models very well. They say running 6-month-old data is like trying to drive a car by looking in the rear view mirror.

We need a more frequent cadence of radio audience delivery, not in just 48 markets, but across the country in all markets. It’s something I feel strongly about that’s starting to get real attention at Nielsen. It’s a forward-looking industry-level initiative that occupies my time now. We are a ways from having all the details ironed out, but I believe there is a big opportunity here for the radio industry.