For years SEOs have relied on Google’s keyword planner tool to learn about keyword volume and to decide which topic to target. It was incredibly useful because any SEO could discover how searchers are searching, what their intent was while searching, and how often they were searching it. We could do some rough math, (maybe we get about 10-20% of that total volume if we end up on page 1 in the top 5) and create an SEO plan in part based on this data. And we could do all of this with the secure knowledge that Google was probably the best, most accurate data source out there.

Those days are gone.

My primary responsibility is SEO, but I did get Search and Display certified on the Adwords network. Being an SEO and understanding how these two channels interact with each other is actually more important than most people probably realize. For example, I was able to discover a big change that occurred in the Keyword Planner that makes it largely useless to SEOs. This is an enormous problem since many of our SEO tools use data from Keyword Planner.

Earlier this year, Google made several updates to their ad platform that aggravated professional SEMs across the industry. You may have heard griping about the switch to the new user interface, and you may have seen some frosty looks exchanged over Google’s decision to ignore your budget and spend up to twice the amount you’ve set over a 28 day period, but the one thing that truly did not get the attention it deserverd from SEOs was their switch from exact match to broad match. This is something that affects the core accuracy of Google keyword planner. 

Exact match keywords are what they sound like. If I type “new shoes” on Google, then that is an exact match for “new shoes” and any volume data I receive for “new shoes” will only show me the volume for that search. Similarly, if I type “brand new shoes”, then for an exact match that’s a completely different search, and the search volume for that is in a separate bucket. Broad match works a little differently. If I type “new shoes”, then broad match will likely add “brand new shoes” and other similar searches into the same bucket. The way this plays out is extremely deceptive for both SEM and SEO, because while Google changed the default behavior of their Ad placements to automatically use broad match, they did not tell anyone that they also changed the default behavior of Keyword Planner to reflect this as well. And luckily I can easily verify this change by comparing my keyword lists from 2016 to the new volume numbers in 2017, and comparing to Google trends data to discover if there really was a huge increase over the prior year in search volume (spoilers: there wasn’t).

If you run a report in keyword planner you have the ability to export to csv. When you do, you get the above file, which shows that average monthly searches are exact match only. This is contrary to what Google’s tooltip shows in Keyword Planner, which is the only place I’ve found any mention of a change so far, pictured below.
Notice that it says “and its close variants”. Okay, close variants such as misspellings I could understand. So how does that compare to previous volume estimates and how did that fare from one year to the next? In October 2016, my report was showing about 4400 searches per month for the search “HVAC repair”.
A little over a year later, on December 29,2017, I ran a search in keyword planner for a volume estimate on “HVAC repair”. Google insists that search volume has just about tripled, to 12,100 searches/month.
Wow, HVAC companies must be killing it this year if demand suddenly tripled on repair jobs, am I right? Well it’s super easy to verify a threefold increase in search volume with the increasingly popular Google Trends. Trend data for “HVAC repair” below:

Demand appears flat, not just for HVAC repair but across many other terms that saw massive increases on Keyword Planner as well, which makes me doubt the accuracy. “Air conditioning repair” jumped from 22,200 searches per month in 2016 to 49,500 searches per month in 2017. “AC repair” went from 14,800 to 33,100 searches each month in just a year. None of those increases are reflected in Google Trends data.

And this issue is conflated by how Keyword Planner uses buckets to group volume estimates. When Keyword Planner says there are 12,100 searches per month, it’s actually in a bucket that’s a part of a range. Maybe that search volume is more like 10,000, but it got rounded up when it entered that bucket. And these buckets have ranges that get larger and larger as the volume goes up. So it starts out small, like 2401-2900 is maybe displayed as 2900, and 5401-6600 is maybe displayed as 6600. But when you get into the tens of thousands or hundreds of thousands, the ranges are enormous and the volume estimates can be off by tens or hundreds of thousands. Now when you broad match it, the volume estimates go up and enter larger buckets.

So let’s take a phrase like “sump pump installations”, which allegedly gets 5400 searches/month according to Keyword Planner (US data only). When I compare it to clickstream data, I’m getting 0-10 searches each month. Dropping the “s” and making it “sump pump installation” returns 501-850 searches each month, which is much more in line with what I would expect to see. God knows what else they’re matching it up with.

So why is this a major problem? First, Google’s AI is remarkably stupid, and broad matches to completely unrelated search terms. We caught them matching “Pella” the window and door company, to “Paella”, a Spanish rice dish. Those two things are not related, but it assumed that the foreign spelling of a different object was a mispelling of the brand. That kind of screw up can skew your numbers badly and lead to poor decisions and wasted ad spend (you can get your money back if you notice their AI doing something like this FYI).

For residential HVAC, I shudder to think how many “ac repair” queries are being broad matched to “car ac repair” and “auto ac repair”, and as an SEO I can’t afford that kind of data pollution when deciding which pages to target for improvement.

Now I know what some of the more seasoned SEOs are wondering: how does my 2016 list compare to 2017? I had 147 keywords in this HVAC keyword list with a search volume of 146,080 searches/month in sum in 2016. When I ran the same list in December, it jumped to 269,610 searches/month. None of that increase was verifiable in Google Trends as a legitimate increase. So how accurate is Google keyword planner? Well…as it turns out, not very. 

What to do about it

First, subscribe to a tool that uses clickstream data so you have more accurate volume measurements. Moz Pro is an affordable tool and their Keyword Explorer (KWE) is fantastic. Using clickstream data they were able to estimate about 95% accuracy compared to Keyword Planner before KP flipped the kill-switch on exact match volume estimates. I’ve compared several of my keywords from old list to the volume estimates in KWE and it came out swinging.

Second, take note of all the tools you use that pull search volume estimates from Google. I love STAT, and it’s probably the most valuable tool in my SEO arsenal alongside Moz and Screaming Frog, but their volume estimates are direct from Google. Don’t trust any tools that use Keyword Planner’s data.

And third, make some noise. All SEO tools need to begin making the switch to clickstream data but it won’t happen unless it becomes apparent to the industry as a whole that we cannot rely on Google to give us vital information about their platform and users. I doubt that they will move backwards from the broad match system they’ve set up (in AI we trust) so all we can do is move forward.

Questions, comments, ideas, holes in my theories, please post them all below. Thanks for reading!

Full disclosure, on 1/1/16 there was an improvement to Google Trends data collection system, which is why many searches appear to spike in 2016. I don’t believe that this is in any way at all related to the data Keyword Planner uses, but even if there was a correlation, the increases to keywords like “HVAC repair” were less than double compared to the previous time period (2015), whereas in Keyword Planner the search volume appears to have nearly tripled. In addition, keyword planner claims to use the last 12 months of data, meaning that the October 2016 searches I ran should have included mostly data from after the “improvements” to their data collection.