Google Exposed! Don’t Trust Webmaster Tools Reporting!
January 25th, 2011 by
"The Simpsons" © 20th Century Fox Television (Don't Sue Me!)
In the middle of December 2011, Google rolled out some new changes to the Webmaster Tools. This update included search queries with top pages, organic CTR, and organic imprressions, as well as, intergrated graphs ala Google Analytics.
Since this update went live, I’ve been wondering if the data that Google is providing is accurate and how does it stand up against Google Adwords Keyword Tool and actual Adwords data.
We have a client who is position 1 for a cosmetic surgery term in a major metropolitan area. So I pulled the two sets of Google data reported and the results were mystifying.
Adwords Keyword Tool:
Click to Enlarge
Keyword Tool reports there are an average of 2,900 local monthly searches for the phrase geo-modified cosmetic surgery.
Google Webmaster Tools:
Click to Enlarge
Webmaster tool reported a mere 140 impressions! That’s a difference of 27k searches! Granted this data represents the holiday season in the US but you can not convince me that 96% of the searches stop. 10 or 15% sure, people have other things to think about during the holidays than getting bigger breast, but 96%! No %&*#ing way. Especially for a client that is in position one for the phrase.
And yes before you say it is an “approximate 12-month average number of user queries”, that would mean two out of the remaining twelve months would have to had double the search queries of the Google Adwords Keyword Tool number (go do the math, I’ll wait…).
Oh and to convince myself this isn’t some isolated event in Google’s data reporting, I pulled some additional information on another client who ranked in position one and was running Google Adwords ads during the same time frame that Webmaster tool reported on.
Adwords Keyword Tool:
Click to Enlarge
Google’s Keyword Tool reports that the local monthly searches are 1,300…
Google Webmaster Tools:
Click to Enlarge
And Webmaster Tools reported 140 impressions from Dec 22 – Jan 22. A big difference than what the Adwords Keyword Tool is reporting. But wait! The data looks even more like jelly when you compare Adwords to Google Webmaster Tool for the same date range (oh and to clarify this client has been strong at position 1 for over a year for this term).
Google Adwords:
Click to Enlarge
For the same exact date range and same exact keyword running simultaneously for the same location (this is a geo-modified keyword for a national level audience) the data is off by almost 50%! That’s failing (terribly) where I come from.
Seeing this data together is actually disheartening. If I was using Google tools to perform an A/B test and got a 50% accuracy in the data I would throw away the test. I understand Google is giving away ( most of) the data for free but there really should be some quality assurance for the Google Adwords Keyword tool. This tool should be giving advertisers the most correct data possible in order to help them create more effective campaigns and better gauge potential market growth.
At the end of the day, I wouldn’t trust Google Webmaster Tools as an authoritative source of search volume for your site. While it is collecting great data about indexing issues and missing pages, it is doesn’t give you accurate enough information to make a decision. I guess in a way, the three sources are like credit scores. If you just look at the one, you miss some important knowledge that could either be a boon or a bust.
Anthony,
This is really interesting data. Did you look at Analytics as a 3rd party source to see what numbers it gave?
Will
I thought about it that would be a good follow up post with some position click data.
Hey, Anthony.
I have also been baffled by the stats being displayed, in my case between Analytics, Adsense, and the web master tools(Top Search queries).
That’s how I ended up on your site.
I found a link that may explain some of the issues you have mentioned above.
http://www.google.com/support/webmasters/bin/answer.py?answer=35252&hl=en
Go to the bottom of the page to the Heading,
ABOUT SEARCH QUERY DATA
Just another thing.
Not wishing to be pedantic, though, you have written MIDDLE OF DECEMBER 2011 for your reference of the Google update to Web master tools.
Wishing you Success,
Daniel.
I don’t want to be too rude here. But it looks like you did a broad match for your keyword.
A broad match will show more results because it includes any keyphrase that contains part of the phrase your are looking for.
For example:
A broad match for Dating will include: Dating, I hate Dating, I love Dating, Why Dating Works, What would I do for the possibility of every being able to be good at dating.
All of the above match the broad search dating. This why you see a large amount of searches.
To remedy this, simply choose Exact on the left hand panel. Or just enter your keyphrase into the tool like this:
[dating] or “dating” instead of just dating.
Make sense? I myself made this mistake countless times. Best of luck to you.
Hey Jeff,
You are correct but the term is actually a relevantly long-tail, low traffic term (when compared to your examples). For laughs, I went ahead and exact matched the first term and despite being several month later, AdWords Tool is reporting 1,300 which is still way off when compared to the AdWords Impressions and Webmaster Tool Impression report. So while you are correct with your assertion, the hypothesis of my blog post (no matter how lazy my sampling was). Thanks for the response!