John Andrews is a Competitive Webmaster and Search Engine Optimization Consultant in Seattle, Washington. This is John Andrews blog on issues of interest to the SEO community and competitive webmasters. Want to know more?

johnon.com  Competitive Web & SEO

Automated Rank Checking: Thanks for Helping, Google

I want to publicly thank Google for helping me show my corporate clients that they should not be running or trying to run automated rank checking tools.

Just about every client asks for a tool to automatically check position in Google. and most big companies insist on it. So that page at Google.com that clearly states “automated rank checking is illegal“,  is a great resource. It helps me help Google eliminate the automated queries.

Also that part about how running automated rank checking software carries negative consequences,like penalization or even banning in some cases, is awesomely helpful.

It’s really important that Google help us help them like this. The need for standardized reporting of the success of SEO efforts is very real. As an SEO and competitive webmaster consultant, I give my clients the best advice I can give, and I back it up with real world facts and reasonable explanations. I have been at this SEO Game for many, many years. I have experience.

I as one of the early webmasters punished by Google back in the late 1990’s  for running WebPosition, one of the first useful reporting tools. I think it was version 2. I used it from the corporate network, and the corporate network got banned from using Google.com. That’s right… all 1000+ employees got dead air when they tried to go to Google.com. It was a very good thing that not many corporate users knew about young Google at that time, or they would have been more upset than they were.

The email response I got from Google back then was brief, rude, and direct: you’re banned because you used “one of those automated tools”. He wouldn’t name the product, but he was happy to tell me there was little I could do to fix the problem unless I begged for forgiveness and promised not to ever do it again.

Lucky for me at that time I had a large enough IP block and I knew all about proxies. The boss was adamant about reporting on position. In fact, my career advancement depended on my being able to show a rather large group of busy scientists that we were achieving exposure for their work in search engines. If I didn’t show that, they would not collaborate.

Anyway, about that page at Google.com where I can send my corporate clients when they ask for a rank reporting tool… I seem to have misplaced the URL. Anyone have it handy?

I know about this one that meekly asserts you shouldn’t use checking tools:

Google’s Terms of Service do not allow the sending of automated queries of any sort to our system without express permission in advance from Google. Sending automated queries absorbs resources and includes using any software (such as WebPosition Gold™) to send automated queries to Google to determine how a website or webpage ranks in Google search results for various queries.

but the last time I sent a client to that page she sent me back a quote from the same guidelines:

Google prefers developing scalable and automated solutions to problems

Ha ha no, the irony was not lost on me. I didn’t enjoy that conversation so much.

Anyway, I seem to have misplaced the URL to the page that says “DON’T RUN AUTOMATED RANK CHECKERS OR WE’LL BAN YOUR ASS“, which is the tone I remember from that email I got from the Google engineer backin ’99 or whatever.

We have to remember, when a client asks for reporting or rank positions, they are seeking accountability. The only answer they will accept is something that impacts the bottom line, which in this case, is Google rank. Without risk, why not go for the reward?

★★ Click to share this article:   Digg this     Create a del.icio.us Bookmark     Add to Newsvine

9 Responses to “Automated Rank Checking: Thanks for Helping, Google”

  1. TheMadHat Says:

    eh, screw them. it’s called PPEO and flipping IP’s every 15 minutes and/or 999 requests. They can try and bite my ass but it won’t do them much good :D

    @Aaron: there is long running discussion of the issue, which is perhaps why it can be disappointing to see Google strong arming its own users instead of being smart about it. Talking heads of SEO can argue that Google can’t penalize a site for checking rankings, or else one could cause one’s competitors to be penalized. That’s not really true in today’s world of Google Trust… Google can most certainly accept automated rank checking as an indicator of intent, and use it to score domains for consideration for moderation or closer review/less trust. It wouldn’t take much of a statement to this effect to deter a large percentage of automated querying.

    Google’s deceptive mannerisms may be catching up with them. They may have been hiding things in the dark so long they are unable to be honest without that honEstly highlighting their past deceits and embarassing them. We all know what happens when that situation appears…

  2. Demerzel Says:

    IMHO: If you aren’t checking, then you aren’t getting good data for analysis.

  3. Hypotheek berekenen Says:

    It has to be said that Google is sending email with a nicer tone these days. And also in action there behavior is much more civilized. For example the now don’t completely block your adsense account but just the domain they don’t like ; )

  4. Jaan kanellis Says:

    I dont think any page on Google says “automated rank checking is illegal”. On the TOS:

    http://www.google.com/accounts/TOS

    it does say this:

    5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google. You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services.

    @jaan: yes you are correct. I was being sarcastic and cynical. There is no URL like I describe. I think if Google wants to deter automated rank checking, it needs to make a clear statement to that effect. I also think that unless they threaten performance penalties (rank & inclusion)  no one will listen, because rank checking is a fundamental evaluation of performance.

  5. Travel50 Says:

    Uhmm, if you send automated queries to google they will ban your ip address (if you send enough).

    However, I don’t think it would be fair to ban your website because then all someone has to do is go to a cyber cafe, set up all sorts of data for a compeitors website, then sit there sending automated queries non stop until their competitor’s website is banned.

    Doesn’t sound like the google way to do things.

    If there’s some way to link your ip with repeated queries and your google webmaster account, then that’s a different story.

    But ranking stats are very very important, especially if you work in SEO for a company and they ask you to produce a report… manually… could take forever. There has to be some middle ground there.

  6. James Boyer Summit NJ Says:

    They really will punish for doing this. What if a competitor constantly runs automated searches on the google possition of your domain??

  7. Utah SEO Says:

    If you’re checking rankings then they can’t ban your website. How would they know what website to ban? You check rankings by submitting a keyword and then scanning the page to see if a certain url exists and where it ranks if it does. But there are several other urls there, and Google wouldn’t know which one is of interest to the app doing the querying, if it’s any of them. This is different if you are doing a “site:” or “link:” request since then you are asking for Google data on a specific domain.

    However, I don’t think Google has ever targeted specific domains due to automatic querying. If anything, they would only ban the ip address of the machine doing the automatic querying.

    My question is about the part in 5.3 of the TOS that says “unless you have been specifically allowed to do so in a separate agreement with Google.” Does anybody know of a company that has ever succeeded in creating this type of “separate agreement”?

    @utah: stepping back, and looking at the big pictuee, automated rank checking stands out as quite different from normal usage (as does proxy use, liking, etc). On average, IPs don’t search that much in nomal mode, and kw’s making up sets do not get searched uniformly as they do as part of rank checking. In general, humans dont often go to page 2,3,etc. -=john

  8. TheMadHat Says:

    @Utah

    It would be fairly easy with the way 99% of commercial rank checking software works. If you’re querying say 200 terms, and stop on the page your website shows up, it will be a simple math problem to figure out which domain you’re “most likely” looking for.

    Build your own, and don’t stop when you find your URL. Keep it running and flipping IP’s to confuse the engines. It’s under the radar and they’re only really going to go after massive abusers (WPG types).

  9. SEO Magician Says:

    The sad truth is that some clients insist on ranking reports. Thgouh I do have some that look at a few terms themselves and if they are on page one life is good. It is always safer to run the least number of terms possible. Also, don’t run full reports more than once a month and split the words up so you are running a few smaller batches.