Tom Dalton :: Doer of Good

The thing with meta tags is that they are a wholly voluntary standard -- there is no universal set of rules to which the various search engine robots adhere. Yahoo and Google have both said that they will follow the generally accepted "robots.txt" exclusion standard, but neither they nor any other major search engine company has ever said they will follow a "revisit-after" or any other such inclusion tag.

Googlebot in particular crawls as often as it can, according to its internal relevance algorithm. Most of the other search engines are also already crawling as much and as fast as they can. Putting a tag to tell them to go faster doesn't make any discernible difference.

The funny example that I saw once, relative to this issue, was the following meta tag:

I don't drink beer, but I think the tag illustrates the principle well. :o)

Here's some more detail on the specific "revisit-after" tag, including how it came to be:

http://www.seoconsultants.com/meta-tags/revisit-after.asp

And that brings us to...

The Retarded Newspaper Boy

See, there are so many different SEO tactics, and it seems like there are hundreds of sites, each with their own set of suggestions -- generally mostly contradictory.

The way I look at it all is to try and understand from the search engines' perspective -- what are they looking for? Not how can we fool them, but how can we better meet their needs? And that basic need is for structured, relevant content that they can crawl, index, and correlate to searches.

See, the Search Engine crawlers are dumb. Imagine, if you will, a retarded newspaper boy. When he comes to your door fumbling with an invoice and trying to string together a sentence, you have three options:

1. Wait and wait for him to get it right
2. Gently take the invoice from him, write out a check for the correct amount, and give it back to him with a nice, cash tip that you put in his shirt pocket
3. Take the invoice and throw it away, then hand the boy a dollar and tell him to "keep the change," sending him merrily (ignorantly) on his way

Option number one is what most people do with their websites. They just build it and hope it gets ranked. That can take a long time, and even if it does get ranked, it probably won't be stellar.

Option number three is what unscrupulous (or just misled) people resort to. Tricks -- stuffing keywords into tags, putting hidden text on pages, auto-generating hundreds of pages, whatever -- can work for a while. But like the father of the retarded newspaper boy who will come and beat you senseless once he tallies the money, the search engines will ultimately penalize you for engaging in 'spammy' behavior. Most of the 'tricks' that I've seen proposed are dangerous in the long run, and the more successful they are in the short run, the more likely they are to result in more serious penalties when (inevitably) they are clamped down.

So option two is the right choice. (Yes! There is a "Right Choice!" I love it when that happens.) Understand what the search engine crawlers are looking for, and give it to them in an easy-to-handle format.

Create clean, meaningful URLs. Have links and text and standard formatting. Write good articles that use the keywords you want people to find you with. The basic, simple steps of SEO that may not have the flash and instant impact of a cloaking system, but that lead to long-term success.

One of our clients was banned from Google, before we started working with them three years ago. When we took over, we had to clean the site and go through a difficult appeals process to get them back in to Google. And then we started our normal process.

Three years later: they have over 100 top-10 rankings on Google alone. Their traffic has doubled every year, while their PPC budget has been slashed to a fraction of what it was. (It helps that we're managing that piece of the campaign more efficiently as well.)

Where can you turn for advice, then? I've found a few places that seem to generally be on the level:

http://www.webmasterworld.com -- Google and Yahoo representatives post here regularly

http://www.marketingsherpa.com -- Loads of case studies -- many not about SEO, but sometimes, and always backed up with solid data

And of course, one of the most solid, professional SEO companies in the known universe:

10x Marketing

But this page wasn't meant to be an ad for my company. I just wanted to share my analogy, mostly, and I couldn't bring myself to send it to the client who asked me the question this morning. :o)

Have a great day!