Tom Dalton :: Doer of Good

Oh. Somebody did my scariest idea for a search engine study, and they did it much more cleverly. I wanted to know -- what percent of people click on the #1 ranking, versus #2, etc. and if that varies by search engine. Which search engine is the most relevant? (That was the key question, in my mind.)

The horrible plan I was fomulating was to get access to Omniture's SiteCatalyst tracking data and measure clicks to various companies' landing pages. Then I'd correlate that data with historical records of the landing pages' rankings in various search engines. That would give me a broad, real-life snapshot. (For example: Company A got 71 clicks to their landing page for Keyword B last month. Keyword B was ranked #2 on MSN. Keyword B was searched for 1100 times on Overture.)

There is so much error in that -- from all the assumptions, from uncategorizable traffic sources, and who knows what else -- but it could be a very, very large scale study.

Professor Thorsten Joachims and colleagues at Cornell University did a much more elegant study: they tracked a sample of users on Google. They showed that 40% of people click on the first result -- and even if you switch the results, 34% of people still click on the first result.

The implication: People are strongly conditioned to click on the first result.

My response: Let me replicate that study, for MSN and Yahoo, and lets see which one gets the best score! Or which one has the most sheep-like users. Hee hee.

Read more for the full paper reference. >>

Every semester, I get a student challenging my assertion that Google doesn't really handle Flash well. "Macromedia released an SDK, and they've worked with Google!" they shout.

It's always the Computer Science kids who say this. And they're technically right, Googlebot (and the other SE crawlers) can parse Flash and read it.

But Google doesn't care about what it reads in the Flash! SE's want to see what people see. And there is no way for them to verify that people will actually see any given piece of a Flash file. It's just too complex.

A recent example highlighted this again. For a Google 'site' search of my new corporate Overlord, the Flash movie on the home page got its own entry:

================
[FLASH] w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w ...
File Format: Shockwave Flash
w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w ...
innuity.com/userimages/innuity_home_center.swf - Similar pages
================

Is that how you want your site to show up in the search engines? No? Very good. Then don't rely on Flash. Do it the right way -- hybrid HTML with judicious Flash insets, with all content duplicated in HTML elsewhere on the page.

In other news, Dido is still some really great music to kick on in the backround at work. Relaxing, electronic grooves to ease the pain of programming and solving a hundred random problems each day. Ahh...

Here's the slides for a presentation I gave at the More Good Foundation (http://moregoodfoundation.org) monthly Webmaster Roundtable. It covers the basics of SEO, with a heavy emphasis on the principles that underly the industry. Too many people get caught up in the tactics and details. That leads to stress and trouble with every algorithm change, and all too often it also leads to sites getting delisted.

A good webmaster needs to understand what the search engines want to be able to deliver it to them. That's what this presentation was designed to help with.

Search Engine Optimization

Feel free to contact me with any questions. That goes for all five of you who read this blog. :o)

I ran into a weird problem today, with a client who built a new website on top of an old one. Of course, the standard SEO advice is to keep all your filenames the same, as much as possible, and put 301 redirects when you have to move content. But this client just wanted a new site, and didn't care about (and didn't really have) any meaningful rankings to begin with.

So the old site was deleted and the new site built. The new site had all the magical SEO sparkles, but after a month it wasn't doing as well as we thought it should be. MSN and Yahoo were okay, but Google was lagging. Why?

I checked in Google (site: command) and saw that all the old pages were still listed along with the new. So to Google, this site was about 90% 404 errors and invalid content. Not looking so hot. I recommended to the consultant working on this client that he set up 301 redirects on all the old pages to the index of the new site, just to clean them out of Big G's index.

Problem solved! I'm a genius.

At least, until the consultant and the programmer working with him came back and asked the shocking question: how do you get Apache to serve all 404 error pages as 301 redirects to a new index page?

It had seemed so elegant a solution, I never worried much about the implementation. But now it was back, haunting me.

The easy way, of course, is to set up something in the htaccess file like the example TamingtheBeast.net lists:

redirect 301 /old/old.htm http://www.you.com/new.htm

But we had something like 1000 old URLs that weren't in a simple set of directories or anything, and they all had long querystrings and -- I would have had to create 1000 separate entries in the file, and I couldn't even find all the old names. Not an option.

Some poking around quickly revealed that nobody had really done this before, for this purpose. I found some sites that actually talked about serving 404s with redirects, but that didn't include the 301 I needed for search engines. And I found a couple places that talked about using mod_rewrite with a 301, but that didn't include the pointer back to the index! (So Google would see that the page had moved permanently, to exactly where it already was, but now with duplicate content from the index -- which would get the site blacklisted. Not ideal.)

As I thought about this, I also realized that we didn't really want every 404 page to 301 back to the index. That can get really confusing for people. So I started looking at the missing pages, to see if there were some consistent attribute about them that I could use to mod_rewrite before throwing a 404 error.

Fortunately, the old site had been written in ColdFusion. Every old page had a ".cfm" in it, and the new site was all php. So I wrote a mod_rewrite in the htaccess file that would use regular expressions to find any ".cfm" and load up a new php script I wrote instead. The php script sent a standard 301 redirect with the index target.

Here's the files I wrote, mostly for my own future reference:

====================
.htaccess modification:
====================
RewriteEngine On
RewriteRule \.cfm http://www.NEWSITE.com/cleaner.php

====================
cleaner.php
====================
Header( "HTTP/1.1 301 Moved Permanently" ); Header( "Location: http://www.NEWSITE.com" );

How cool is that? I'm a genius.

I wrote this for work, but we're making it public, so I figured I'd share it here, too. I'm repurposing content!

Anyway, here's the quiz. Seven common SEO tactics -- do you know which are good, and which will get your site delisted, and why?

1. Buy multiple domain names with different keywords and point them all to your site
2. Add unique meta-tags to every page on your site
3. Increase the keyword density of your pages to five percent, or twenty percent, or even fifty percent
4. You can’t get away with simple white text on a white background, but use CSS to mask the color and do the same thing
5. Detect the search engine ‘crawlers’ and show them different, keyword-rich versions of your flash or other pages
6. Put ‘alt tags’ on all of your images and other content
7. Edit your robots.txt file to include a command to “index=all; revisit=daily”

Answers are after the jump...

Writing up a final paper for my Media Management class, I found myself getting interested in the practical aspects of competitive intelligence (CI) for SEO.

I found a page that outlines some interesting "black hat" SEO tactics, with a smattering of other, underhanded ideas:

http://www.seobook.com/archives/000964.shtml

Mostly, I'm just writing this post to bookmark that page for my future reading-and-thinking. Right now, I must run to class and turn in my paper. :o)

Mobile search is the subject du jour: how cool is it?

About 1/1000 as cool as regular search, if you measure coolness by search volume.

Here are the numbers I gathered and combined to create this amazing statistic.

Click Z Mobile Stats issued a report that gave a breakdown of mobile activities, from email and search to portals and business news. I don't buy it, because every cell phone and mobile device that I'm aware of defaults to a portal and I doubt most people have reset that, and yet they list portals far below email and search. Unless every user is doing many searches and emails each time they open their phone, that doesn't seem accurate. But it's the only number I could find, so we'll go with it: 5 million mobile searches per month.

Then I got the total number of fixed searches from Nielsen NetRatings. They use a baseline number of 5 billion searches per month for their search engine market share chart which, though it is not doctrine, seems to be pretty accurate as near as I can tell.

Then comes the tricky part where I divide those numbers and end up with the 1/1000 figure. For every 1000 searches performed from a PC or other fixed device, one is performed from a wireless device.

Given that searching from a mobile device is about 100 times harder, and that people are 10 times more likely to be looking to satisfy an immediate need (and let flow the liquid of commerce) (does that sound like a bathroom joke?), mobile and fixed search are already on par with each other. But neither of those figures are true, so we're not there yet.

But we will be!

The big question is always, "What factors does Google weigh?" What matters to Google?

Of course, they're not publishing the comprehensive list with weights and details. And there's really no way to know what factors Google is actually looking at.

But a good start is to work out what Google can actually see and then figure out might be significant.

So what can Google see?

Here's my approach at a comprehensive list:
============================================
Domains
- length of domain name
- keywords in domain name
- extension (.com, .edu. .tw, .spam)
- hyphens or other nonstandard characters
- registration date
- registration duration
- WHOIS data (physical address, anonymous?, changes?)
- IP addresses referenced

Site-level
- 'discovery date' when Google first saw the site
- number of pages in the site
- number of inbound links to the site
- directory structre
- IP addresses

Page-level
- size of page
- number of scripts, images, css, other external linked/embedded objects
- title
- meta tags
- specific HTML tags (h1, h2, etc.)
- content
- keyword density
- keyword diversity
- link density
- number of inbound links
- anchor text of inbound links
- source of inbound links
- all those link elements for outbound links
- location within site
- inbound links from within the domain (relative prominence within the domain)
- update frequency
- update size
- discovery date

User Activity
- clickthrough rates
- estimated time on page
- bookmarking, social networking data

Competetive Data
- basically all these same factors for other sites/pages targeting same keywords
- update frequency, link popularity, and user activity especially

========================
That's all for now. Got any more? I'll add 'em to the list.

Look at me! I can play with new things like all the billion other Google sycophants!

This is my magical search box. Just like Google, but opt-in. Spam-free!

Just testing. I don't think this will revolutionize the Search Engine World, but it's what I've always wanted in bookmarks. I want a Firefox plugin that will automatically add sites I bookmark to my personal Google search folder.

David Berkowitz, the director of planning at 360i, offers this tip for SEO:

"Anyone can declare themselves an expert and create links to their category. Http://google.com/coop (also in Beta) offers a powerful way to get web sites on top of the search results. When users subscribe, if they search in the topic, you’ll come up on top of the natural search rankings, even higher than Google base."

I've added the emphasis on the three crazy parts of that statement.

1. Anyone can declare themselves an expert -- and how credible is a system that allows for that? And create links to their category. Impartial, expert links, no doubt.

2. When users subscribe -- and why would they do that? As we learned in point number one, anyone can declare themselves an expert. In the Internet world, this means everyone will. Actual expertise notwithstanding. SEO-minded individuals will be even more likely to create many categories because they want the Adsense revenue that (in theory) follows. Finally, the only SEO value of Google Coop is when sites over-represent themselves in their search results.

3. If they search -- which turns on the same issue as point two. Google coop is a great idea in certain contexts. Random users searching on sites they've just run across, restricting their results to whatever that site deems worthy, is not one of those contexts.

I'm excited for Google Coop, because I've always wanted to be able to restrict a search to a set of sites that I've bookmarked previously. But to restrict my search to a set of sites that somebody else bookmarked previously?

There aren't that many experts. And to espouse this as a tactic for SEO wannabes just means the proportion of experts using this tool will be even smaller. And the value of the tool even less.

So, contact me for expert knitting advice. Or mesothelioma. I've got some great search sites for you!

Businesses should start printing their logos on their roofs.

As people use Google Maps and other satellite-image based systems to navigate, buildings that look from above roughly like they do from the street will be much more meaningful. When I see a map that tells me to turn right at exit 289, I don't know what that means. Is that the exit just after CostCo?

If I could see the satellite picture and recognize that -- poof! Easy navigation. And the CostCo is that much more firmly implanted in my mind as a Good Thing.

Here's the problem with a "blog" search engine: how do you decide what's a blog or not?

Here's a Not A Blog: http://www.webanalyticsassociation.org/en/dir/?66

And that's the first result on a search for "innuity" in Technorati today.

So it's bad news for blog search engines and good news for anybody who wants to really spam a blog search engine with tons of directory entries or other lame content that hurts the Internet overall.

And that's bad news for anybody who wanted to do a Master's thesis on blog posts. And good news for anybody who wants to... make it hard for people to do Master's theses on blog posts...

And the toppings contain potassium benzoate.

Google changed its algorithm? Yahoo is using a different data source? Backlinks are being devalued if they come from sites with more incoming than outgoing links?

I don't care!

I am no longer employed by Vista/Innuity/10x Marketing/whatever. So I no longer have to care. I never really did, anyway, until they asked me to look into SEO and see what we could do. Turned out we could do quite a lot. So I spent several years checking Webmasterworld daily and testing all sorts of factors and reading horrible things like the Google patent applications.

My life has joy and meaning once again, though, as I have stepped away from all of that.

I'm going to do web development and IT stuff for the State Department. For the US Government!

So, expect to see a lot of posts about bureaucracy and big, big networks. And who knows what else.

So so happy!

I just checked my Google PageRank. It's a 3 of 10. That means it's six orders of magnitude less important than Amazon.

If I were running for class president, I'd be just behind the kid with the perpetually runny nose, in terms of popularity.

If I were running for ranks on the keyword "paul allen yacht octopus," I'd be behind a bunch of spammy, auto-generated sites that recycle each others content and participate in shady, reciprocal linking schemes. ("This link systems uses a trapezoid!")

As it happens, though I'm not really running for ranks on that keyword, I am ranked behind those spammy sites. Those shady link schemes work, by and large. For spammy sites.

Why only spammy sites? If they work for those sites, why not for your big business?

Because those spammy sites are not individually important to their owners. They are like bubbles on the ocean. Individual bubbles can come and go, but the foam remains. Any single spammer will have dozens, hundreds, or even thousands of sites that each contribute a tiny bit of revenue to the overall operation.

Businesses put their eggs all in one basket. One website. One shot at rankings and revenue. Play with fire, get burned, and go out of business.

There are safe ways to get free backlinks that won't destroy your site. And there are lots and lots of dumb ways to get links that just might completely hose you.

But, as I have said before, I don't have to care about this anymore. I'm mostly just writing this for the money. This is a Pay-Per-Post post, of the sort which I anticipate writing every couple days until I've saved enought to buy a Wii. :o)

Having just started work for the Federal Government (in all its glory and splendor) I am reminded again of the joy of drowning in strange language. Acronyms, in my case -- why write a full sentence when a handful of letters will do? IJESMME,S? (It's just ever so much more efficient, see?)

Search Engine Marketing was never really like that to me. It grew as I grew with it, so I don't remember ever having an overwhelming adjustment. But I watched a lot of our new employees struggle with it. 404 errors! Meta tags! 301 redirects! Internal links!

So sponsor of this post and Austin SEO Firm Apogee Search Marketing's search engine glossary is a good thing. It doesn't have 404 error, which is a painful omission -- but maybe that one's common enough that everybody already knows it. Strongbad makes fun of it. That didn't used to be mainstream, but it surely is now. (Unless you don't know Strongbad, in which case you need to stop reading this blog and go Google it.) Anyway, most every other jargony word you'd care to know is in there.

They even have a special entry for Matt Cutts. Cute!

So that's all I've got for tonight. Be good!

When I worked for an SEO company, we often found ourselves telling clients that we were like the cobbler's children who had no shoes of their own. So busy working for others, we had no time to promote ourselves. Also, SEO is inherently an extremely competitive industry to try and rank for.

Though I still wake up happy each morning that I no longer do SEO, I watch the industry from a distance and was interested to see that Submitawebsite.com has chosen to use paid links to promote their own site. This very post is a sponsored post, and I'm supposed to link to them with the following text:

link popularity

Of course, the best links are contextual, everyone says -- meaning they should be used in a sentence, like you learned in Elementary school:

Websites sell more product when they rank better in search engines as a result of link building.

Nobody really knows whether Google actually can (or does) look at the immediate surrounding text, or if it's just a function of what other text is in the document. And the result is vulnerable to gaming either way -- you can just use a machine-generated page with thesaurus entries surrounding your target phrase, maybe mingled in with garbled phrases culled from the Guttenberg project.

But actually getting humans to write all your content can be cost-effective and actually-effective, through sponsored links.

But I'm still glad I don't do it from the agency side anymore. Now, picking up a few extra bucks to help me on the path to my Wii... That's a different story altogether!