Tom Dalton :: Doer of Good

Have you heard this one before?

A programmer walks into his manager's office. "I've started the update. All the data is being refreshed now."

The manager asks, "How long will it take to finish?"

"A long time," the programmer replies.

"Well, how long?" asks the manager. He's wondering -- will we have this data in time to send the final report this afternoon, or should he push it off until tomorrow?

"I have no clue," says the programmer.

The manager pauses for a moment, thinking to himself that there ought to be a clue. "How many words have been refreshed already? And how long has it taken to do that?"

"Four hundred words are already done, and it's been about an hour and a half."

"And how many words are we refreshing?"

"Two and a half million. Well, no. A lot of those don't need to be refreshed. Just over two million, probably."

The manager scribbles a few calculations. "So, it will take just under a year to finish."

"Yeah," replies the programmer. "Wait, no! Er... Yeah."

The manager suggests, "Maybe we can speed that up, somehow."

After a moment's thought, the programmer says, "Well, maybe by a factor of two thousand or so... if I use the 'update multiple words at a time' function."

The manager suggests again, "Why don't you go look into that."

(Audience laughs.)

It's weird, managing technical projects. Makes me wonder how often I completely miss the real point of the project when I'm actually doing the technical stuff myself. It's hard for a programmer to really see both sides -- the technical complexities and the business needs. That's what a good manager is supposed to do, I guess. Understand both, so the right trade-offs and decisions get made.

"I wish to be useful, and every kind of service necessary to the public good becomes honorable by being necessary. If the exigencies of my country demand a peculiar service, its claim to perform that service are imperious."
-- Nathan Hale, spy for the colonial forces under General George Washington

One of our country's first spies, Nathan Hale also left us the immortal phrase: "I regret that I have but one life to give for my country." Character coupled with patriotism placed Nathan Hale among the great people who helped bring forth our nation.

In 'The Lord of the Rings,' the ring is a potent force -- and some of the men want to take it up and use it against their enemies. Tolkein's ring is a wholly evil creation, however, and the book shows the inevitable end of any who would use the ring. Power corrupts, temptation destroys. Even the pure-hearted Frodo is left empty and exiled after he is forced to carry it. Aragorn, the true king, is shown to be a wise leader because he refuses to take it up.

The magic escape prepared for the forces of men by Tolkein, however, is to take the ring and destroy it. Throw it back where it came from, and remove its force from the world entirely.

If we take the ring as a metaphor for lying, we don't have the option. There is no mountain into which we can throw lies, to remove the danger of deception from our world.

We need men of character to slip on the ring and wear it as we fight.

"And now, as Moroni knew the intention of the Lamanites, that it was their intention to destroy their brethren, or to subject them and bring them into bondage that they might establish a kingdom unto themselves over all the land; and he also knowing that it was the only desire of the Nephites to preserve their lands, and their liberty, and their church, therefore he thought it no sin that he should defend them by stratagem; therefore, he found by his spies which course the Lamanites were to take."
-- Mormon, Nephite historian and general

The scriptures teach that defense of righteousness and country are of overriding importance. I will probably never have to wear the ring, but I stand in support of those who do.

I've tried to stay out of the fray, so far, telling clients that they can pick whatever tracking system they want -- that SiteCatalyst and WebTrends both offer a good tracking system and we can work with whatever. Trying to do some conversion enhancement for a client who uses WebTrends today, though, has been so frustrating that I am forced to throw my lot in with Omniture.

SiteCatalyst, in my humble (professional, but still humble) opinion, beats the pants off of WebTrends. The default reports in SC are so much more intuitive and helpful. Every single thing I wonder about my current client's site requires me to jump through hoops to answer in WebTrends.

For instance: from the home page, which page is the most popular next page?

The default report that WebTrends gives me shows the most popular second page -- regardless of what people's first page was. How is that useful? And even from that default report, I can't figure out how to get the report that I really want. Clicking on the individual pages doesn't bring up the reports starting from those pages; it brings up those pages in new browser windows.

It's almost painful.

Another interesting (frustrating) example: when I look at the "referring domains" report, WT gives me five bars. One is "direct traffic," which drives the most by far, and then there's one for each of the major search engines. And nothing else. Nothing to indicate what percentage of the total traffic I'm looking at. When I clicked the button that I thought might expand the report and give me all the information, it "minimized" it for me, instead.

Improving conversion rates should be based on data. When that data is reduced to a few, colorful bars, my recommendations are similarly reduced.

I'll spare you the rest of the examples. If you want to switch to Omniture, give me a call! We can hook you up for a good price, install it for you, and help you make good decisions with the information.

One of the problems I've been pondering for a while is how to automatically arrange a set of words into the logical order.

"New Restaurant York," for instance -- should probably be written as "new york restaurant."

How do we get computers to know that? One idea is to just run each possible arrangement of words through Google (in quotes), and see which comes up with the most results. But that uses so much bandwidth that Google will shut down any IP address pretty quickly, if you need to test more than a handful of words.

However, Google Suggest provides the data we need, much more quickly and without formatting it into a messy html page.

http://www.google.com/complete/search?hl=en&js=true&qu=tom+dalton

I wonder how closely they're watching this link for traffic surges. They've got to allow more queries on that address than the main Google pages. It's an interesting option, anyway.

I still think the best approach, though, is to spider the net and make my own database to work from. And there are so many other cool things that I could do with that data.

How do you reduce an article or other selection of text to a concise list of its important keywords?

For instance, from the above sentence, I probably only need the following words:

reduce
article
selection
text
concise
list
important
keywords

That could easily enough be done, I suppose, with a list of words to remove or exclude from consideration. But another problem comes up with different forms of words:

selection, selected, selections, etc...

That's where "Word stemming" comes into play. Each word can be reduced a core set of characters. "Select" migt be a good 'stem' for all the words listed above.

Martin Porter developed an algorithm to perform this kind of word stemming over 20 years ago. It's been tested and refined, and he's published many specific implementations of the algorithm in various languages.

http://www.tartarus.org/~martin/PorterStemmer/

Groovy beans.

But here's where I see a need for refinement: proper nouns. I don't want to see Microsoft stemmed to something else. Or IBM, or other company or people names. Maybe simply adjusting the process to skip capitalized words...? But then how do we account for words at the start of sentences? And what if Microsoft is at the start of a sentence?

I can see that I must think more on this issue. And maybe search more, to see who else has already solved it. I love the Internet!

When all you have is a hammer, everything looks like a nail.

When all your ad agency has is a hammer, and they get paid thousands of dollars every time they swing that hammer -- they don't go looking for better tools.

Traditional advertising agencies have a hammer that pays them well. Television and big, creative, expensive, in-your-face-edgy media campaigns. The shiny sparkle that drives cars through rainforests and hurls executives off cliffs in parafoils. Why do we see this so much?

1. Most agencies get paid a percentage of the amount they bill for media spending.
2. Most people in ad agencies want to be filmmakers and artists.

Note the conspicuous absence of number three: TV ads are wildly successful. In fact, in terms of overall ROI (Return on Investment, or bang for the buck, or whatever) TV is almost the worst possible advertising medium.

The Internet is not nearly as cool. I mean, really. I'll admit it. Even Nike.com -- the king of bloated bandwidth requirements and edgy-cool flash -- is pretty lame, compared to a cool TV spot. (Though, frankly, a lot of TV spots are pretty lame these days, too.)

But "lame" shouldn't matter as much as "effective." David Ogilvy, ancient prophet of advertising, said "if it doesn't sell, it isn't creative." The only TV ads that ever sell are those call-in-30-seconds-for-a-free-squeegee spots. If TV remote controls could store credit card numbers and provide a "buy now" button, TV would be a great sales channel. (Get it? Channel? Har. Shoot me.)

On the other hand, a decent website can be an incredibly powerful sales tool. It can qualify, pitch, follow-up, close, and actually process a sale. Or hundreds of sales simultaneously.

And let's think about cost for a second. The simple production cost of a single TV ad rivals the most expensive of websites. And then a TV ad costs even more to air! A website just sits there, quietly selling for an almost negligible monthly fee.

If you're using an ad agency that wants you to invest in well-produced radio and TV spots, or full-page magazine ads, that's great. But understand that your agency is inherently conflicted when it comes to your website. And your website deserves more attention than it's probably getting.

In a recent meeting, we outlined our strategy for a local search feature we wanted to deploy.

"But... That's just like what Google Local does!" came the startled response.

"Yes."

"..."

"So, they think it's worth investing huge amounts of money in. They're going to popularize the concept and create the space. They will even train people how to use the technology! That's fantastic!"

Then came the saddest words I ever heard: "But, we'll be competing with Google. Why even bother?"

I used to feel the same way, actually. I'd come up with an idea I thought was interesting, and start playing with it. I'd churn it over and over, refining it and imagining how I'd implement it. Then I'd go look and find that someone had already done something like it. Disappointed, I'd move on.

Looking back on it, I think I was mostly looking for an excuse not to actually try the idea. But more prominently, I think people generally believe that an idea must be 'disruptive' and totally novel to be worth pursuing.

I don't buy it. In that same meeting, someone else joked, "Might as well compete with Microsoft!"

Bring it on! Look what Apple has done. Look what Novell is! Sure, these companies are not nearly as large as Microsoft. But they're bigger than your company! I would be very happy to be company #4 in a list that included companies like that.

(Now jumping to a related, but different issue...)

Bigger companies have significant advantages in marketing muscle and development resources. But a single, skilled programmer and a clever marketer can respond much more quickly to market forces and bring products to life in weeks and months, not years and (think Longhorn) decades.

I read a report recently that talked about Wal-Mart's drop in efficiency, as it has grown. "Play in your own backyard" was the general idea of the article -- it suggested that most companies lose most of their efficiency as they grown beyond a certain point. Their only competetive advantage at that point is their size.

So why does every company keep pushing for the billion-dollar score? Or trillian, or whatever. Why can't a company simply pick a size it wants to be, then work on being the most efficient, effective company of that size that it can be?

We don't have to be Microsoft. We can live quite happily in the shadow of Google. We can provide productive jobs for employees and a positive, happy working environment. We can create quality products that improve the lives of the people who use them. We can stay efficient and profitable, without pushing everything so hard that individual people start to lose meaning.

Where can a business provide the most real benefit to society and the economy? 'Grabbing a bigger slice of the market' is a zero-sum goal. You can only succeed by hurting others. 'Growing the market' -- sometimes. If you lie, play on negative emotions, or promote bad products, then you're not exactly helping society move along. Sure, it can be profitable. But it's not good. 'Making a better, good product' is what I feel most good about.

So, I ask myself, is the I-Pod good?

Hm. I'll have to think about it.

And what sort of bigot am I? Thinking I know what's GOOD for society. Geesh.

How can you make more people do what you want them to, on your website?

Here are some ideas, ranked in order of effectiveness as measured by Lauren Freedman, CEO of the e-tailing group:

Seasonal promotions: 94 percent
E-mail as merchandising vehicle: 91 percent
Keyword search: 91 percent
Sales or specials: 90 percent
Cross-sells: 84 percent
Free Shipping: 79 percent
What's New: 78 percent
Up-sells: 75 percent
Advanced Search: 70 percent
Affiliate Program: 68 percent

Great stuff. Expect to see this implemented in my company. (Because, as it happens, I'm working on our conversion rate enhancement 'product.')