July 23, 2006

Directories, Search, and Link Spam

In my searches for adding new features to our business blog directory, I was amazed to find that most directories do nothing to limit link spam. For example, a click to business Blogs on Bloggernity leads to some of the following:

1. RSS feed links versus static links to the Blog itself

2. Adult oriented material. Is this the business Blog category?

3. Dictionary article-like submissions

Yet another search from Blogflux (Category Business)
1. First result is "The Practice of Leadership", has one month of postings, all of which are syndicated articles that aren't the author's!
2. The first obvious non-English website caused my browser to close due to all the heavy images and active x controls

These are just a few examples, but the point is, you don't need to go far to encounter it.

OK, you can never really expect to control content on the Web. After all, this is how Google won the search engine wars over Yahoo!'s editorial team. However, I propose two fundamental questions:
1. Are directories dead to search? Let me rephrase that. Is there a benefit for a consumer to use a directory than just a dedicated search engine?
2. In either case, should there be human intervention in results?

The first question relates to the utility to the end user. I think its fair to say that more people are using Google these days than Dmoz However, the fact that directories still exist suggests that there must be an intrinsic value other than providing one way links.

Most directory formats are search engine friendly, which allows website content to be found and indexed by search engines abroad. Thus, traffic can be redirected to resources throughout the web and still provide a sense of utility. Traffic arbitrage if you will, is a widely accepted business model that can drive enormous amounts of quality traffic ready for monetization. Shopping comparison Websites such as Shopping.com for example, live and breath by this formula and are considered the next generation search in the vertical arena.

Yet directories can offer something that search engines can't and that is the mighty power of human editors! Editorialized content can certainly avoid some of the pitfalls above, especially if you're concerned about a family safe web environment. In addition, content can be selected or featured at will, where it is recognized as a privilege to become included. And even though some of the most technologically advanced search engines have tried to mimic human behavior, none have succeeded in doing so.

Regardless, search or directory, a methodology must be employed to deal with the ever increasing problem of link spam. Nevertheless, both are guilty in supporting it because more content and search results equal greater revenue opportunity. And even though search and directories will never have perfect model for minimizing spam, doing something about it can go a long way!