From TNPC issue #4.02...

Search Engines - From Payments to Penalties

by T.J. Lee
January 25, 2001

Back in TNPC #3.25 I wrote about how the Internet was slowly moving from an "everything is free" to an "everything for a fee" model (and I caught some serious flack over the improper use of the word "model," but I digress). This trend will affect every computer user who surfs the Internet. That just about covers everyone whether you pay directly or not as I'll explain, all because of what's happening with the search engines we use to find things on the Web.

As I said in that prior article the major player Web search engines are offering a pay for placement plan (where Web page creators pay money to have their site show up higher in a list of returned hits, regardless of whether their site has more relevant content). Furthermore, many engines now allow folks to pay a fee to get their site "considered" for listing in its page database ahead of other sites that do not pay the consideration fee.

I believe Yahoo! was the first to come up with this idea. Yahoo! has always been a very difficult engine to get pages listed in. You'd submit your pages, you'd check back, you'd submit again, you'd check back, it could takes weeks or months before you could actually find your pages by searching in Yahoo!, if they showed up at all. Not being blind to the concept of supply and demand the geeks at Yahoo! came up the idea of making the masses pay just to be "considered." For a fee Yahoo! promises to review pages and consider them for inclusion in their database. This fee doesn't guarantee the pages are actually listed, but it gets them to the top of the stack for sites to be looked at. And it paid off for Yahoo!.

Now most of the main search engines have a consideration fee program and more Webmasters are shelling out the bucks to ensure that their pages at least get looked at. Considering the sheer volume of Web pages out there, this approach makes some sense. If someone is willing to pay to be considered, the theory goes, then their site must have some sort of valid content worthy of consideration. This theory would, hopefully, let the search engines pay a bit more attention to sites that are more relevant. If you create Web sites you've no doubt wrestled with whether or not to pay for consideration or just stick with the "submit URL" free method and hope for the best.

But things have gotten a lot more interesting in the search engine listing game. Inktomi has recently stated that pages submitted via their free "submit URL" link will actually be penalized in search result listings, that is they'll appear farther down the list than pages listed via the paid consideration method or that are found by their Web-crawling spider. This time the theory goes that the pages submitted via the free submit URL link only encourages low quality pages getting into the listings.

Danny Sullivan who runs the Search Engine Watch site clued me in to what was happening at Inktomi and how this can impact the listing of pages. Danny is one of the Web's foremost authorities on what the search engines are doing at any given time. He expects that search engines will eventually stop accepting free submissions of Web pages altogether and I agree with him. He thinks that the best way to get a Web site listed and prominently placed is via the various link crawling spiders that the search engines have running around the Web. The spider program goes to a page and follows the links on that page to find other pages. The more links leading to a page the more relevant the spider ranks that page. This is, I believe, the most reasonable theory yet... a page that has good content will have more links pointing to it from other pages than do pages that no one links to.

This means that the more sites that link to a page, the better that page will fare in the search engines. This is known as getting a good "popularity" rating. This is the reason that Webmasters are more aggressive about requesting links to their pages. Which reminds me, if you have a Web site of your own please link to our newsletter page and proclaim your proud membership in the Naked Horde! Hey, we even have some suitable graphics you can use:
http://www.TheNakedPC.com/horde.html

Shameless self-promotion aside, the search engines and the results they provide affect all of us. This trend to get everything on a paying business basis will have a significant impact on how the Web works. If you want to find out more about search engines and what they're up to, check out Danny's Search Engine Watch site.
http://www.TheNakedPC.com/t/402/tr.cgi?sew

You can reach T.J. Lee at:
mailto:tj_lee@TheNakedPC.com

Return to Top

TNPC Hot Tips:

• Email out of control? Spam filling your inbox? People trying to steal your identity? Same here - until I applied these tips. You can too in a new multimedia e-book. Tame Your Email.

• DO YOU MAKE THESE MONEY MISTAKES? Do you know that trying to pay off your high interest rate debts first and/or paying extra on more than one debt is the SLOWEST way to get out of debt? Don't make these same mistakes. Learn more at by clicking here...