SEO Surveying The Search Engine Landscape Optimization” You’ve got a problem. You want people to visit your Web site; that’s the purpose, after all, to bring people to your site to buy your product, or learn about service, or hear about the cause you support, or for whatever another purpose you’ve built the site. So you’ve decided you need to get traffic from the search engines- not an unreasonable conclusion, as you find out in this chapter. But there are so many search engines!
You have the obvious ones the Googles, AOL’s, Yahoo!s, and MSNs of the world but you’ve probably also heard of others: HotBot, Dogpile, Ask Jeeves, Netscape, EarthLink, LookSmart.. even Amazon provides a Web search on almost every page. There are Lycos and InfoSpace, Teoma and WiseNut, Mamma.com and WebCrawler. To top it all off, you’ve seen advertising asserting that, for only $49.95 (or $19.95, or $99.95, or whatever sum seems to make sense to the advertiser), you too can have your Web site listed in hundreds, nay, thousands of search engines. You may have even used some of these services, only to discover that the flood of traffic you were promised turns up missing.
Search Engine Landscape Optimization
Well, I’ve got some good news. You can forget almost all the names I just listed well, at least you can after you’ve read this chapter. The point of this chapter is to take a complicated landscape of thousands of search sites and whittle it down into the small group of search systems that really matter. (Search sites? Search systems? Don’t worry, I explain the distinction in a moment.
If you really want to, you can jump to “Where Do People Search,” near the end of the chapter, to the list of search systems you need to worry about and ignore the details. ButI’ve found that, when I give this list to someone, he or she looks at me like I’m crazy because they know that some popular search sites aren’t on the list. This chapter explains why.
Investigating Search Engines and Directories:-itwords.org
The term search engine has become the predominant term for search system or search site, but before reading any further, you need to understand the different types of search, um, thingies, you’re going to run across. Basically, you need to know about four thingies.
Search indexes or engines are the predominant types of search tools you’ll run across. Originally, the term search engine referred to some kind of search index, a huge database containing information from individual Web sites.
Large search-index companies own thousands of computers that use software known as spiders or robots (or just plain bots) to grab Web pages and read the information stored in them. These systems don’t always grab all the information on each page or all the pages in a Web site, but they grab a significant amount of information and use complex algorithms-calculations based on complicated formulae to index that information. Google, showing in Figure 1-1, is the world’s most popular search engine, closely followed by Yahoo! and MSN.
A directory is a categorized collection of information about Web sites. Rather than containing information from Web pages, it contains information about Web sites. The most significant search directories are owned by Yahoo! (dir.yahoo. com) and the Open Directory Project (www.dmoz.org). (You can see an example of Open Directory Project information, displayed in Google – ir.google.common Figure 1-2.)
Directory companies don’t use spiders or bots to download and index pages on the Web sites in the directory; rather, or each Web site, the directory contains information, such as a title and description, submitted by the site owner. The two most important directories, Yahoo! and Open Directory, have staff members who examine all the sites in the directory to make sure they’re placed into the correct categories and meet certain quality criteria. Smaller directories often accept sites based n the owners’ submission, with little verification.
Here’s how to see the difference between Yahoo!’s search results and the Yahoo! directory:
- Go to www.yahoo.com.
- Type a word into the Search box.
- Click the Search button: The list of Web sites that appears is called the Yahoo! Search results, which are currently provided by Google.
- You see a line that says something like Category: Footwear Retailers. You also see the line underneath some of the search results.
- Click either the tab or link: You end up in the Yahoo! directory. (You can go directly to the directory by using dir.yahoo.com.
I wasn’t sure what to call these things, so I made up a name: non-spidered indexes. A number of small indexes, less important than major indexes such as Google, don’t use spiders to examine the full contents of each page in the index. Rather, the index contains background information about each page, such as titles, descriptions, and keywords.
In some cases, this information comes from the meta tags pulled off the pages in the index. I tell you about meta tags in Chapter 2.) In other cases, the person who enters the site into the index provides this information. A number of the smaller systems discussed in Chapter 13 are of this type.
Some systems provide pay-per-click listings. Advertisers place small ads into the systems, and when users perform their searches, the results contain some of these sponsored listings, typically above and to the right of the free listings.