CSI Miami or LSI Google? - Part I of II

What is Latent Semantic Indexing?

Latent Semantic Indexing, otherwise known as LSI, is a new way for search engines such as Google, MSN and Yahoo, to view and rank a websites content in a manner much more like you’d expect a human to rank a site. Behind the scenes these new LSI algorithms analyze pages not only for keywords, but also for similar words AND other RELATED words which might be expected to be present.

For example, a web page about golf, when analyzed using LSI, should expect to contain related words, such as “golfcart”, “clubs”, “courses”, “green”, “fairway” etc.  In general, while we do not know the exact mathematical formula used for LSI, we do know that its real function is to determine if the content of a site is of value to the visitor or not.

Why, and how, did LSI come about?

A few years ago, Google purchased a company called ‘Applied Semantics’ whose software was used to organize information from websites in a manner similar to the way that humans might do.  Google bought this company so that they could match Adwords advertisers with the appropriate web pages where the ads would be shown as Adsense ads.  Initially, Google matched keywords on the webpages to keywords in the ads and a website owner earned money for every click he received from the Adsense ads on his or her site.

However, people are always trying to find quick and easy ways to make money online (and elsewhere) and a problem soon arose that millions of websites were being created simply to contain relevant long-tailed keyword phrases to capture traffic from Google.  Content on these, often machine-generated sites, was third-rate at best. Imagine searching for ‘Golf’ and finding a website with nothing more than advertisements for products, when all you wanted was information about golf.  Google could not differentiate between this fake “content” and the sites which actually contained valuable content, with REAL information about what was searched for.

Other ‘Black Hat’ techniques, such as filling webpages with keywords and using linking strategies to increase rankings started being used, which made the problem worse still. This reciprocal linking, however, was soon given the now famous “Google-slap” and even Yahoo and MSN have started changing their algorithms to make reciprocal links have a far lower ‘weight’, so they aren’t worth as much to the ranking status.

Part II tomorrow.

Luke