Referential Integrity vs. Latent Semantic Indexing

Ever heard of LSI? Well I hope you have, because that MIGHT mean that you read my article about it recently.  A huge part of making money on the internet depends upon how much traffic you get.  Let’s face it, having the best products in the world at fantastic prices is pointless if you can’t get people to come and visit your store (by which I mean website of course).

Google’s Ranking System

To get your website ranked high (preferably on the first page of a Google SERP (Search Engine Page Rank), you need to have, among other things, keywords (words or combinations of words) in your website that are relevant to a potential visitior’s enquiry.  You also need to have high-quality links from other websites; that is, links from sites that are both relevant to YOUR site and that ALSO have a high page-rank.  This is, at least in part, how the Google Algorithm works (but I have no doubt whatsovever that it’s a lot more complicated, and discerning, than that!).

The Step UP

Rumor has it that those clever people at Google are now using something called Referential Integrity to determine what web page should rank it.  Referential Integrity is meant to be a significant step-up from Latent Semantic Indexing in that, unlike LSI, it uses an understanding of the interaction of people rather than strings of words that SHOULD be found together in everyday writing.

How It Works

Google and the other search engines, are really only trying to act as an agent; a company that uses technology to put searchers in touch with companies who WANT to be found…………..well, that and Google also wants to make profits obviously.

The way that Referential Integrity works is quite simple: human nature dictatest that if two people, or two groups of people, want to connect to each other, Google provides a widespread tool to allow this to happen.  This reminds me of the old adage “when the student is ready, the teacher will appear”.

The Technology

The first part of the equation is very much like Latent Semantic Indexing: Google finds the millions of search terms (keywords) and creates a matrix/index of them. Google then figures-out these strings of words MEANINGS by comparing the LINKS between pages (these being the blue/underlined text on the pages, sometimes known as Anchor Text).  These are the links that tell the Google software (spiders or crawlers) exactly how the pages relate to each other.

Next

Then, Google software builds a SECOND matrix that compares ONLY the Anchor text.  Now, with these TWO matrices, they can work their “magic”.  When their software COMPARES these two indexes, they come up with a resulting single index that is far more accurate than either one by itself.  It’s this “reference” between the two indexes (matrices) that gives its name to Referential Integrity.  The ‘integrity” part of the name comes from the fact that the links on a page are an accurate description of the words that are on the page; they have “integrity” because that are congruent with each other.

The Future

While nobody really knows what the future is for search engines, looking at recent history can give you some idea of how search-engines are developing. It’s pretty certain that their is always going to be a need for search-engines, at least for the next 10 years or so, but how they develop are ideas for the boffins at Google (and Yahoo and Ask and….).

It’s very likely that soon voice recognition will be used, so you can simply ask your computer something and Google will provide the pages that are most relevant to what you asked for.  After that?  Maybe image recognition, however, I suspect that THAT is quite a long way off.  For the time being, you’ll just have to be as open minded about what you are searching for, and also as specific as possible without using too many words.

Luke.