How Search Engines Work - Think like the machine, and put yourself in play
From our years of interviewing clients and prospective clients for our services, we are aware that the typical business owner does not really understand how a search engine works. That's because they are busy running their businesses.
One common misconception is that the rankings are somewhat random, or that the search engines can "think" like a human being, and make cognitive decisions, much like a person might, if given a set of data.
It is also common to imagine that the search engines are assigning some kind of "quality" ranking to the sites that they place at the top for a search term, or that there is some form of human analysis taking place.
This can result in statements like "Why does Google like that site, and not my site? That other site looks awful!" In fact, there is not a human being making that decision, and the search engine can't "see" what you see.
Except for specific situations when real employees intervene, these engines are not the least bit human. The human influence is generally limited to establishing the rules. That may eventually change, but there are billions of Web pages, and billions of searches. So it all happens using the algorithm calculations, and that algorithm uses pre-defined rules. With Google, it has been stated by their own engineers that they have nearly 200 factors in their algorithm, with various weights assigned to each factor.
This machined-based computation can result in anomalies and exceptions, but, in the end, it also provides a way to determine what works best and consistently. That insight comes from reviewing a lot of real world situations.
Having worked with hundreds of Websites since the late 1900s, we have the perspective to see the patterns that tend to work, day in and day out. We can all discuss the finer points until the cows come home, but the basics have been well established for several years.
Text , Text, and More Text!
So, search engines are really just data machines. They have software tools, called robots, that follow the links on the Internet, read the data that it encounters, (in this case, the Web pages), catalogue it, (that's called "indexing") and spew it back out, when someone searches for a specific piece of data (in this case, a search term).
Think of the robots like hungry little insatiable animals that eat words for lunch, and they do it constantly, never getting full. To them, text is a great meal.
You must also realize that they can't actually see images, but they can read the textual information that is presented about those images.
Words and text rule the day. To a search engine, there is really nothing else. You have to feed that animal, with the right textual content. You can feed it gobbledygook, or you can feed it exactly what you want it to read. It's under YOUR control. Unfortunately, we see a lot if situations where a Website is unwittingly feeding the robots gobbledygook, due to a basic lack of understanding.
Remember this: The machine reacts to what you present to it as textual content, and how you present it. It generally does not extrapolate or interpret what you present.
A Real Search
Let's now look at that lucrative search for "New York real estate" that we mentioned previously, as it would be submitted to a search engine. What follows is not an exact description of what takes place, but it's one that will help you visualize what generally happens inside the search engine.
The first thing that the engine does is to compile a list of all the pages in their index of known pages that have the words "New", "York", "real", "estate" somewhere on the page.
Please realize that this is ALL it can do. It cannot really say, hmmm, this person is looking for "New York real estate", so they might also want "Manhattan real estate" results as well. A human might draw that kind conclusion, but a machine cannot, unless it has been specifically programmed to do that.
So, if you are a Manhattan real estate professional, and your Web pages do not say "New York real estate", you will NEVER rank anywhere for that term. The search engine will not deduce that just because your pages say "Manhattan real estate", that you are also into New York real estate as well.
Think about it. It's a machine. The search engine must be told what you are about, via the textual material presented on your Web pages. We'll come back to this concept, again and again.
As this was written, the MSN search engine currently shows 8,558,742 page results containing "New York real estate". Those are the pages that are "in play" for this search term. Everything else in the whole MSN index (and that is billions of other pages) is completely ignored from this point forward. There are 8 million or so "players". One of them has to be first!
So the engine starts a winnowing process. Some pages have all those words, but the words are scattered around the page. Other pages have the term "New York real estate", grouped together, in sequence, on that page. Certainly, the engine takes that into account, as a means to better focus the search.
Of those pages that sequence the term, some might also have the term "New York real estate" in the title and the headline HTML tags on that page. Again, that represents a page that is specifically focused on the term "New York real estate", and it should score a bit better in the algorithm for this term.
So, we are halfway there. They've compiled their data out of the index, based on the "on page" factors at play. By "on page", we mean the content that is shown on that page. The engine now wants to see what "off page" factors come into play, and these can be very significant. Off page factors generally relate to the links that point to that specific page, but also may include other links to other pages of the domain.
The search engine robots basically just follow links around to find content, its information that can be used to further help refine a search. Those links point back to the content page that is being referenced in the link, and the link itself quite often has text in it, in the form of the "anchor text".
The anchor text is usually the underlined, highlighted text portion of a link that we all see on Websites all the time. If you click on the link anchor text, it takes you to the actual page for that link. That page is the "landing page" for the link, since that's where you land when you click on that link.
These days, the search engines can examine how many links point to a particular page, as well as a whole Website, how long those links have been in place, when they were placed, the anchor text of those links, as well as the text of the rest of the page where the link was found. Again, text still plays a huge role here.
They can use all of this information to determine the "off page" factors that come into play for a search term.
Going back to our term "New York real estate", they can look for that anchor text in links, and then pages that those links point to can be given additional weight. After all, a link to a page that is focused on "New York real estate" that has "New York real estate" as the anchor text in the link, generally implies a strong correlation between the link and the content of that page.
Keep in mind that the search engines are trying hard to deliver relevant results for the term "New York real estate" to their users. The tighter they can focus that on pages that appear to be about that term, the better they have done their job. That's why they use all the tools and data that they can muster to the task.
Remarkably, after a few milliseconds, the engine comes back with their search results report for the search term "New York real estate". This is sometimes referred to as a SERP, an acronym for "search engine results page". A SERP is simply the page that is presented back to the person doing that search.
As we said, someone has to be first, second, third, etc. The site that is positioned best for the term "New York real estate", using the above scenario of on page and off page factors, is likely to be first.
As you can see, if you have not fed the engine text that is related to "New York real estate", and structured it in the ways that provide an advantage, and then established some links that will provide that page some off-page support, there is virtually NO CHANCE of ranking well for the search term "New York real estate", especially out of 8 million other competing pages.
The rest of our presentation will be a discussion of how to put your site into play for the search terms that are important to your business. Again, it's not a matter of chance. It is a matter of positioning to compete.
