AffiliateBeginnersGuide.com

Top ranking on search engines using web site logs?

Probably many of you are using some kind of tool to check web site (page) search engine position. Yes, there are useful online tools and software as is WebPosition, WebCEO, IBP... where you can check position of some web page, tracking history of position on search engines

The fact is that in web site log, that history is already written, not precise as is in above tools, but enough to see if some web site page is positioned on the first page of SERP, or it is on second or third...

Why to go to the search engines when search engines are coming to your site? Not exactly search engine, visitor is who is carrying information of your search engine position. Analyzing referral string, you could know what keyword was used and on what page of SERP is positioned your page for that keyword. We are talking here about raw site log, not statistic you can find in your hosting account. It needs to be downloaded from your server to be analyzed.

Google example

http://www.google.com/search?q=my+keyword&hl=en&start=10

This web site log entry is stripped down to referral string only. Sure, that log line shows visitor's IP, time, landing page and browser used. In above example, your site page is positioned on the second page (start=10) on Google SERP (search engine results page) for keyword "my keyword".

Yahoo example

http://search.yahoo.com/search?p=my+keyword&b=11

Like above for google referral, here "b=11" shows that our page is positioned on the second page of search engine results for that keyword.

Geolocation

Sometimes is very important to know in which Country visitor is located. To better target products what are restricted to one or more Countries, check your logs for other search engine domains (subdomains). Search engines are redirecting visitors to their local servers, serving results according to their location.

Clear example is Yahoo, where uk.search.yahoo.com shows that visitor is using Yahoo UK to find your website (web page). If you see more visitors from UK, you should push product for UK visitors, or to get more links to that page from sites based in targeted Country. Same is for Google (google.ca - Canada, google.com.au - Australia)

When visitor is using search.yahoo.com, or google.com or MSN/LIVE, or any other search engine (not local), there is an other way to find from what Country is visitor - from IP address. That is described on IP address to Country page.

Web page position history and automation

Raw site logs are full of pure gold, but we need to separate that gold from other data. So we need to use little programming to pull out what is interesting for us. This article is focused on search engine position, and here is example how to get useful data from web site log.

It is required that you know simple programming, language is not important. Flat text files are used to store data, to make this simple.

First, in program directory (folder) make subdirectory for every tracked search engine (i.e. google/ for Google).

Second, copy raw web site log to that directory or somewhere when program could find it.

In program, we need to define search engine domains (pattern match), folder where we add keywords files for that search engines, and patterns (pairs) to match and extract keyword and position from referral url.(q=,start=)

Option is, when parsing log lines, referral strings containing question mark could be added to one separate text file to find unknown search engines, if they are worth to track later.

Now, let's start parsing web log lines. In this example, we are looking for referrals from Google.com Program routine must find google.com (or with wildcards) in referral field of log line. If google.com is found (in referrals field), next is to get page name (or category/page numbers combined), date (month,day), and complete referral string.

Next step is, extracted page name (without extension and path) will be name of our text file in "google" directory (folder). Extension of that file could be .txt, and option is to add month to file name if you want to have all searches for one month in one file. So our file looks like "pagename-Jan.txt" or "pagename-1.txt".

Next is to add data extracted from web log line to our small database (pagename-Jan.txt)

Every line in file contains pipe or comma separated fields. So, first field is date (with or without month, see page name), and second is referral string. Why referral string, and not extracted keyword and position? It is up to you, what field you will add to line, but adding (archiving) referral string to database, you have an option to check exact position later using that url.

Now when we are finished with parsing and extracting data, it is the time to use that data.

Just fast look, opening that (page name) database in any text viewer, and you know what keyword(s) was used to find that page, when, and where is page positioned on SERP. Position changes are available also (first page, second,etc...). Anyway, with little more programming, better formatted report could be displayed. More fun guaranteed :-)

Top ranking?

Top ranking for keyword "mp3" is not what we are talking about here, but top ranking for keyword "I want to buy mp3 player" and similar "long tail" keywords are what we are looking for. And top position for promising long tail keyword could mean 1 click = 1 sale, or far better conversion rate than usually. So, that is not only about tracking page position on search engines, new keywords means new ideas and new optimized pages. And many of those keywords (ideas) are already in your web site log. Analyze, optimize...:-)