We help IT Professionals succeed at work.

We've partnered with Certified Experts, Carl Webster and Richard Faulkner, to bring you two Citrix podcasts. Learn about 2020 trends and get answers to your biggest Citrix questions!Listen Now

x

Apache web server.  Need referring URL from my access_log

mintsii
mintsii asked
on
Medium Priority
203 Views
Last Modified: 2010-04-20
I have apache 1.3 that is integrated with oracle 9i.   I recently used this web server to recieve a HIGH volume of traffic from one site in particular that I placed advertisements on, lets call it "test.com".  I want to see all of the traffic strings in the access_log that came from test.com.  

I did a grep search through the access log for anything with test.com in it, and it didnt come back with much.  I thought that I should be able to get the referring URL's from the access_log... isnt this correct?   If it is correct, how do I do it?

THankS!
Comment
Watch Question

That is in general correct with a couple of provisions:

1) the log file is set up to include referrers
2) the web browsers actually returned the correct referrers (most tend to but some ppl disable this on purpose, others just do it wrong)

To be honest, proviso 1 is the most likely culprit (presuming of course your grep statement was correct :)

have a look through a line or 2 of the log manually and see if you can see any referrers
Also, check the logs haven't been rotated - you may be looking in the wrong log file
the referer is totally unreliable (see previous comment also), either the browser does not send it or fakes it, or any proxy or firewall inbetween removes it.
If you're interested in reliable hits, you can't use the referer info of your log files, you have to use a special link on your server which gets the destination URL as parameter (which might open your script for Link spoofing).
CERTIFIED EXPERT

Commented:
I think that despite the problems with the referer data, it is usually reliable when applied to home users.
On the other hand, latest studies of web marketing show that a CTR of 0.5% is considered ok nowadays.
Gone are the days of 5% CTR and 2% conversion rates......

Can you confirm that the total amount of traffic during the campaign was substantially higher than before the campaign?

I suggest that instead of greping the log file load it into excel or access, or better yet get a log analyzer.

ShalomC

Author

Commented:
Should the log file be setup to include referrers by default, or do I have to set it up that way.  If so, how do I set it up.  

I am close to positive that I do not have any referrers in my log.  Is there an easy way to test for referrers?  IE, should I grep the line referrers?
there're examples how to use CustomLog in httpd.conf with referer-logging
to expand on the above comment, The logging setup is in httpd.conf :) - Specifically, you set what columns are displayed - usually something like IP address, Date/Time, Requested Page and response code (or something to that effect)

And this may eb a silly question but why did you want to l;og referring URLs? Just to get an idea of who sent ppl to your campaign?

Author

Commented:
We spent large amounts of money on ads on a particular site.. we would like to see how mach traffic came to our site because of the ads.  Also, can someone give me an example of what the line in the httpd.conf file would look like if you wanted to get referers logged.
> .. can someone give me an example of what the line in the httpd.conf file would look like ..
something like following :

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog /opt/apache/logs/vcms.access_log combined

>  We spent large amounts of money on ads on a particular site...
spend a one-day work for a script to be called for your ads, and you get reliable results

Not the solution you were looking for? Getting a personalized solution is easy.

Ask the Experts
Access more of Experts Exchange with a free account
Thanks for using Experts Exchange.

Create a free account to continue.

Limited access with a free account allows you to:

  • View three pieces of content (articles, solutions, posts, and videos)
  • Ask the experts questions (counted toward content limit)
  • Customize your dashboard and profile

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.