Apache web server. Need referring URL from my access_log

I have apache 1.3 that is integrated with oracle 9i.   I recently used this web server to recieve a HIGH volume of traffic from one site in particular that I placed advertisements on, lets call it "test.com".  I want to see all of the traffic strings in the access_log that came from test.com.  

I did a grep search through the access log for anything with test.com in it, and it didnt come back with much.  I thought that I should be able to get the referring URL's from the access_log... isnt this correct?   If it is correct, how do I do it?

THankS!
mintsiiAsked:
Who is Participating?
 
ahoffmannConnect With a Mentor Commented:
> .. can someone give me an example of what the line in the httpd.conf file would look like ..
something like following :

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog /opt/apache/logs/vcms.access_log combined

>  We spent large amounts of money on ads on a particular site...
spend a one-day work for a script to be called for your ads, and you get reliable results
0
 
basiclifeCommented:
That is in general correct with a couple of provisions:

1) the log file is set up to include referrers
2) the web browsers actually returned the correct referrers (most tend to but some ppl disable this on purpose, others just do it wrong)

To be honest, proviso 1 is the most likely culprit (presuming of course your grep statement was correct :)

have a look through a line or 2 of the log manually and see if you can see any referrers
Also, check the logs haven't been rotated - you may be looking in the wrong log file
0
 
ahoffmannCommented:
the referer is totally unreliable (see previous comment also), either the browser does not send it or fakes it, or any proxy or firewall inbetween removes it.
If you're interested in reliable hits, you can't use the referer info of your log files, you have to use a special link on your server which gets the destination URL as parameter (which might open your script for Link spoofing).
0
[Webinar] Kill tickets & tabs using PowerShell

Are you tired of cycling through the same browser tabs everyday to close the same repetitive tickets? In this webinar JumpCloud will show how you can leverage RESTful APIs to build your own PowerShell modules to kill tickets & tabs using the PowerShell command Invoke-RestMethod.

 
shalomcCTOCommented:
I think that despite the problems with the referer data, it is usually reliable when applied to home users.
On the other hand, latest studies of web marketing show that a CTR of 0.5% is considered ok nowadays.
Gone are the days of 5% CTR and 2% conversion rates......

Can you confirm that the total amount of traffic during the campaign was substantially higher than before the campaign?

I suggest that instead of greping the log file load it into excel or access, or better yet get a log analyzer.

ShalomC
0
 
mintsiiAuthor Commented:
Should the log file be setup to include referrers by default, or do I have to set it up that way.  If so, how do I set it up.  

I am close to positive that I do not have any referrers in my log.  Is there an easy way to test for referrers?  IE, should I grep the line referrers?
0
 
ahoffmannCommented:
there're examples how to use CustomLog in httpd.conf with referer-logging
0
 
basiclifeCommented:
to expand on the above comment, The logging setup is in httpd.conf :) - Specifically, you set what columns are displayed - usually something like IP address, Date/Time, Requested Page and response code (or something to that effect)

And this may eb a silly question but why did you want to l;og referring URLs? Just to get an idea of who sent ppl to your campaign?
0
 
mintsiiAuthor Commented:
We spent large amounts of money on ads on a particular site.. we would like to see how mach traffic came to our site because of the ads.  Also, can someone give me an example of what the line in the httpd.conf file would look like if you wanted to get referers logged.
0
All Courses

From novice to tech pro — start learning today.