Apache web server. Need referring URL from my access_log

I have apache 1.3 that is integrated with oracle 9i.   I recently used this web server to recieve a HIGH volume of traffic from one site in particular that I placed advertisements on, lets call it "test.com".  I want to see all of the traffic strings in the access_log that came from test.com.  

I did a grep search through the access log for anything with test.com in it, and it didnt come back with much.  I thought that I should be able to get the referring URL's from the access_log... isnt this correct?   If it is correct, how do I do it?

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

That is in general correct with a couple of provisions:

1) the log file is set up to include referrers
2) the web browsers actually returned the correct referrers (most tend to but some ppl disable this on purpose, others just do it wrong)

To be honest, proviso 1 is the most likely culprit (presuming of course your grep statement was correct :)

have a look through a line or 2 of the log manually and see if you can see any referrers
Also, check the logs haven't been rotated - you may be looking in the wrong log file
the referer is totally unreliable (see previous comment also), either the browser does not send it or fakes it, or any proxy or firewall inbetween removes it.
If you're interested in reliable hits, you can't use the referer info of your log files, you have to use a special link on your server which gets the destination URL as parameter (which might open your script for Link spoofing).
Shalom CarmelCTOCommented:
I think that despite the problems with the referer data, it is usually reliable when applied to home users.
On the other hand, latest studies of web marketing show that a CTR of 0.5% is considered ok nowadays.
Gone are the days of 5% CTR and 2% conversion rates......

Can you confirm that the total amount of traffic during the campaign was substantially higher than before the campaign?

I suggest that instead of greping the log file load it into excel or access, or better yet get a log analyzer.

Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

mintsiiAuthor Commented:
Should the log file be setup to include referrers by default, or do I have to set it up that way.  If so, how do I set it up.  

I am close to positive that I do not have any referrers in my log.  Is there an easy way to test for referrers?  IE, should I grep the line referrers?
there're examples how to use CustomLog in httpd.conf with referer-logging
to expand on the above comment, The logging setup is in httpd.conf :) - Specifically, you set what columns are displayed - usually something like IP address, Date/Time, Requested Page and response code (or something to that effect)

And this may eb a silly question but why did you want to l;og referring URLs? Just to get an idea of who sent ppl to your campaign?
mintsiiAuthor Commented:
We spent large amounts of money on ads on a particular site.. we would like to see how mach traffic came to our site because of the ads.  Also, can someone give me an example of what the line in the httpd.conf file would look like if you wanted to get referers logged.
> .. can someone give me an example of what the line in the httpd.conf file would look like ..
something like following :

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog /opt/apache/logs/vcms.access_log combined

>  We spent large amounts of money on ads on a particular site...
spend a one-day work for a script to be called for your ads, and you get reliable results

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Web Servers

From novice to tech pro — start learning today.