Solved

Why do I see all my browsing activities in access.log and error.log?

Posted on 2004-09-27
17
247 Views
Last Modified: 2010-03-04
Hi Experts,

I am a programmer and as such not really accustomed to configuring web servers. I just recently installed Apache and configured in a way I thought good, but now the log files happen to show ALL the activity I have on the web, instead of only the incoming activity. Not that I have anything to hide, but I'd rather see only the incoming HTTP requests and the real errors that happen.

What happens now is that I see something like this line in error.log after visiting the site airdisaster.com (one line):
[Sun Sep 26 15:49:21 2004] [error] [client 127.0.0.1] File does not exist: D:/internet/apache/htdocs/w, referer: http://www.airdisaster.com/cgi_bin/database.cgi

This shows up around the same time in access.log (one line):
127.0.0.1 - - [26/Sep/2004:15:49:21 +0200] "GET /cgi-bin/ads/ad8799a.cgi/NI/if/v=1.0J/sz=468x60A/r=http%253A%252F%252Fwww.airdisaster.com%252Fcgi_bin%252Fdatabase.cgi/1983/RETURN-CODE HTTP/1.1" 404 390 "http://www.airdisaster.com/cgi_bin/database.cgi" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; T312461; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"

I did find Cydoor (with Ad Aware) on my machine, but it was not active, and I'm not sure it could cause this. Also, once removed, I still see the logs appear. Also, sometimes I do NOT find the links I visited, especially when using Opera. But that may be coincidentally. As an example: I never find www.activestate.com in the logs, but I always find www.airdisaster.com in the logs.

What is happening here? Can anyone help me keeping my logs clean? Are these just some kind of ads trying to find information at my site? Or what else?

Any help or ideas will be appreciated.

Regards,
Abel
0
Comment
Question by:abel
  • 8
  • 8
17 Comments
 
LVL 15

Expert Comment

by:periwinkle
ID: 12165549
Are you using the free version of Opera?  The free version of Opera is ad-ware - i.e. it supports itself using advertisements.  Perhaps this is why?
0
 
LVL 39

Author Comment

by:abel
ID: 12183031
Hi Periwinkle,

No, that was not what I meant. Basically, all the browsers have this habit, and it _appears_ that Opera has it less. But I don't think it is in the browser. Actually I presume it's in the Apache configuration. For some reason it logs all 127.0.0.1 (localhost) requests. Maybe this is due to the fact that I indeed configured localhost as a webserver as well?

If anyone has any ideas about this, it'd be great!

Regards,
Abel
0
 
LVL 15

Expert Comment

by:periwinkle
ID: 12183241
Hi Abel -

(thinking out loud)Airdisaster.com isn't your site, correct?  Am I correct that, according to the log entries, your web server is executing a CGI script every time you make a request out to another site?  (almost like a proxy server?)
0
 
LVL 39

Author Comment

by:abel
ID: 12185036
Yes. Absolutely. That's exactly my point. I hoped that "but now the log files happen to show ALL the activity I have on the web" was making that clear, but upon rereading that, I understand that my point was no so obvious after all.

Indeed, a proxy, that is what the Apache logs look like. I don't want that (of course). Ok, let me point out a few things:

- My home computer, that I use for internetting (like typing this text) and mailing, is the same one that has the web server residing on it.
- I use a continuous connection through ADSL, a firewall and such (no NAT)
- The logs act _like_ a proxy, but not exactly as one
- In fact, they only log my browsing activities partially (no www.experts-exchange.com in the logs there, for instance)
- It looks like "most" as in "a lot". I haven't figured out yet why some are not logged, but maybe I should not search in that direction.

Is this odd? Doctor, am I very ill? Is there a cure? Or is there a simple pill that takes all my headaches away?

Cheers!
Abel

0
 
LVL 39

Author Comment

by:abel
ID: 12185084
Forgot about this one:
> your web server is executing a CGI script
No, nothing is executed. As a fact, all the HTTP return values are 404 (not found). Even though my web browser simply shows the page (luckily so).


Talking about logging entries, here are a few "foreign" logging entries that are definitely not mine (the IP number is alien to me). They look "quite" normal to me and show me that the other way around (logging aliens) works.

61.97.128.2 - - [28/Sep/2004:10:57:25 +0200] "CONNECT 1.3.3.7:1337 HTTP/1.0" 405 296 "-" "-"
218.5.107.218 - - [26/Sep/2004:16:15:46 +0200] "CONNECT 66.94.230.34:80 HTTP/1.1" 405 299 "-" "-"
218.5.107.218 - - [26/Sep/2004:16:15:49 +0200] "GET http://www.yahoo.com/ HTTP/1.1" 200 - "-" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
64.246.165.200 - - [27/Sep/2004:09:43:05 +0200] "GET /robots.txt HTTP/1.1" 404 278 "http://www.whois.sc/" "SurveyBot/2.3 (Whois Source)"
0
 
LVL 15

Expert Comment

by:periwinkle
ID: 12185138
At first, the log entries made me think that there was ad-ware on your server that was trying to execute something locally (hence the CGI script comment);  now, the more I think about it, the more I'm aiming towards your server is acting like a proxy.

I agree - we should try and determine why some of your browsing is logged, and some is not... that will probably lead us to why your browsing is being logged at all.

>> Is this odd? Doctor, am I very ill? Is there a cure? Or is there a simple pill that takes all my headaches away?

LOL... that was my smile for the day - thanks!

0
 
LVL 15

Expert Comment

by:periwinkle
ID: 12185145
Which version of Apache do you have installed?
0
 
LVL 15

Expert Comment

by:periwinkle
ID: 12185159
In your httpd.conf file, do you have any of the following:

ProxyRequests on
ProxyVia on
<Proxy *>
...
</Proxy>

???
0
Free Gift Card with Acronis Backup Purchase!

Backup any data in any location: local and remote systems, physical and virtual servers, private and public clouds, Macs and PCs, tablets and mobile devices, & more! For limited time only, buy any Acronis backup products and get a FREE Amazon/Best Buy gift card worth up to $200!

 
LVL 39

Author Comment

by:abel
ID: 12193627
The only proxy-like lines I find are:

#LoadModule proxy_module modules/mod_proxy.so
#LoadModule proxy_connect_module modules/mod_proxy_connect.so
#LoadModule proxy_http_module modules/mod_proxy_http.so
#LoadModule proxy_ftp_module modules/mod_proxy_ftp.so

And they are all commented out.

But looking through the .conf file I come across the following lines:

# Change this to Listen on specific IP addresses as shown below to
# prevent Apache from glomming onto all bound IP addresses (0.0.0.0)
#
#Listen 12.34.56.78:80
Listen 80


Maybe these ones make it happen that a lot of things the browsers are doing is logged. Let's try it out. I'll set it to only listen to the external IP addresses I have and we'll see what happens.

What'd you think, doc?
0
 
LVL 39

Author Comment

by:abel
ID: 12193699
Hmm, my first findings are positive. I tried www.examcram2.com, which was logging notoriously many entries when I visit it, and now it does not anymore. Let's wait a while and see if this pill indeed cures all illnesses ;-)
0
 
LVL 15

Expert Comment

by:periwinkle
ID: 12199366
Listen 80 is what tells Apache to listen on Port 80.  If you don't do that, then you can't perform virtual IP-based hosting.
0
 
LVL 39

Author Comment

by:abel
ID: 12352484
Hi Periwinkle,

It's been a while, but the problem really seems to have disappeared by now.

What I was wondering about, though, does this impose problems on my way of handling localhost-sites and/or multiple sites on one system? Was the cure worse then the illness?

Cheers
0
 
LVL 15

Accepted Solution

by:
periwinkle earned 150 total points
ID: 12352506
abel -

In re-reading your response, you are listening only to the external addresses, and not the internal ones - I believe that is the correct solution.  I misunderstood that you were planning to not listen to port 80 at all!
0
 
LVL 39

Author Comment

by:abel
ID: 12578814
Hi Periwinkle,

Sorry for this "abandonation". You yourself have helped me answering the question and resolving the problem. The points are thus yours. Thanks for all your help.

Regards,
Abel
0
 
LVL 39

Author Comment

by:abel
ID: 12578828
Anybody reading this solution: you'll have to read the whole thread and not only the solution to find out what was going on.
0
 
LVL 15

Expert Comment

by:periwinkle
ID: 12582637
Hi Abel -

Absolutely no worries - I was just wearing my 'Community Volunteer' (CV)  hat and helping to clear out the old questions - I'm very happy that your problem is now solved!
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

As Wikipedia explains 'robots.txt' as -- the robot exclusion standard, also known as the Robots Exclusion Protocol or robots.txt protocol, is a convention to prevent cooperating web spiders and other web robots from accessing all or part of a websit…
If your site has a few sections that need to be secure when data is transmitted between the server and local computer, such as a /order/ section for ordering or /customer/ which contains customer data, etc it would of course be recommended to secure…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now