Apache2.2 Error Logs - File Does Not Exist

Well, even on my development server which is not public, I have the same errors. There is not a spider or crawler looking at those files... or did I misunderstand you?

Well, the 192.168.*.* is a static IP on the internal LAN... all our internal IPs start that way. I do not run phpmyadmin, mysql, or any of those other "not found" things. My dev server is Windows 2003 Server Edition with tight security. It has backup software, virus scanners, etc. My dev server IP is the 192.168.12.1. Additionally, I only get these errors when I restart Apache (or reboot - start Apache). I am 100% confident that no one is accessing the server other than through HTTP. I access it through Remote Desktop, Shared Folders, or - most commonly - VM Infrastructure as the machine is really just a VM. The only other people who access that server are the project managers who use the HTTP protocol to connect and they aren't searching for things like <root>/blog - trust me.

The development server is behind the corporate firewall which is strict as... well it's strict/annoying. The only users who have security to the server are the administrators (which is our network admins and myself). I also can access it using my network login to the local domain through Shared Folders (i.e. CompanyDomain\UserID).

The "attacks" are on the development server and my redundant "live web servers" in the DMZ. So, there's firewalls infront and behind.

It seems unlikely that it is a "script kiddie", but ... any suggestions?

Thanks!

You didn't quite understand what grahamnonweiler and I were suggesting. OK, because it is happening on the LAN makes it a bit different but anyhow:
- no mater how tight security you have on your server (virus scanners, firewalls, ...) you are letting through all http traffic, right?
- the errors suggest that someone is probing various urls - possibly probing for server vulnerabilities
- the someone could be a virus or spyware on any machine in the network which has http access to this server
- the someone could even be a security vulnerability scanner on your network or some other product.

Could you describe more:
- from which server were the error logs you posted? What IP does it have? Is it in DMZ or on your LAN?
- what does Apache access log say for the times you get that errors?

Based on the description from damijim - in that as soon as he starts Apache these entries appear - and that he is running under VMWare - then it is most probably a "security vulnerability scanner" that is being run by the network admins.

If this is the case then the scanner will have access to both "public" and "local" servers - regardless of whether they are virtual or live.

The most important this to understand damijim - is that both Blaz and myself are trying to tell you that this traffic is based on HTTP access to Apache - so if someone typed in http://192.168.12.1/phpMyAdmin then it would show up as a 404 error in your Apache logs.

A security vulnerability scanner sends out these type of requests continually (or a periodic basis) to ensure that no one has created a "security hole" on the company's server. If the scanner is mis-configured it could be running constantly and in which case find the network administrator who installed it and get them to configure it correctly.

I appreciate the help you all. I will talk to the network admins and see if they run a "vunerability scanner".

Blaz - I will provide you with the information in about an hour. I'm about to leave for work once I get my new PC up and going. (It's a sweet Core 2 Duo, 2GB 800Mhz DDR2, ASUS mobo, cheap video card, 7200rpm SATA WD. I had to build it on a budget, but it beats my 2 year old Sony Vaio VGN-A290 like crazy. Sorry, had to gloat for a moment... just woke up. ;P)

Blaz -
1) The error logs I posted were from one of the DMZ servers. It's VM-IP is 192.168.12.80, but under network connections in Windows it is assigned the static ip of 65.83.xx.xx . (Sorry for the .xx.xx, but the site is live, but under development and marketing wouldn't like me posting the site yet.)
2) 192.168.12.1 is the Default Gateway for that web server. I'm confused now.

On my development (internal) server.
1) The error logs look a little different, example:
"[Fri Sep 28 09:45:39 2007] [error] [client 172.16.56.44] File does not exist: D:/Apache2.2/htdocs/procore_solutions/global/flash/procore.swf, referer: http://prowebdev-vm/index.shtml"
 -which is standard/normal. 172.16.56.44 is my static IP on the LAN/domain.
So, I guess  the problem is mainly my DMZ servers.


DMZ Error Log:
[Mon Oct 08 10:47:23 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.5.4
[Mon Oct 08 10:47:23 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.2.3
[Mon Oct 08 10:47:23 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.9.0
[Mon Oct 08 10:47:24 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.9.0.1
[Mon Oct 08 10:47:24 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.8.2.2
[Mon Oct 08 10:47:26 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.7.0-pl2
[Mon Oct 08 10:47:29 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.6.4-pl4
[Mon Oct 08 10:47:29 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.8.1
[Mon Oct 08 10:47:30 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.2.7
[Mon Oct 08 10:47:30 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpMyAdmin-2.2.0
[Mon Oct 08 10:47:31 2007] [error] [client 192.168.12.1] File does not exist: D:/Apache2.2/htdocs/procore_solutions/phpmyadmin


The last line in my access.log is:
192.168.12.1 - - [28/Sep/2007:16:07:18 -0400] "GET / HTTP/1.0" 403 202
192.168.12.1 - - [28/Sep/2007:16:07:20 -0400] "GET / HTTP/1.0" 403 202
192.168.12.1 - - [28/Sep/2007:16:07:22 -0400] "GET / HTTP/1.0" 403 202
192.168.12.1 - - [28/Sep/2007:16:07:34 -0400] "GET / HTTP/1.0" 403 202


Maybe I should review my httpd.conf again or check out the other redundant server in the DMZ.

Any suggestions would be appreciated!

Also, I checked the other two DMZ servers that host cobbenergy.com and cobbemc.com that I configured as well, and they do not have the same problem. They have different IPs, but use the same gateway. If that information helps at all. Thanks.

Correct.

So, any ideas why it is searching for paths on my 2 redundant DMZ VM servers? It doesn't do this on the other DMZ VM servers hosting our other sites.

Alright, I talked to IT of our parent company who hosts the VMs/servers. He said it's the load balancer running health checks on the system. Is there anyway to configure httpd.conf to ignore these 404 requests from the 192.168.12.1?

In general - no. But it is quite easy to configure yourself to ignore these 404 errors in the log :-)

Maybe you could do some rewrites to check for these standard probing URLs and redirect them to a page (see mod_rewrite). But probably this would cause an alarm in your IT department - a security leak URL address would respond to the requests.