Detect and combat possible robot
Twice now I've had a curious situation that hung up my web server. Looking at my ColdFusion logs, I see a lot of hits to a particular page. Many hits early in the morning that hung up the query and apparently hung my CF web service which I had to restart. Typically, if I had some type of robot attacking my site, I would see a bunch of errors coming in, I am not seeing that this time.
How can I discover and combat a possible electronic access to my site?
How do I discover the IP address and URL of each request to my site?
Messages in my Logs:
The request has exceeded the allowable time limit Tag: CFQUERY The specific sequence of files included or processed is: D:\inetpub\wwwRoot\rates\h
Thread: ajp-bio-8014-exec-361, processing template: D:\inetpub\wwwRoot\rates\h
Running: Coldfusion 11 SQL Server Amazon EC2 Server and RDS managed database
Thanks for any help
How can I discover and combat a possible electronic access to my site?
How do I discover the IP address and URL of each request to my site?
Messages in my Logs:
The request has exceeded the allowable time limit Tag: CFQUERY The specific sequence of files included or processed is: D:\inetpub\wwwRoot\rates\h
Thread: ajp-bio-8014-exec-361, processing template: D:\inetpub\wwwRoot\rates\h
Running: Coldfusion 11 SQL Server Amazon EC2 Server and RDS managed database
Thanks for any help
ASKER
Thanks David, I already have one. I am thinking this is not an innocent spider, this is someone trying to screen scrape or worse. How can I determine the URL they are using and their IP address?
in IIS did you configure logging? check for the logs in the intepub\logs directory.
ASKER
I do have logs there, is there an easy/free tool that I can use to read those logs?
ASKER
I just opened the log file using a text editor, there is not a lot in there; seems not to be logging every page view - should I turn that on or is it too much? Do can you tell me best practices and how to do?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
http://www.robotstxt.org/robotstxt.html