Solved

Application.cfm - Error handling and bots

Posted on 2014-04-24
11
196 Views
Last Modified: 2014-05-17
I recently added Application.cfm to my site with error handling that will email me when an error occurs.  My ErrorHandler.cfm

<cfsavecontent variable="errortext">
      <cfoutput>
        An error occurred: http://#cgi.server_name##cgi.script_name#?#cgi.query_string#<br />
        Time: #dateFormat(now(), "short")# #timeFormat(now(), "short")#<br />
       
        <cfdump var="#error#" label="Error">
        <cfdump var="#form#" label="Form">
        <cfdump var="#url#" label="URL">
      </cfoutput>
</cfsavecontent>

<!--- finally email yourself the error details --->

<cfmail to="xxx@xxx.com" from="xxx@xxx.com" subject="Error: #error.message#" type="html">
      #errortext#
</cfmail>

I got hammered by bots which were hitting bad links ... in many occasions they seemed to be adding varibles to my url and others were hitting oldd outfdated urls.  Using some ingenuity, I managed to code most of those error hits away.

Recently, I just made some major changes to a section of the site whcih included changing some urls.  Immediatley I began to get bot hits on my old urls ...! grrrr  I am tired of waking up to 200+ emails ...

My question:  How do bots work?  Will they keep trying urls that fails forever or do they learn and eventually stop trying?  Do they actually ADD variables to an url?  If so, why do they do that ... is it an attack?

Max
0
Comment
Question by:MaxwellTurner
  • 5
  • 3
  • 3
11 Comments
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021514
What kind of variables/parameters are they trying to pass to the url?
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40021548
Stuff like this added onto the end or the url:

/RK=0/RS=NiWUYF0HQ81loD_V9ZAtehOe7Yc-

Also, many of the errors come from a page listing ITEMS - those urls pass a variable called ITEM_NUMBER and sometimes there will be characters added onto the ITEM_NUMBER.  It will just keep trying different 'false' ITEMNUMBERs over and over and over ... these items numbers that are being passed do not exists in the database

MAX
0
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021552
Hmm that's really odd. Sounds like some 'hacker' bots that are looking for vulnerable servers that are easy prey. Honestly I wouldn't worry too much about them and the only way really to stop them would be just to black list the IPs that generate these weird errors. So lets say a certain ip generates more than 4 errors black list them or something of that nature
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40021563
I wish it was that easy but the IP's seem to change regularily ... one of the worst culprits had 'bingbot' in the browser name .... and a google bot too ... some are more obscure names but due to the nature of their activity, I'm pretty sure they are bots.

I have just been coding in CFIF's to deal it - E.g. I check the itemnumber coming in and redirect to a 'Sorry not Found' message.  Normally that is good practice anyways ... I never did it in the first place because the ITEMNUMBERS are validated on the previous page and a false one SHOULD never be passed.

Oh well, c'est la vie I guess ... I'll just have to nab them one by one ... it keeps me gainfully employed anyways!  :)

Max
0
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021568
Haha fair enough sorry for your troubles and that I can't help more.
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 15

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40021861
i do not know if that works, but put script within your code that may help in making bots to do not mail like this:

this following CFC can help you, i have not used it but seeems to solve the problem a bit

http://browscapcfc.riaforge.org/

See this entry might of of some help

http://www.anujgakhar.com/2010/01/26/what-is-the-best-way-to-deal-with-spidersbotscrawlers/
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40045102
I have not found a suitable solution for this question and would like to close it ... does anyone have issue with this?

Max
0
 
LVL 15

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40046845
Before Closing, did you find any thing you would like to share, so improvement can be made, if possible, else you can close down
0
 
LVL 1

Accepted Solution

by:
MaxwellTurner earned 0 total points
ID: 40059791
In the end I just dealt with each error individually using code to eliminate the error.  In most cases the bots were usings old urls and I was getting "variable does not exist" and similar errors, so in some cases I just added a temporary CFPARAM to avoid an error.

Max
0
 
LVL 15

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40060924
this is not a good solution but hit & trail. So Good Luck
0
 
LVL 1

Author Closing Comment

by:MaxwellTurner
ID: 40071918
The solutions posted were not suitable for me problem and I just ended up dealing with the issue by using code to avoid the bot errors
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Introduction Since I wrote the original article about Handling Date and Time in PHP and MySQL (http://www.experts-exchange.com/articles/201/Handling-Date-and-Time-in-PHP-and-MySQL.html) several years ago, it seemed like now was a good time to updat…
Do you come here a lot? Are you lazy like me and don't want to go through the "trouble" of having to click your Dock's Safari icon and then having to click your Experts Exchange Favorites bookmark to get here? Well then this article is for you.
The viewer will learn the basics of jQuery, including how to invoke it on a web page. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery.: (CODE)
This Micro Tutorial will demonstrate how to add subdomains to your content reports. This can be very importing in having a site with multiple subdomains.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now