?
Solved

Application.cfm - Error handling and bots

Posted on 2014-04-24
11
Medium Priority
?
205 Views
Last Modified: 2014-05-17
I recently added Application.cfm to my site with error handling that will email me when an error occurs.  My ErrorHandler.cfm

<cfsavecontent variable="errortext">
      <cfoutput>
        An error occurred: http://#cgi.server_name##cgi.script_name#?#cgi.query_string#<br />
        Time: #dateFormat(now(), "short")# #timeFormat(now(), "short")#<br />
       
        <cfdump var="#error#" label="Error">
        <cfdump var="#form#" label="Form">
        <cfdump var="#url#" label="URL">
      </cfoutput>
</cfsavecontent>

<!--- finally email yourself the error details --->

<cfmail to="xxx@xxx.com" from="xxx@xxx.com" subject="Error: #error.message#" type="html">
      #errortext#
</cfmail>

I got hammered by bots which were hitting bad links ... in many occasions they seemed to be adding varibles to my url and others were hitting oldd outfdated urls.  Using some ingenuity, I managed to code most of those error hits away.

Recently, I just made some major changes to a section of the site whcih included changing some urls.  Immediatley I began to get bot hits on my old urls ...! grrrr  I am tired of waking up to 200+ emails ...

My question:  How do bots work?  Will they keep trying urls that fails forever or do they learn and eventually stop trying?  Do they actually ADD variables to an url?  If so, why do they do that ... is it an attack?

Max
0
Comment
Question by:MaxwellTurner
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 3
  • 3
11 Comments
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021514
What kind of variables/parameters are they trying to pass to the url?
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40021548
Stuff like this added onto the end or the url:

/RK=0/RS=NiWUYF0HQ81loD_V9ZAtehOe7Yc-

Also, many of the errors come from a page listing ITEMS - those urls pass a variable called ITEM_NUMBER and sometimes there will be characters added onto the ITEM_NUMBER.  It will just keep trying different 'false' ITEMNUMBERs over and over and over ... these items numbers that are being passed do not exists in the database

MAX
0
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021552
Hmm that's really odd. Sounds like some 'hacker' bots that are looking for vulnerable servers that are easy prey. Honestly I wouldn't worry too much about them and the only way really to stop them would be just to black list the IPs that generate these weird errors. So lets say a certain ip generates more than 4 errors black list them or something of that nature
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40021563
I wish it was that easy but the IP's seem to change regularily ... one of the worst culprits had 'bingbot' in the browser name .... and a google bot too ... some are more obscure names but due to the nature of their activity, I'm pretty sure they are bots.

I have just been coding in CFIF's to deal it - E.g. I check the itemnumber coming in and redirect to a 'Sorry not Found' message.  Normally that is good practice anyways ... I never did it in the first place because the ITEMNUMBERS are validated on the previous page and a false one SHOULD never be passed.

Oh well, c'est la vie I guess ... I'll just have to nab them one by one ... it keeps me gainfully employed anyways!  :)

Max
0
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021568
Haha fair enough sorry for your troubles and that I can't help more.
0
 
LVL 16

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40021861
i do not know if that works, but put script within your code that may help in making bots to do not mail like this:

this following CFC can help you, i have not used it but seeems to solve the problem a bit

http://browscapcfc.riaforge.org/

See this entry might of of some help

http://www.anujgakhar.com/2010/01/26/what-is-the-best-way-to-deal-with-spidersbotscrawlers/
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40045102
I have not found a suitable solution for this question and would like to close it ... does anyone have issue with this?

Max
0
 
LVL 16

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40046845
Before Closing, did you find any thing you would like to share, so improvement can be made, if possible, else you can close down
0
 
LVL 1

Accepted Solution

by:
MaxwellTurner earned 0 total points
ID: 40059791
In the end I just dealt with each error individually using code to eliminate the error.  In most cases the bots were usings old urls and I was getting "variable does not exist" and similar errors, so in some cases I just added a temporary CFPARAM to avoid an error.

Max
0
 
LVL 16

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40060924
this is not a good solution but hit & trail. So Good Luck
0
 
LVL 1

Author Closing Comment

by:MaxwellTurner
ID: 40071918
The solutions posted were not suitable for me problem and I just ended up dealing with the issue by using code to avoid the bot errors
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Do you come here a lot? Are you lazy like me and don't want to go through the "trouble" of having to click your Dock's Safari icon and then having to click your Experts Exchange Favorites bookmark to get here? Well then this article is for you.
This article offers some helpful and general tips for safe browsing and online shopping. It offers simple and manageable procedures that help to ensure the safety of one's personal information and the security of any devices.
The viewer will learn how to count occurrences of each item in an array.
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …
Suggested Courses

800 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question