Solved

Application.cfm - Error handling and bots

Posted on 2014-04-24
11
198 Views
Last Modified: 2014-05-17
I recently added Application.cfm to my site with error handling that will email me when an error occurs.  My ErrorHandler.cfm

<cfsavecontent variable="errortext">
      <cfoutput>
        An error occurred: http://#cgi.server_name##cgi.script_name#?#cgi.query_string#<br />
        Time: #dateFormat(now(), "short")# #timeFormat(now(), "short")#<br />
       
        <cfdump var="#error#" label="Error">
        <cfdump var="#form#" label="Form">
        <cfdump var="#url#" label="URL">
      </cfoutput>
</cfsavecontent>

<!--- finally email yourself the error details --->

<cfmail to="xxx@xxx.com" from="xxx@xxx.com" subject="Error: #error.message#" type="html">
      #errortext#
</cfmail>

I got hammered by bots which were hitting bad links ... in many occasions they seemed to be adding varibles to my url and others were hitting oldd outfdated urls.  Using some ingenuity, I managed to code most of those error hits away.

Recently, I just made some major changes to a section of the site whcih included changing some urls.  Immediatley I began to get bot hits on my old urls ...! grrrr  I am tired of waking up to 200+ emails ...

My question:  How do bots work?  Will they keep trying urls that fails forever or do they learn and eventually stop trying?  Do they actually ADD variables to an url?  If so, why do they do that ... is it an attack?

Max
0
Comment
Question by:MaxwellTurner
  • 5
  • 3
  • 3
11 Comments
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021514
What kind of variables/parameters are they trying to pass to the url?
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40021548
Stuff like this added onto the end or the url:

/RK=0/RS=NiWUYF0HQ81loD_V9ZAtehOe7Yc-

Also, many of the errors come from a page listing ITEMS - those urls pass a variable called ITEM_NUMBER and sometimes there will be characters added onto the ITEM_NUMBER.  It will just keep trying different 'false' ITEMNUMBERs over and over and over ... these items numbers that are being passed do not exists in the database

MAX
0
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021552
Hmm that's really odd. Sounds like some 'hacker' bots that are looking for vulnerable servers that are easy prey. Honestly I wouldn't worry too much about them and the only way really to stop them would be just to black list the IPs that generate these weird errors. So lets say a certain ip generates more than 4 errors black list them or something of that nature
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40021563
I wish it was that easy but the IP's seem to change regularily ... one of the worst culprits had 'bingbot' in the browser name .... and a google bot too ... some are more obscure names but due to the nature of their activity, I'm pretty sure they are bots.

I have just been coding in CFIF's to deal it - E.g. I check the itemnumber coming in and redirect to a 'Sorry not Found' message.  Normally that is good practice anyways ... I never did it in the first place because the ITEMNUMBERS are validated on the previous page and a false one SHOULD never be passed.

Oh well, c'est la vie I guess ... I'll just have to nab them one by one ... it keeps me gainfully employed anyways!  :)

Max
0
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021568
Haha fair enough sorry for your troubles and that I can't help more.
0
3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

 
LVL 15

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40021861
i do not know if that works, but put script within your code that may help in making bots to do not mail like this:

this following CFC can help you, i have not used it but seeems to solve the problem a bit

http://browscapcfc.riaforge.org/

See this entry might of of some help

http://www.anujgakhar.com/2010/01/26/what-is-the-best-way-to-deal-with-spidersbotscrawlers/
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40045102
I have not found a suitable solution for this question and would like to close it ... does anyone have issue with this?

Max
0
 
LVL 15

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40046845
Before Closing, did you find any thing you would like to share, so improvement can be made, if possible, else you can close down
0
 
LVL 1

Accepted Solution

by:
MaxwellTurner earned 0 total points
ID: 40059791
In the end I just dealt with each error individually using code to eliminate the error.  In most cases the bots were usings old urls and I was getting "variable does not exist" and similar errors, so in some cases I just added a temporary CFPARAM to avoid an error.

Max
0
 
LVL 15

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40060924
this is not a good solution but hit & trail. So Good Luck
0
 
LVL 1

Author Closing Comment

by:MaxwellTurner
ID: 40071918
The solutions posted were not suitable for me problem and I just ended up dealing with the issue by using code to avoid the bot errors
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Preface This is the third article about the EE Collaborative Login Project. A Better Website Login System (http://www.experts-exchange.com/A_2902.html) introduces the Login System and shows how to implement a login page. The EE Collaborative Logi…
I found this questions asking how to do this in many different forums, so I will describe here how to implement a solution using PHP and AJAX. The logical flow for the problem should be: Write an event handler for the first drop down box to get …
Shows how to create a shortcut to site-search Experts Exchange using Google in the Chrome browser. This eliminates the need to type out site:experts-exchange.com whenever you want to search the site. Launch the Search Engine Menu: In chrome, via you…
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now