Solved

Application.cfm - Error handling and bots

Posted on 2014-04-24
11
202 Views
Last Modified: 2014-05-17
I recently added Application.cfm to my site with error handling that will email me when an error occurs.  My ErrorHandler.cfm

<cfsavecontent variable="errortext">
      <cfoutput>
        An error occurred: http://#cgi.server_name##cgi.script_name#?#cgi.query_string#<br />
        Time: #dateFormat(now(), "short")# #timeFormat(now(), "short")#<br />
       
        <cfdump var="#error#" label="Error">
        <cfdump var="#form#" label="Form">
        <cfdump var="#url#" label="URL">
      </cfoutput>
</cfsavecontent>

<!--- finally email yourself the error details --->

<cfmail to="xxx@xxx.com" from="xxx@xxx.com" subject="Error: #error.message#" type="html">
      #errortext#
</cfmail>

I got hammered by bots which were hitting bad links ... in many occasions they seemed to be adding varibles to my url and others were hitting oldd outfdated urls.  Using some ingenuity, I managed to code most of those error hits away.

Recently, I just made some major changes to a section of the site whcih included changing some urls.  Immediatley I began to get bot hits on my old urls ...! grrrr  I am tired of waking up to 200+ emails ...

My question:  How do bots work?  Will they keep trying urls that fails forever or do they learn and eventually stop trying?  Do they actually ADD variables to an url?  If so, why do they do that ... is it an attack?

Max
0
Comment
Question by:MaxwellTurner
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 3
  • 3
11 Comments
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021514
What kind of variables/parameters are they trying to pass to the url?
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40021548
Stuff like this added onto the end or the url:

/RK=0/RS=NiWUYF0HQ81loD_V9ZAtehOe7Yc-

Also, many of the errors come from a page listing ITEMS - those urls pass a variable called ITEM_NUMBER and sometimes there will be characters added onto the ITEM_NUMBER.  It will just keep trying different 'false' ITEMNUMBERs over and over and over ... these items numbers that are being passed do not exists in the database

MAX
0
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021552
Hmm that's really odd. Sounds like some 'hacker' bots that are looking for vulnerable servers that are easy prey. Honestly I wouldn't worry too much about them and the only way really to stop them would be just to black list the IPs that generate these weird errors. So lets say a certain ip generates more than 4 errors black list them or something of that nature
0
Stressed Out?

Watch some penguins on the livecam!

 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40021563
I wish it was that easy but the IP's seem to change regularily ... one of the worst culprits had 'bingbot' in the browser name .... and a google bot too ... some are more obscure names but due to the nature of their activity, I'm pretty sure they are bots.

I have just been coding in CFIF's to deal it - E.g. I check the itemnumber coming in and redirect to a 'Sorry not Found' message.  Normally that is good practice anyways ... I never did it in the first place because the ITEMNUMBERS are validated on the previous page and a false one SHOULD never be passed.

Oh well, c'est la vie I guess ... I'll just have to nab them one by one ... it keeps me gainfully employed anyways!  :)

Max
0
 
LVL 5

Expert Comment

by:Pasha Kravtsov
ID: 40021568
Haha fair enough sorry for your troubles and that I can't help more.
0
 
LVL 16

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40021861
i do not know if that works, but put script within your code that may help in making bots to do not mail like this:

this following CFC can help you, i have not used it but seeems to solve the problem a bit

http://browscapcfc.riaforge.org/

See this entry might of of some help

http://www.anujgakhar.com/2010/01/26/what-is-the-best-way-to-deal-with-spidersbotscrawlers/
0
 
LVL 1

Author Comment

by:MaxwellTurner
ID: 40045102
I have not found a suitable solution for this question and would like to close it ... does anyone have issue with this?

Max
0
 
LVL 16

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40046845
Before Closing, did you find any thing you would like to share, so improvement can be made, if possible, else you can close down
0
 
LVL 1

Accepted Solution

by:
MaxwellTurner earned 0 total points
ID: 40059791
In the end I just dealt with each error individually using code to eliminate the error.  In most cases the bots were usings old urls and I was getting "variable does not exist" and similar errors, so in some cases I just added a temporary CFPARAM to avoid an error.

Max
0
 
LVL 16

Expert Comment

by:Gurpreet Singh Randhawa
ID: 40060924
this is not a good solution but hit & trail. So Good Luck
0
 
LVL 1

Author Closing Comment

by:MaxwellTurner
ID: 40071918
The solutions posted were not suitable for me problem and I just ended up dealing with the issue by using code to avoid the bot errors
0

Featured Post

Turn Insights Into Action

You’ve already invested in ITSM tools, chat applications, automation utilities, and more. Fortify these solutions with intelligent communications so you can drive business processes forward.

With xMatters, you'll never miss a beat.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction If you're like most people, you have occasionally made a typographical error when you're entering information into an online form.  And to your consternation, the browser remembers the error, and offers to autocomplete your future entr…
If you are a web developer, you would be aware of the <iframe> tag in HTML. The <iframe> stands for inline frame and is used to embed another document within the current HTML document. The embedded document could be even another website.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.

691 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question