Solved

Checking a website hub for endless spider link recursion

Posted on 2006-07-13
3
216 Views
Last Modified: 2010-04-20
Is there an application to test a website and/or a directory/subdirectory site on a hard drive to point out the possiblities of endless loop/recursion/crawling of a search engine or spider which could totally decimate available bandwidth/usage limitations?
0
Comment
Question by:ejoeeb
3 Comments
 
LVL 18

Accepted Solution

by:
Sam Panwar earned 125 total points
ID: 17105523
Hi,

Please download GFI Lan guard It is ver good tool which is may be fullfil your requirement

Automatically detect security vulnerabilities on your network
Provides in-depth information about all machines/devices

http://www.gfi.com/downloads/downloads.aspx?pid=LANSS&lid=en
0
 

Assisted Solution

by:rubanek
rubanek earned 125 total points
ID: 17107699
Without knowing the code that controls the visiting crawler you'll never know if it's stuck in a loop.  All you can really do is monitor your sites activity and if you see a bot hanging out for longer than normal times, you should block thier IP.

I would think that most crawlers like google or yahoo would be smart enough to not get stuck in a loop.
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

One of the typical problems I have experienced is when you have to move a web server from one hosting site to another. You normally prepare all on the new host, transfer the site, change DNS and cross your fingers hoping all will be ok on new server…
Introduction This article explores the design of a cache system that can improve the performance of a web site or web application.  The assumption is that the web site has many more “read” operations than “write” operations (this is commonly the ca…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now