Advertisement

05.27.2008 at 06:56AM PDT, ID: 23434814
[x]
Attachment Details
[x]
The Solution Rating System

With so many solutions, how can you tell which solutions are most likely to help you and which ones are not? To provide you with a tool to use, we rate our solutions based on various elements that most accurately determine if a solution is a quality solution. To explain what factors affect the solution rating, here are the elements we take into consideration when formulating our solution rating.

  • The Grade of the Solution
  • The Zone Rank of the Expert Providing the Solution
  • The Number of Author and Expert Comments
  • The Number of Experts Contributing
  • The Feedback of the Community

Your Input Matters
Because of the way the system is set up, the most important variable in this equation is you. As a member of Experts Exchange, you are able to cast your vote on the quality of the solutions in regard to how complete, accurate, helpful and easy to understand each solution is. When you provide your feedback, each rating is adjusted accordingly. So, if you see a solution that has a poor rating that you think is a good solution, let us know by rating it. As you do, the rating will be adjusted and will become more accurate for other members of our site.

If you have any suggestions that you would like to make for our rating system, please ask a question in the Suggestions Zone of Community Support.

Thank you!

7.6

Stopping Googlebot using up ridiculous amounts of bandwidth

Asked by comcar in Search Engines, Miscellaneous Web Development, Internet Search Engine Optimization

Tags:

Hi,

Our bandwidth is being almost entirely taken up by Googlebot. Whilst we usually don't go over 100GB a month of traffic, Google has on it's own used up around 270GB, combined with our normal traffic this has sent us way over our limit.

We had an issue with Yahoo's bot but used the robots.txt file to set a crawl delay and that is back down to a reasonable figure, but using Google Webmasters crawl rate tool (set to slow) has made no difference at all and the bandwidth figures continue to rise. We've tried to block Googlebot from certain folders but this still makes no difference, again using robots.txt.

We don't want to risk harming our high ranking on Google but we had achieved this when it was using 20GB or less.

Whilst changes are made to the site constantly no other bots have reacted this badly, and when Yahoo did that was easily controlled. Last month the total bandwidth capped at about 280, and this month it's already at 355GB and estimated for 405GB. Please help! Does anyone know a way of slowing Googlebot down?Start Free Trial
[+][-]05.27.2008 at 08:27PM PDT, ID: 21657491

At Experts Exchange, members can ask their questions to thousands of technology professionals, also known as Experts. Experts compete and collaborate to answer those questions by leaving comments like this one.

Start your 7-day free trial to view this Expert Comment or ask the Experts your question.

 
[+][-]05.27.2008 at 11:53PM PDT, ID: 21658241

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 7-day free trial to view this Author Comment or ask the Experts your question.

 
[+][-]05.28.2008 at 12:06AM PDT, ID: 21658284

View this solution now by starting your 7-day free trial. Setting up your free trial is quick, easy, and secure. We will return you to this solution, unlocked, when you're done.

 

About this solution

Zones: Search Engines, Miscellaneous Web Development, Internet Search Engine Optimization
Tags: Bots, Bandwidth, Traffic
Sign Up Now!
Solution Provided By: jason1178
Participating Experts: 2
Solution Grade: B
 
 
[+][-]05.28.2008 at 12:36AM PDT, ID: 21658410

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 7-day free trial to view this Author Comment or ask the Experts your question.

 
 
Loading Advertisement...
20080716-EE-VQP-32 / EE_QW_2_20070628