Solved

How much Web traffic is spiders and bots?

Posted on 2007-04-03
3
508 Views
Last Modified: 2013-12-07
I'd like to find some kind of authoritative analysis or study on the average percentage of page views to a Web site that are coming from spiders and bots. For example, if I don't filter out any spiders or bots from my Webtrends report, can I say that XX% of those page views were not valid? I realize the number could vary but would appreciate any educated guesses or additional reading materials, thanks.
0
Comment
Question by:mamadouthiam
3 Comments
 
LVL 27

Accepted Solution

by:
mrcoffee365 earned 250 total points
ID: 18845912
We find that about 7% of a medium-traffic site is requests from bots.  That comes to 20% of the bandwidth, presumably because the bots always get all of the large docs from the site.
0
 
LVL 33

Assisted Solution

by:shalomc
shalomc earned 250 total points
ID: 18856556
http://arxiv.org/ftp/cs/papers/0701/0701173.pdf

The summary says: "Our best estimate is that spiders contributed 46% of sessions and 20% of the Web traffic."

I think that the ratios may differ greatly with the type of web site. For example, my web site behaves more like mrcoffee365 mentioned.
0
 

Author Comment

by:mamadouthiam
ID: 18859261
Good info so far but would like some more input - will probably split point (reference to articles, docs appreciated).

mama
0

Featured Post

DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Read about why website design really matters in today's demanding market.
This article will inform Clients about common and important expectations from the freelancers (Experts) who are looking at your Gig.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question