Link to home
Start Free TrialLog in
Avatar of Qw M
Qw MFlag for United States of America

asked on

URL Friendly rewrite is blocking the crawl spider

Hi,

I have a big problem. I made a URL rewrite for my website. www.petmark.org
Now the crawl spider doesn't index all my pages.

For example I tried to test the crawl spider at:
http://tentblogger.com/spider/index.php
and only some pages are indexed.  

Can you please help me!
Is my link structure wrong?


Please help me!
Thank you,
ASKER CERTIFIED SOLUTION
Avatar of Dave Baldwin
Dave Baldwin
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Qw M

ASKER

@DaveBaldwin: So is Google going to index all my pages?

Thank you!
They don't normally 'index' all product pages even when you submit a site map.  It is difficult for all your pages to have the kind of page rank or ratings that will bring them to the top in a search.