Qw M
asked on
URL Friendly rewrite is blocking the crawl spider
Hi,
I have a big problem. I made a URL rewrite for my website. www.petmark.org
Now the crawl spider doesn't index all my pages.
For example I tried to test the crawl spider at:
http://tentblogger.com/spider/index.php
and only some pages are indexed.
Can you please help me!
Is my link structure wrong?
Please help me!
Thank you,
I have a big problem. I made a URL rewrite for my website. www.petmark.org
Now the crawl spider doesn't index all my pages.
For example I tried to test the crawl spider at:
http://tentblogger.com/spider/index.php
and only some pages are indexed.
Can you please help me!
Is my link structure wrong?
Please help me!
Thank you,
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
They don't normally 'index' all product pages even when you submit a site map. It is difficult for all your pages to have the kind of page rank or ratings that will bring them to the top in a search.
ASKER
Thank you!