?
Solved

spider a website to generate sitemap

Posted on 2013-06-25
2
Medium Priority
?
505 Views
Last Modified: 2013-07-01
project: seo for a website where I am not given ftp access.  I need to give them a sitemap.php (not xml) that datechanges

client is satisfied with going to xml-sitemap.com and wants to use their services.

question:
is phpcrawler-0.8
the best tool to gather all the links (images,videos,php pages) of a website
0
Comment
Question by:rgb192
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
2 Comments
 
LVL 111

Accepted Solution

by:
Ray Paseur earned 2000 total points
ID: 39275091
phpcrawler was the worst tool in my experience.  The best tool I have found is Zoom Indexer.  If you just want to copy the site, httrack will usually work.

If you want to add the sitemap to the site, this will be a deal-breaker: "where I am not given ftp access."
0
 

Author Closing Comment

by:rgb192
ID: 39290516
I read about httrack.  Thanks

zoom indexer requires installation.
0

Featured Post

WordPress Tutorial 1: Installation & Setup

WordPress is a very popular option for running your web site and can be used to get your content online quickly for the world to see. This guide will walk you through installing the WordPress server software and the initial setup process.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article discusses how to implement server side field validation and display customized error messages to the client.
There are times when I have encountered the need to decompress a response from a PHP request. This is how it's done, but you must have control of the request and you can set the Accept-Encoding header.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
Suggested Courses

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question