[Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Google Sitemaps

Posted on 2006-06-02
9
Medium Priority
?
591 Views
Last Modified: 2013-12-16
Hello!

I'm using Google Sitemaps and the Python Sitemaps Generator.

I've tried about every option in the config file, and the only one that's really suitable is the external urllist.txt option.

Scanning the filesystem is not sutable as it has too many files I don't want in the sitemap. Plus, I do a lot of rewriting so the SE friendly URLS will not be included.
Scanning the access logs is not suitable because it also lists 404 errors.

And I cannot really keep the urls.txt file updated manually, so I wanted to ask if anyone has any ideas or knows of any scripts that will keep this file updated by scanning through HTTP links on the website?

Thanks in advance!
-Julian.
0
Comment
Question by:Julian Matz
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 4

Accepted Solution

by:
ChrisMacleod earned 2000 total points
ID: 16818751
0
 
LVL 15

Expert Comment

by:periwinkle
ID: 16819514
There are many, many 3rd party solutions for google sitemaps;  see:

http://code.google.com/sm_thirdparty.html

In particular, look at the Downloadable Tools and the Online Generators - perhaps one of these will work well for you?
0
 
LVL 10

Expert Comment

by:desertcities
ID: 16821381
Hi Julian,

I've had good success with Audit My PC's online generator:
<http://www.auditmypc.com/free-sitemap-generator.asp>

And believe it or not, Coffeecup Software has a very decent Sitemapper program.  I actually enjoy using it as it makes very clean sitemaps with graphics and for Google it has many options for various files and folders and queries to ignore.  You can use their 30 day trial version.  Their paid version is $29.00.  

<http://www.coffeecup.com/google-sitemapper/>

Mark
0
Free Backup Tool for VMware and Hyper-V

Restore full virtual machine or individual guest files from 19 common file systems directly from the backup file. Schedule VM backups with PowerShell scripts. Set desired time, lean back and let the script to notify you via email upon completion.  

 
LVL 21

Author Comment

by:Julian Matz
ID: 16823591
Hi Periwinkle and Desertcities,
Thanks for your comments, but I was looking for something that runs on my own server, preferably with cron. Something like the Google python script, except one that crawls http-based links and not the file-system.

Hi ChrisMacleod,
Thanks for the link. It's exactly what I was looking for... well, originally I was looking for something that just creates the urllist.txt file so that the google-sitemaps-gen script can use it to create the XML sitemap, but this one does the whole lot at once, which is cool too. It can also be used with cron, so it's perfect from what I can see.

-Julian.
0
 
LVL 4

Expert Comment

by:ChrisMacleod
ID: 16823605
Your welcome Julian, i am think about purchasing this one also.
0
 
LVL 21

Author Comment

by:Julian Matz
ID: 16823628
Well, now I can recommend it :)

It's extremely easy to install, easy to configure, is handy for reporting broken links, easy to use web-based, easy to set up with cron. For 15.00 US, I don't think you can go wrong.

The only thing was that I didn't receive the download link straight after payment. I only got it today, so it probably takes a couple hours for the e-mail to come through, or it's sent manually.... Not an issue really, personally I just hate waiting :)
0
 
LVL 15

Expert Comment

by:periwinkle
ID: 16825687
FWIW, some of the downloadable tools run on your server;  glad that you found a solution that runs welll for you.
0
 
LVL 21

Author Comment

by:Julian Matz
ID: 16827382
Periwinkle, I did check out your link, but the "Downloadable Tools" all seem to be for Windows. I also checked all the PHP resources under "Code Snippets", but in the end I just decided to go with the Standalone Generator because it did look very efficient and it only cost 14 or 15 dollars...
0
 
LVL 15

Expert Comment

by:periwinkle
ID: 16832978
Julian - no problem - just wanted to point out (for posterity) that not everything at that link was windows-based.  The perl programs would have run as well on your server.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Google always has something new and amazing up its sleeve, and the most current thing that they have been working on is another step in the evolution of Google Search, from machine learning to its brilliant successor, deep learning.
Dramatic changes are revolutionizing how we build and use technology. Every company is automating, digitizing, and modernizing operations. We need a better, more connected way to work together as teams so we can harness the insights from our system…
The purpose of this video is to demonstrate how to insert an Iframe into WordPress. This will be demonstrated using a Windows 8 PC. Go to your WordPress login page. This will look like the following: mywebsite.com/wp-login.php : Open Page or Post…
Learn how to set-up PayPal payment integration in your Wufoo form. Allow your users to remit payment through PayPal upon completion of your online form. This is helpful for collecting membership payments, customer payments, donations, and more.
Suggested Courses
Course of the Month20 days, 10 hours left to enroll

868 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question