• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 461
  • Last Modified:

Spider

Does anyone know where I can get a free script or program
that can list info it finds for a given URL or address?

Eg.
prompt> spider http://where.com/files

and have it index and record what it finds at that address and whatever is underneath it.

It should be for Unix or Linux.

Thanks.
0
scottg
Asked:
scottg
1 Solution
 
MasseyMCommented:
Search Download.com Winfiles.com or Shareware.com
0
 
scottgAuthor Commented:
All those sites are for garbage windows software.  I need a script or program for a real OS.  Sorry.
0
Easily Design & Build Your Next Website

Squarespace’s all-in-one platform gives you everything you need to express yourself creatively online, whether it is with a domain, website, or online store. Get started with your free trial today, and when ready, take 10% off your first purchase with offer code 'EXPERTS'.

 
ndnetCommented:
0
 
scottgAuthor Commented:
Thanks, but they're primarily for index local directories, not crawling web sites.

0
 
evilgregCommented:
Is this something similar to the "web-crawling" question in the CGI topic area? If so, it should not be too hard to make a perl script to do what you are asking. You'll also need to provide more information (see my questions under that topic).
0
 
OttaCommented:
See: http://WWW.InContext.COM
for the InContext "Spider" software.
Not quite $0.00, but under $99.
0
 
scottgAuthor Commented:
Thats web authoring software, not a crawler.

Thanks anyway.
0
 
OttaCommented:
InContext also markets a "crawler".
0
 
scottgAuthor Commented:
Thanks,but I can't even find it on the site.   It's probably too much cash anyway.
0
 
OttaCommented:
The InContext software is named "FlashSite".
That URL contains the text:

| Speed up your Web browsing by using FlashSite,
| winner of the Editor's Choice award from PC Magazine,
| to download all or any part of a Web site
| to your hard drive for customized power-browsing experience.


0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Easily Design & Build Your Next Website

Squarespace’s all-in-one platform gives you everything you need to express yourself creatively online, whether it is with a domain, website, or online store. Get started with your free trial today, and when ready, take 10% off your first purchase with offer code 'EXPERTS'.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now