Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2033
  • Last Modified:

alternative to wget to grab entire asp site

I've been using wget for years, and love it. Unfortuantely it seems to miss some pages of a site i am trying to mirror locally. This site uses a post form on an asp page to retrieve product details. It seems that wget only retrieves some of the products.

Are there any other solutions that will grab me a local copy of this site?

the two commands i have tried are:
  nohup wget -nc -r -w14 --random-wait -olog http://www.SITENAME.com &
and
  nohup wget -m http://www.sitename.com &

both lacked the same files..
0
Wibble_
Asked:
Wibble_
1 Solution
 
LiganicCommented:
Try adding -U "Mozilla/5.0 (compatible; MSIE 6.0; Windows NT 5.1)"  to disguise wget as Browser, a lot sites are blocking wget.
0
 
ezatonCommented:
In addition to the suggestion above, 'curl' is a similar command with similar capabilities.
0
 
AdHe5Commented:
WebHTTrack is a good alternative...
I use it mostly under windows but there's also a linux port, you can get it there:

http://www.httrack.com/page/2/en/index.html
0

Featured Post

Nothing ever in the clear!

This technical paper will help you implement VMware’s VM encryption as well as implement Veeam encryption which together will achieve the nothing ever in the clear goal. If a bad guy steals VMs, backups or traffic they get nothing.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now