Link to home
Start Free TrialLog in
Avatar of Wibble_
Wibble_

asked on

alternative to wget to grab entire asp site

I've been using wget for years, and love it. Unfortuantely it seems to miss some pages of a site i am trying to mirror locally. This site uses a post form on an asp page to retrieve product details. It seems that wget only retrieves some of the products.

Are there any other solutions that will grab me a local copy of this site?

the two commands i have tried are:
  nohup wget -nc -r -w14 --random-wait -olog http://www.SITENAME.com &
and
  nohup wget -m http://www.sitename.com &

both lacked the same files..
Avatar of Liganic
Liganic

Try adding -U "Mozilla/5.0 (compatible; MSIE 6.0; Windows NT 5.1)"  to disguise wget as Browser, a lot sites are blocking wget.
In addition to the suggestion above, 'curl' is a similar command with similar capabilities.
ASKER CERTIFIED SOLUTION
Avatar of AdHe5
AdHe5

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial