alternative to wget to grab entire asp site

Posted on 2007-07-22
Last Modified: 2013-11-19
I've been using wget for years, and love it. Unfortuantely it seems to miss some pages of a site i am trying to mirror locally. This site uses a post form on an asp page to retrieve product details. It seems that wget only retrieves some of the products.

Are there any other solutions that will grab me a local copy of this site?

the two commands i have tried are:
  nohup wget -nc -r -w14 --random-wait -olog &
  nohup wget -m &

both lacked the same files..
Question by:Wibble_
    LVL 3

    Expert Comment

    Try adding -U "Mozilla/5.0 (compatible; MSIE 6.0; Windows NT 5.1)"  to disguise wget as Browser, a lot sites are blocking wget.
    LVL 7

    Expert Comment

    In addition to the suggestion above, 'curl' is a similar command with similar capabilities.
    LVL 3

    Accepted Solution

    WebHTTrack is a good alternative...
    I use it mostly under windows but there's also a linux port, you can get it there:

    Featured Post

    How your wiki can always stay up-to-date

    Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
    - Increase transparency
    - Onboard new hires faster
    - Access from mobile/offline

    Join & Write a Comment

    Author Note: Since this E-E article was originally written, years ago, formal testing has come into common use in the world of PHP.  PHPUnit ( and similar technologies have enjoyed wide adoption, making it possib…
    Password hashing is better than message digests or encryption, and you should be using it instead of message digests or encryption.  Find out why and how in this article, which supplements the original article on PHP Client Registration, Login, Logo…
    The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
    Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

    746 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now