Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

wget saving web page help

Posted on 2011-03-23
4
Medium Priority
?
129 Views
Last Modified: 2016-05-10
Hello,

I need a bit of help with wget.

When a user submits a URL I want to use wget to create an archive/backup of that specific page.  I want to include all the contents of the page i.e. css, images, js etc...

I have the following code and it's working about 90% of what I need.

exec("wget -e robots=off --limit-rate=250k -F -P /home/USERNAME/public_html/results/". $rnd1 ."/". $rnd2 ."/"." -p -k -E ". $site_url ."");

Open in new window


The problem with this code is if a user submits a URL like this:

http://techcrunch.com/2011/03/22/digital-textbook-startup-inkling-nabs-multi-million-dollar-investment-from-mcgraw-hill-and-pearson/

The backup will be structured this way:

[ techcrunch.com - Folder ] / [ 2011 - Folder ] / [ 03 - Folder ] / [ 22 - Folder ] / [ digital-textbook-startup-inkling-nabs-multi-million-dollar-investment-from-mcgraw-hill-and-pearson - Folder ]

Techcrunch File
The html will load all the images from main site (techcrunch.com)

However if the user submits a URL like this:

http://blog.joerogan.net/archives/2889

The backup will contain all the images, css, etc...

Joerogan File


I hope this makes sense.  If not I will try to clarify.
0
Comment
Question by:jambla
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
4 Comments
 
LVL 9

Accepted Solution

by:
absx earned 2000 total points
ID: 35198540
Hi,

There's just too many features in wget to ever get the command correct manually. I would suggest playing around with a tool like wgetGUI (http://www.jensroesner.de/wgetgui/), until you have a set of options that does exactly what you need, and then picking these arguments for the script.
0
 

Author Comment

by:jambla
ID: 35202147
Hello absx,

Thanks for the link, I will have a look to see if it can help me out.


Any one else have any suggestions?
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

FreeBSD on EC2 FreeBSD (https://www.freebsd.org) is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
These days socially coordinated efforts have turned into a critical requirement for enterprises.
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …
Suggested Courses

597 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question