Solved

creating a mirror with wget

Posted on 2009-05-12
5
597 Views
Last Modified: 2012-05-06
Hi All,

I'm trying to create a mirror of a website by using wget. It works pretty well on most pages. The HTML of the problem pages tries to include css files which were not downloaded by wget. What is the easiest way to solve this? See the code below.
<style type="text/css" media="screen">@import "/path/file.css";</style>

Open in new window

0
Comment
Question by:alberthendriks
5 Comments
 
LVL 8

Expert Comment

by:thetmanvn
Comment Utility
Pretty details on doing mirror with wget

http://linuxreviews.org/quicktips/wget/

Another Solution: Using HTTrack for Linux

http://www.httrack.com/page/2/en/index.html

Good luck
0
 
LVL 13

Expert Comment

by:qwerty021600
Comment Utility
<STYLE TYPE="text/css" MEDIA="screen">
<!--
  @import url(/path/file.css);
-->
</STYLE>
0
 
LVL 40

Accepted Solution

by:
omarfarid earned 500 total points
Comment Utility
what wget command did you use?
0
 
LVL 2

Author Comment

by:alberthendriks
Comment Utility
I use the code below. The idea of the strange base-url is that I could string-replace it later. However it appears nowhere, and it doesn't seem relevant.
wget --base="{{__base-url__}}" \

     --convert-links \

     -i urls.txt \

     --user-agent="Firefox faker for migration" \

     --directory-prefix ~/public_html/migratie \

     --mirror \

     --save-headers \

     --limit-rate=45k \

     -N \

     -o wget.log \

     --page-requisites \

     --cache=off \

     --exclude-directories=foo,bar

Open in new window

0
 
LVL 2

Author Closing Comment

by:alberthendriks
Comment Utility
Nobody seems to know a solution. I chose to give the points to someone who doesn't pretend to have one.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Daily system administration tasks often require administrators to connect remote systems. But allowing these remote systems to accept passwords makes these systems vulnerable to the risk of brute-force password guessing attacks. Furthermore there ar…
SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now