Celebrate National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

creating a mirror with wget

Posted on 2009-05-12
5
Medium Priority
?
614 Views
Last Modified: 2012-05-06
Hi All,

I'm trying to create a mirror of a website by using wget. It works pretty well on most pages. The HTML of the problem pages tries to include css files which were not downloaded by wget. What is the easiest way to solve this? See the code below.
<style type="text/css" media="screen">@import "/path/file.css";</style>

Open in new window

0
Comment
Question by:alberthendriks
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 
LVL 8

Expert Comment

by:thetmanvn
ID: 24362491
Pretty details on doing mirror with wget

http://linuxreviews.org/quicktips/wget/

Another Solution: Using HTTrack for Linux

http://www.httrack.com/page/2/en/index.html

Good luck
0
 
LVL 13

Expert Comment

by:qwerty021600
ID: 24362661
<STYLE TYPE="text/css" MEDIA="screen">
<!--
  @import url(/path/file.css);
-->
</STYLE>
0
 
LVL 40

Accepted Solution

by:
omarfarid earned 1000 total points
ID: 24365935
what wget command did you use?
0
 
LVL 2

Author Comment

by:alberthendriks
ID: 24372254
I use the code below. The idea of the strange base-url is that I could string-replace it later. However it appears nowhere, and it doesn't seem relevant.
wget --base="{{__base-url__}}" \
     --convert-links \
     -i urls.txt \
     --user-agent="Firefox faker for migration" \
     --directory-prefix ~/public_html/migratie \
     --mirror \
     --save-headers \
     --limit-rate=45k \
     -N \
     -o wget.log \
     --page-requisites \
     --cache=off \
     --exclude-directories=foo,bar

Open in new window

0
 
LVL 2

Author Closing Comment

by:alberthendriks
ID: 31580436
Nobody seems to know a solution. I chose to give the points to someone who doesn't pretend to have one.
0

Featured Post

7 Extremely Useful Linux Commands for Beginners

Just getting started with Linux? Here's a quick start guide that has 7 commands that we believe will come in handy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

rdate is a Linux command and the network time protocol for immediate date and time setup from another machine. The clocks are synchronized by entering rdate with the -s switch (command without switch just checks the time but does not set anything). …
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Suggested Courses

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question