Solved

creating a mirror with wget

Posted on 2009-05-12
5
603 Views
Last Modified: 2012-05-06
Hi All,

I'm trying to create a mirror of a website by using wget. It works pretty well on most pages. The HTML of the problem pages tries to include css files which were not downloaded by wget. What is the easiest way to solve this? See the code below.
<style type="text/css" media="screen">@import "/path/file.css";</style>

Open in new window

0
Comment
Question by:alberthendriks
5 Comments
 
LVL 8

Expert Comment

by:thetmanvn
ID: 24362491
Pretty details on doing mirror with wget

http://linuxreviews.org/quicktips/wget/

Another Solution: Using HTTrack for Linux

http://www.httrack.com/page/2/en/index.html

Good luck
0
 
LVL 13

Expert Comment

by:qwerty021600
ID: 24362661
<STYLE TYPE="text/css" MEDIA="screen">
<!--
  @import url(/path/file.css);
-->
</STYLE>
0
 
LVL 40

Accepted Solution

by:
omarfarid earned 500 total points
ID: 24365935
what wget command did you use?
0
 
LVL 2

Author Comment

by:alberthendriks
ID: 24372254
I use the code below. The idea of the strange base-url is that I could string-replace it later. However it appears nowhere, and it doesn't seem relevant.
wget --base="{{__base-url__}}" \

     --convert-links \

     -i urls.txt \

     --user-agent="Firefox faker for migration" \

     --directory-prefix ~/public_html/migratie \

     --mirror \

     --save-headers \

     --limit-rate=45k \

     -N \

     -o wget.log \

     --page-requisites \

     --cache=off \

     --exclude-directories=foo,bar

Open in new window

0
 
LVL 2

Author Closing Comment

by:alberthendriks
ID: 31580436
Nobody seems to know a solution. I chose to give the points to someone who doesn't pretend to have one.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Daily system administration tasks often require administrators to connect remote systems. But allowing these remote systems to accept passwords makes these systems vulnerable to the risk of brute-force password guessing attacks. Furthermore there ar…
Setting up Secure Ubuntu server on VMware 1.      Insert the Ubuntu Server distribution CD or attach the ISO of the CD which is in the “Datastore”. Note that it is important to install the x64 edition on servers, not the X86 editions. 2.      Power on th…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

943 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now