Link to home
Start Free TrialLog in
Avatar of etech0
etech0Flag for United States of America

asked on

Wget on a URL with the vertical bar symbol (|)

I'm trying to run wget (in Windows XP) on a website with the vertical bar symbol | in its URL, and it won't go through.
How can I get this to work?

Thanks.
Avatar of Dave Baldwin
Dave Baldwin
Flag of United States of America image

You might be able to encode it as '%7C' and get it to go thru.
Avatar of etech0

ASKER

It looks like it got further, but I get an error - "cannot write to " and the URL.
Do you know how to fix this?
I don't even know what you're talking about.  "cannot write to " and the URL sounds like you're trying to write to the page on the server.  That's not likely to happen.  Sure it's not trying to write to the local hard drive?
Avatar of etech0

ASKER

Please see attached screenshot of error message. User generated image
Avatar of etech0

ASKER

Where did I do that?
Thanks.
The normal download is to a directory and file that has the same name as the URL.  '-r' means download everything I guess.  Since you are downloading a page with the query string that has illegal characters for a filename, that is where the problem is coming from.
Avatar of etech0

ASKER

So how do I fix that?
I'm not sure that you can.  Here is the manual: http://www.gnu.org/software/wget/manual/wget.html#Download-Options   It says that there the query strings are going to be part of the file name and there's no option to not do that.
Avatar of etech0

ASKER

There's got to be a way. The only thing I can think of is to use the -nd switch, to specify no directory. The only reason I'm not using that is because the -o switch (to set a destination) was giving me trouble. Can you tell me how to use the -o switch when downloading more than just one page?
Thanks.
On the manual page, it says that the '-o' switch puts everything in ONE file which isn't what you want.  And no, there doesn't have to be a way with wget to do this.  There are other methods with different programming languages where you can choose what you download and what file name you store it under.  But wget isn't that flexible.
Avatar of etech0

ASKER

Not the -0 switch, the -o switch. It sets the destination directory.
I don't think so, this is what I see on http://www.gnu.org/software/wget/manual/wget.html :

‘-o logfile’
‘--output-file=logfile’
    Log all messages to logfile. The messages are normally reported to standard error.


‘-O file’
‘--output-document=file’
    The documents will not be written to the appropriate files, but all will be concatenated together and written to file.
Avatar of etech0

ASKER

What about the -p switch, would that do it?
Avatar of etech0

ASKER

Or maybe using > ?
All options but the '-O' use the complete URL as the file name and that's where your problem is.  The URL you are using has characters that are not legal in Windows file names.  '-r' doesn't do you any good because that for 'recursive' downloads which means downloading all the links in the original URL.
Avatar of etech0

ASKER

So do you have any possible solution for me? I want to download all the mp3 files from that webpage.
There are no direct links to MP3s on that page.  The links all go to a download page or a player, probably so you can't download them directly.
Avatar of etech0

ASKER

I see. Is there any way to get to the mp3s?
I can't read the Hebrew on the page but it looks like you can click on the download links.  You might have to be logged in or signed up to do that.  By the way, even when I put 'English' in the query string, the page did not show in English.
Avatar of etech0

ASKER

There is a button "English" that will switch over the site language.
Ok, if I go to http://www.kolhalashon.com/New/Shiurim.aspx?Lang=English&English=True there is a download button for each item.  But it is thru a download page and not a direct download link.  It did let me download an MP3.
Avatar of etech0

ASKER

Yes. Is there some sort of way to do this with a batch file to get each one?
ASKER CERTIFIED SOLUTION
Avatar of Dave Baldwin
Dave Baldwin
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of etech0

ASKER

Thanks, will look at it.
Yes, I know. That's why I only wanted from that specific page - of a certain artist.
Thanks.
Avatar of etech0

ASKER

Looks like I'm stuck with doing this manually.
Thanks for trying, though!
You're welcome, sorry it didn't work better for you.