etech0
asked on
Wget on a URL with the vertical bar symbol (|)
I'm trying to run wget (in Windows XP) on a website with the vertical bar symbol | in its URL, and it won't go through.
How can I get this to work?
Thanks.
How can I get this to work?
Thanks.
You might be able to encode it as '%7C' and get it to go thru.
ASKER
It looks like it got further, but I get an error - "cannot write to " and the URL.
Do you know how to fix this?
Do you know how to fix this?
I don't even know what you're talking about. "cannot write to " and the URL sounds like you're trying to write to the page on the server. That's not likely to happen. Sure it's not trying to write to the local hard drive?
You can't use '|' or '/' in a Windows file name or path.
http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx
http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx
ASKER
Where did I do that?
Thanks.
Thanks.
The normal download is to a directory and file that has the same name as the URL. '-r' means download everything I guess. Since you are downloading a page with the query string that has illegal characters for a filename, that is where the problem is coming from.
ASKER
So how do I fix that?
I'm not sure that you can. Here is the manual: http://www.gnu.org/software/wget/manual/wget.html#Download-Options It says that there the query strings are going to be part of the file name and there's no option to not do that.
ASKER
There's got to be a way. The only thing I can think of is to use the -nd switch, to specify no directory. The only reason I'm not using that is because the -o switch (to set a destination) was giving me trouble. Can you tell me how to use the -o switch when downloading more than just one page?
Thanks.
Thanks.
On the manual page, it says that the '-o' switch puts everything in ONE file which isn't what you want. And no, there doesn't have to be a way with wget to do this. There are other methods with different programming languages where you can choose what you download and what file name you store it under. But wget isn't that flexible.
ASKER
Not the -0 switch, the -o switch. It sets the destination directory.
I don't think so, this is what I see on http://www.gnu.org/software/wget/manual/wget.html :
‘-o logfile’
‘--output-file=logfile’
Log all messages to logfile. The messages are normally reported to standard error.
‘-O file’
‘--output-document=file’
The documents will not be written to the appropriate files, but all will be concatenated together and written to file.
‘-o logfile’
‘--output-file=logfile’
Log all messages to logfile. The messages are normally reported to standard error.
‘-O file’
‘--output-document=file’
The documents will not be written to the appropriate files, but all will be concatenated together and written to file.
ASKER
What about the -p switch, would that do it?
ASKER
Or maybe using > ?
All options but the '-O' use the complete URL as the file name and that's where your problem is. The URL you are using has characters that are not legal in Windows file names. '-r' doesn't do you any good because that for 'recursive' downloads which means downloading all the links in the original URL.
ASKER
So do you have any possible solution for me? I want to download all the mp3 files from that webpage.
There are no direct links to MP3s on that page. The links all go to a download page or a player, probably so you can't download them directly.
ASKER
I see. Is there any way to get to the mp3s?
I can't read the Hebrew on the page but it looks like you can click on the download links. You might have to be logged in or signed up to do that. By the way, even when I put 'English' in the query string, the page did not show in English.
ASKER
There is a button "English" that will switch over the site language.
Ok, if I go to http://www.kolhalashon.com/New/Shiurim.aspx?Lang=English&English=True there is a download button for each item. But it is thru a download page and not a direct download link. It did let me download an MP3.
ASKER
Yes. Is there some sort of way to do this with a batch file to get each one?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks, will look at it.
Yes, I know. That's why I only wanted from that specific page - of a certain artist.
Thanks.
Yes, I know. That's why I only wanted from that specific page - of a certain artist.
Thanks.
ASKER
Looks like I'm stuck with doing this manually.
Thanks for trying, though!
Thanks for trying, though!
You're welcome, sorry it didn't work better for you.