ambuli
asked on
download files from web
Hi Experts,
I want to write a small script to download some files from a site. Is there any good example on how to do this? I need to give some options when downloading( The CGI accepts some params)
thank you.
I want to write a small script to download some files from a site. Is there any good example on how to do this? I need to give some options when downloading( The CGI accepts some params)
thank you.
ASKER
Hi thank you. But, when I try using some of these code I am getting the the following error.
I tried a sample from cpan as well, and I am gettiung the same error.
Can't locate object method "get" via package "LWP::UserAgent" at download.pl line 9.
I tried a sample from cpan as well, and I am gettiung the same error.
Can't locate object method "get" via package "LWP::UserAgent" at download.pl line 9.
#!/usr/bin/perl -w
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->timeout(10);
$ua->env_proxy;
my $response = $ua->get('http://search.cpan.org/');
if ($response->is_success) {
print $response->decoded_content; # or whatever
}
else {
die $response->status_line;
}
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
If you want to try a failed response with the same code I gave you in my previous answer, try this (I'm changing the $uri variable):
Hope this helps.
Francisco
use warnings;
use strict;
use IO::Socket;
my $host = 'search.cpan.org'; # Don't add slash at the end here
my $uri = '/notExistentPageTest'; # Put here you complete uri starting by slash always
my $port = 80;
my $s = new IO::Socket::INET (
Proto => 'tcp',
PeerAddr => $host,
PeerPort => $port,
);
$s->write("GET $uri HTTP 1.0\r\n");
$s->write("HOST: $host\r\n");
$s->write("\r\n");
my $response = join '', <$s>;
$response =~ s/\r\n/\n/g;
my $header = substr($response, 0, index($response, "\n\n"));
$response = substr($response, length($header) + 2);
if ($header =~ /^.*?200\s+OK/i) {
print "Request successful, this is the content:\n\n";
print "$response\n";
} else {
print "Response failed, this is the header and content:\n\n";
print "$header\n\n$response\n";
}
Hope this helps.
Francisco
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thank you very much.
When I try it, I am not getting any response. To give you some more context. I have a python script that kind of works. But, I am trying to implement/integrate that in my perl script as part of another program.
I am trying out your examples now, but if you can get anything from the following python script please let me know.
Thank you Francisco.
When I try it, I am not getting any response. To give you some more context. I have a python script that kind of works. But, I am trying to implement/integrate that in my perl script as part of another program.
I am trying out your examples now, but if you can get anything from the following python script please let me know.
Thank you Francisco.
mport urllib
import zipfile
import cStringIO
import datetime
import os
import sys
DOWNLOAD_URL = "http://1ykf/cgi-bin/log.pl?%s"
DOWNLOAD_LOG_TYPES = ('devinfo', 'eventlog', 'javaprocesses')
TEXT_FILES = ['devinfo', 'eventlog', 'javaprocesses']
BUGDISP_FILE = 'bdispbytes'
def usage():
print __doc__
def download_logs(ids):
print "IDs to download:" , ids
params = [('mode', 'bulkids'),
('format', 'Bulk_ZIP'),
]
for eachFileType in DOWNLOAD_LOG_TYPES:
params.append(('attachment', eachFileType))
params.append(['id_list', ",".join(ids)])
url = DOWNLOAD_URL % urllib.urlencode(params)
try:
webFile = urllib.urlopen(url)
except IOError as ioe:
print ioe
sys.exit('Exiting... Failed to download logs')
print "Downloading from: " + url
buf = cStringIO.StringIO(webFile.read())
webFile.close()
return buf
if __name__ == '__main__':
if(len(sys.argv)<2):
usage()
sys.exit(1)
ids = sys.argv[1:]
Hello Ambuli,
I don't see anything in your Python script doing anything special that Perl is not doing. There might be a problem with your Perl installation or permissions to open sockets.
Assuming you are running the script on Linux:
1) Are you running an reasonable recent version of Perl? (e.g. Perl 5.8 or higher - you can see the version by running the command perl -v)
2) Is the UID running the script the same that you use for your Python script when it works?
Besides those questions, are you behind a firewall or proxy? Do you have any kind of antivirus/firewall application that can be blocking Perl's connectivity?
HTH
I don't see anything in your Python script doing anything special that Perl is not doing. There might be a problem with your Perl installation or permissions to open sockets.
Assuming you are running the script on Linux:
1) Are you running an reasonable recent version of Perl? (e.g. Perl 5.8 or higher - you can see the version by running the command perl -v)
2) Is the UID running the script the same that you use for your Python script when it works?
Besides those questions, are you behind a firewall or proxy? Do you have any kind of antivirus/firewall application that can be blocking Perl's connectivity?
HTH
ASKER
Thank you all. I am running it on windows. I will go through a examples and try.
C:\perl -v
This is perl, v5.8.5 built for MSWin32-x86-multi-thread
Copyright 1987-2004, Larry Wall
C:\perl -v
This is perl, v5.8.5 built for MSWin32-x86-multi-thread
Copyright 1987-2004, Larry Wall
https://www.experts-exchange.com/questions/26625647/Perl-Script-to-Download-Files.html?sfQueryTermInfo=1+10+30+download+file+perl+script
You can start woth this example i guess. https://www.experts-exchange.com/questions/26336012/PERL-Download-Script.html?sfQueryTermInfo=1+10+30+download+file+perl+script
This example is dealing with exactly a file descriptor downloads csv file. https://www.experts-exchange.com/questions/23647819/Bad-file-descriptor-with-file-download-script.html?sfQueryTermInfo=1+10+30+download+file+perl+script