• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 554
  • Last Modified:

LWP and LIBCURL

Does anyone LWP capbilities when it comes to using it as HTTP client. Is it easier or better the CURL/LIBCURL?

I am trying to decide between using LIBCURL and LWP in perl scripts to communicate to servers via http.

I know CURL is more portable. I am not sure if LWP will run on IBM AIX and HP UX servers.
0
munzer
Asked:
munzer
  • 5
  • 4
  • 3
  • +1
2 Solutions
 
ahoffmannCommented:
IIRC LWP uses perl's built in functionality while LIBCURL requres curl (and/or its libs) to be installed, henc LWP might be easyer portable
0
 
clockwatcherCommented:
LWP is certainly the more perlish option.  You'll find a lot more examples using it.  As for portability, I would imagine that it runs on anything that has IO::Socket  support.  So it should be extremely portable.  It runs on AIX-- pretty sure it's included as a standard module on AIX.  I can't remember having had to install it on my box and I'm using it.  Personally have used it on Linux, Windows, AIX, VMS, and Mac OSX.  I have no experience with HP UX but I'm positive that it runs under it-- http://www.nntp.perl.org/group/perl.cpan.testers/1999/08/msg88.html

Libcurl is probably the speedier of the two-- since that's pretty much it's sole purpose.  But the bottleneck is most likely the connection itself rather than the library.  So if it is speedier, I doubt if it's that considerable.  If you do decide to use libcurl, there's a wrapper available http://search.cpan.org/~crisb/WWW-Curl-2.0/easy.pm.  Shelling curl is by far the least perlish way of doing it, but if that's the way you want to go it'll work too.
0
 
clockwatcherCommented:
0
Cloud Class® Course: CompTIA Cloud+

The CompTIA Cloud+ Basic training course will teach you about cloud concepts and models, data storage, networking, and network infrastructure.

 
munzerAuthor Commented:
What do you all think of this. Does this mean curl is more flexible or they can both do the same job.


LWP::Simple's functions are handy for simple cases, but its functions don't support cookies or authorization; they don't support setting header lines in the HTTP request; and generally, they don't support reading header lines in the HTTP response (most notably the full HTTP error message, in case of an error). To get at all those features, you'll have to use the full LWP class model.

I saw a page where they did not list ibm aix as a supported platform. I just cant find it.
0
 
ahoffmannCommented:
if your ibm runs perl, it most likely also has the LWP module, while you need to verify if curl module can be used ('cause it requires an additional libcurl.a)
0
 
clockwatcherCommented:
There's a reason it's called LWP::Simple.  If you're interested in setting header information, modifying the request body or manipulating the response-- then use LWP::UserAgent, HTTP::Request, and HTTP::Response directly rather than LWP::Simple.  You have way more control over what you can do with it than you will have with curl.
0
 
clockwatcherCommented:
The page you think you found is definitely wrong if they said it won't run on any version of AIX.  

Here's an example of using it from my AIX:

  perl -MLWP::Simple -e 'print join("\n",(`uname -a`,head("http://www.experts-exchange.com")))'

  AIX DTI 2 5 000CBB9C4C00

  text/html;charset=UTF-8
  33371

  Apache-Coyote/1.1

I also went ahead and built it from sources on my AIX box just to see if it had any problems.  Turns out it's a pure perl module.  You've got to have a pretty messed up perl to have it not work and build successfully on a system.  AIX certainly doesn't have any problems with it:  I seriously doubt if any OS claiming to run perl successfully would have a problem with it.

[mark@DTI:~]$ uname -a
AIX DTI 2 5 000CBB9C4C00
[mark@DTI:~]$ cd libwww-perl-5.806
[mark@DTI:~/libwww-perl-5.806]$ perl -v

This is perl, v5.8.0 built for aix

Copyright 1987-2002, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using `man perl' or `perldoc perl'.  If you have access to the
Internet, point your browser at http://www.perl.com/, the Perl Home Page.

[mark@DTI:~/libwww-perl-5.806]$ perl Makefile.PL

You appear to be directly connected to the Internet.  I have some tests
that tries to access some sites on the net to verify that the new HTTP/1.1
support works as it should.

Do you want to enable these tests? [y]

This package comes with some sample programs that I can try
to install in /usr/local/bin.

   Note that you can avoid these questions by passing
   the '-n' option to 'Makefile.PL'.

Do you want to install lwp-request? [y] n
Do you want to install lwp-mirror? [y] n
Do you want to install lwp-rget? [y] n
Do you want to install lwp-download? [y] n

Checking for URI........... ok
Checking for HTML::Parser.. ok
Checking for MIME::Base64.. ok
Checking for Net::FTP...... ok
Checking for Digest::MD5 .. ok

Writing Makefile for LWP
[mark@DTI:~/libwww-perl-5.806]$ make test
Manifying blib/man3/LWP::RobotUA.3
Manifying blib/man3/LWP::Debug.3
Manifying blib/man3/Net::HTTP.3
Manifying blib/man3/HTTP::Daemon.3
Manifying blib/man3/LWP::Authen::Ntlm.3
Manifying blib/man3/HTTP::Cookies.3
Manifying blib/man3/LWP.3
Manifying blib/man3/LWP::ConnCache.3
Manifying blib/man3/LWP::Protocol.3
Manifying blib/man3/lwptut.3
Manifying blib/man3/HTTP::Request::Common.3
Manifying blib/man3/HTTP::Message.3
Manifying blib/man3/HTML::Form.3
Manifying blib/man3/HTTP::Cookies::Netscape.3
Manifying blib/man3/File::Listing.3
Manifying blib/man3/WWW::RobotRules::AnyDBM_File.3
Manifying blib/man3/Net::HTTP::NB.3
Manifying blib/man3/HTTP::Status.3
Manifying blib/man3/lwpcook.3
Manifying blib/man3/LWP::MediaTypes.3
Manifying blib/man3/HTTP::Date.3
Manifying blib/man3/HTTP::Negotiate.3
Manifying blib/man3/HTTP::Request.3
Manifying blib/man3/LWP::DebugFile.3
Manifying blib/man3/WWW::RobotRules.3
Manifying blib/man3/HTTP::Headers::Util.3
Manifying blib/man3/HTTP::Response.3
Manifying blib/man3/HTTP::Cookies::Microsoft.3
Manifying blib/man3/HTTP::Headers.3
Manifying blib/man3/LWP::Simple.3
Manifying blib/man3/LWP::UserAgent.3
Manifying blib/man3/Bundle::LWP.3
Manifying blib/man3/LWP::MemberMixin.3
/usr/local/bin/perl t/TEST 0
base/common-req........ok
base/cookies...........ok
base/date..............ok
base/headers-auth......ok
base/headers-etag......ok
base/headers-util......ok
base/headers...........ok
base/http..............ok
base/listing...........ok
base/mediatypes........ok
base/message-old.......ok
base/message-parts.....ok
base/message...........ok
base/negotiate.........ok
base/protocols.........ok
base/response..........ok
base/status............ok
base/ua................ok
html/form-param........ok
html/form..............ok
robot/rules-dbm........ok
robot/rules............ok
robot/ua-get...........ok
robot/ua...............ok
local/autoload-get.....ok
local/autoload.........ok
local/chunked..........ok 5/5# No tests run!
local/chunked..........ok
local/get..............ok
local/http-get.........ok
local/http.............ok
local/protosub.........ok
live/apache-listing....ok
live/apache............ok
live/jigsaw-auth-b.....ok
live/jigsaw-auth-d.....ok
live/jigsaw-chunk......ok
live/jigsaw-md5-get....ok
live/jigsaw-md5........ok
live/jigsaw-neg-get....ok
live/jigsaw-neg........ok
live/jigsaw-te.........ok
live/validator.........ok
All tests successful.
Files=42, Tests=833, 70 wallclock secs ( 4.46 cusr +  1.58 csys =  6.04 CPU)

0
 
munzerAuthor Commented:
clockwatcher

thanks, would you say that an http api using perl/curl to or perl/lwp can be reliable to send messages to database and retrieve messages from database constantly. I want to schedule th eperl as batch jobs running continuously.

Somone told me that http is goof for retreiving page and not tunneling data and I should check tcp/ip option.
0
 
Adam314Commented:
HTTP runs over tcp/ip
0
 
munzerAuthor Commented:
yes I knew that. But http is an application that runs on top of tcp/ip what is used to retreive web pages.

For exampl, you use smtp for email, ftp for file transfer, etc.  You do not use http for email or file transfer. It might not work.

0
 
Adam314Commented:
I was commenting in regards to your last comment:
>>Somone told me that http is goof for retreiving page and not tunneling data and I should check tcp/ip option. <<

If you are trying to talk to a webserver, then you will want to use HTTP.  If you are talking to some other server, you will want to use whatever protocol is needed by that server.
0
 
munzerAuthor Commented:
well we have two unix remote servers: one has an oracle database and the other has a file system.
the database machine is behind a firewall. The oracle application server (mod_plsql) which allows you to run oracle stored procedures over the web is outside the firewall (3rd machine in front of the database).

The remote filesystem server basically will send a message (Quey or Update) to the database and then does its file management based on that. For exampl,e if a user flagged a certain book to be deleted in the database,a perl batch job will be sent to the database to perform a query, and the db will send  that book number. perl will delete that book from the filesystem and then send an update message to the database telling it the book has been deleted. The database updates a flag and then send perl on the other server confirmation that "Update OK" or "Update failed'.

This is where I am wondering whether "HTTP" will work or not. The problems that need to be handled are timeouts and errors.
0
 
ahoffmannCommented:
either you have to set up other server processes (for example a web server with CGI)  to handle your various requests, or simply use ssh to start the remote perl scripts
0
 
munzerAuthor Commented:
well correct me if i am wrong

if the perl machine wants to only make http calls to that oracle web server then I do not need to have a web server/cgi running on that perl machine. BEcause it simply makes a http request using CURL or LWP and receives a response.

If the database needs to run any perl scripts on the perl machine then YES I must have a web server on the perl machine.
0
 
Adam314Commented:
That sounds correct.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

The 14th Annual Expert Award Winners

The results are in! Meet the top members of our 2017 Expert Awards. Congratulations to all who qualified!

  • 5
  • 4
  • 3
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now