?
Solved

LWP and LIBCURL

Posted on 2007-07-21
15
Medium Priority
?
540 Views
Last Modified: 2013-12-25
Does anyone LWP capbilities when it comes to using it as HTTP client. Is it easier or better the CURL/LIBCURL?

I am trying to decide between using LIBCURL and LWP in perl scripts to communicate to servers via http.

I know CURL is more portable. I am not sure if LWP will run on IBM AIX and HP UX servers.
0
Comment
Question by:munzer
  • 5
  • 4
  • 3
  • +1
15 Comments
 
LVL 51

Assisted Solution

by:ahoffmann
ahoffmann earned 400 total points
ID: 19543387
IIRC LWP uses perl's built in functionality while LIBCURL requres curl (and/or its libs) to be installed, henc LWP might be easyer portable
0
 
LVL 25

Accepted Solution

by:
clockwatcher earned 1600 total points
ID: 19543418
LWP is certainly the more perlish option.  You'll find a lot more examples using it.  As for portability, I would imagine that it runs on anything that has IO::Socket  support.  So it should be extremely portable.  It runs on AIX-- pretty sure it's included as a standard module on AIX.  I can't remember having had to install it on my box and I'm using it.  Personally have used it on Linux, Windows, AIX, VMS, and Mac OSX.  I have no experience with HP UX but I'm positive that it runs under it-- http://www.nntp.perl.org/group/perl.cpan.testers/1999/08/msg88.html

Libcurl is probably the speedier of the two-- since that's pretty much it's sole purpose.  But the bottleneck is most likely the connection itself rather than the library.  So if it is speedier, I doubt if it's that considerable.  If you do decide to use libcurl, there's a wrapper available http://search.cpan.org/~crisb/WWW-Curl-2.0/easy.pm.  Shelling curl is by far the least perlish way of doing it, but if that's the way you want to go it'll work too.
0
 
LVL 25

Expert Comment

by:clockwatcher
ID: 19543423
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:munzer
ID: 19580075
What do you all think of this. Does this mean curl is more flexible or they can both do the same job.


LWP::Simple's functions are handy for simple cases, but its functions don't support cookies or authorization; they don't support setting header lines in the HTTP request; and generally, they don't support reading header lines in the HTTP response (most notably the full HTTP error message, in case of an error). To get at all those features, you'll have to use the full LWP class model.

I saw a page where they did not list ibm aix as a supported platform. I just cant find it.
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 19580463
if your ibm runs perl, it most likely also has the LWP module, while you need to verify if curl module can be used ('cause it requires an additional libcurl.a)
0
 
LVL 25

Expert Comment

by:clockwatcher
ID: 19583731
There's a reason it's called LWP::Simple.  If you're interested in setting header information, modifying the request body or manipulating the response-- then use LWP::UserAgent, HTTP::Request, and HTTP::Response directly rather than LWP::Simple.  You have way more control over what you can do with it than you will have with curl.
0
 
LVL 25

Expert Comment

by:clockwatcher
ID: 19583859
The page you think you found is definitely wrong if they said it won't run on any version of AIX.  

Here's an example of using it from my AIX:

  perl -MLWP::Simple -e 'print join("\n",(`uname -a`,head("http://www.experts-exchange.com")))'

  AIX DTI 2 5 000CBB9C4C00

  text/html;charset=UTF-8
  33371

  Apache-Coyote/1.1

I also went ahead and built it from sources on my AIX box just to see if it had any problems.  Turns out it's a pure perl module.  You've got to have a pretty messed up perl to have it not work and build successfully on a system.  AIX certainly doesn't have any problems with it:  I seriously doubt if any OS claiming to run perl successfully would have a problem with it.

[mark@DTI:~]$ uname -a
AIX DTI 2 5 000CBB9C4C00
[mark@DTI:~]$ cd libwww-perl-5.806
[mark@DTI:~/libwww-perl-5.806]$ perl -v

This is perl, v5.8.0 built for aix

Copyright 1987-2002, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using `man perl' or `perldoc perl'.  If you have access to the
Internet, point your browser at http://www.perl.com/, the Perl Home Page.

[mark@DTI:~/libwww-perl-5.806]$ perl Makefile.PL

You appear to be directly connected to the Internet.  I have some tests
that tries to access some sites on the net to verify that the new HTTP/1.1
support works as it should.

Do you want to enable these tests? [y]

This package comes with some sample programs that I can try
to install in /usr/local/bin.

   Note that you can avoid these questions by passing
   the '-n' option to 'Makefile.PL'.

Do you want to install lwp-request? [y] n
Do you want to install lwp-mirror? [y] n
Do you want to install lwp-rget? [y] n
Do you want to install lwp-download? [y] n

Checking for URI........... ok
Checking for HTML::Parser.. ok
Checking for MIME::Base64.. ok
Checking for Net::FTP...... ok
Checking for Digest::MD5 .. ok

Writing Makefile for LWP
[mark@DTI:~/libwww-perl-5.806]$ make test
Manifying blib/man3/LWP::RobotUA.3
Manifying blib/man3/LWP::Debug.3
Manifying blib/man3/Net::HTTP.3
Manifying blib/man3/HTTP::Daemon.3
Manifying blib/man3/LWP::Authen::Ntlm.3
Manifying blib/man3/HTTP::Cookies.3
Manifying blib/man3/LWP.3
Manifying blib/man3/LWP::ConnCache.3
Manifying blib/man3/LWP::Protocol.3
Manifying blib/man3/lwptut.3
Manifying blib/man3/HTTP::Request::Common.3
Manifying blib/man3/HTTP::Message.3
Manifying blib/man3/HTML::Form.3
Manifying blib/man3/HTTP::Cookies::Netscape.3
Manifying blib/man3/File::Listing.3
Manifying blib/man3/WWW::RobotRules::AnyDBM_File.3
Manifying blib/man3/Net::HTTP::NB.3
Manifying blib/man3/HTTP::Status.3
Manifying blib/man3/lwpcook.3
Manifying blib/man3/LWP::MediaTypes.3
Manifying blib/man3/HTTP::Date.3
Manifying blib/man3/HTTP::Negotiate.3
Manifying blib/man3/HTTP::Request.3
Manifying blib/man3/LWP::DebugFile.3
Manifying blib/man3/WWW::RobotRules.3
Manifying blib/man3/HTTP::Headers::Util.3
Manifying blib/man3/HTTP::Response.3
Manifying blib/man3/HTTP::Cookies::Microsoft.3
Manifying blib/man3/HTTP::Headers.3
Manifying blib/man3/LWP::Simple.3
Manifying blib/man3/LWP::UserAgent.3
Manifying blib/man3/Bundle::LWP.3
Manifying blib/man3/LWP::MemberMixin.3
/usr/local/bin/perl t/TEST 0
base/common-req........ok
base/cookies...........ok
base/date..............ok
base/headers-auth......ok
base/headers-etag......ok
base/headers-util......ok
base/headers...........ok
base/http..............ok
base/listing...........ok
base/mediatypes........ok
base/message-old.......ok
base/message-parts.....ok
base/message...........ok
base/negotiate.........ok
base/protocols.........ok
base/response..........ok
base/status............ok
base/ua................ok
html/form-param........ok
html/form..............ok
robot/rules-dbm........ok
robot/rules............ok
robot/ua-get...........ok
robot/ua...............ok
local/autoload-get.....ok
local/autoload.........ok
local/chunked..........ok 5/5# No tests run!
local/chunked..........ok
local/get..............ok
local/http-get.........ok
local/http.............ok
local/protosub.........ok
live/apache-listing....ok
live/apache............ok
live/jigsaw-auth-b.....ok
live/jigsaw-auth-d.....ok
live/jigsaw-chunk......ok
live/jigsaw-md5-get....ok
live/jigsaw-md5........ok
live/jigsaw-neg-get....ok
live/jigsaw-neg........ok
live/jigsaw-te.........ok
live/validator.........ok
All tests successful.
Files=42, Tests=833, 70 wallclock secs ( 4.46 cusr +  1.58 csys =  6.04 CPU)

0
 

Author Comment

by:munzer
ID: 19586919
clockwatcher

thanks, would you say that an http api using perl/curl to or perl/lwp can be reliable to send messages to database and retrieve messages from database constantly. I want to schedule th eperl as batch jobs running continuously.

Somone told me that http is goof for retreiving page and not tunneling data and I should check tcp/ip option.
0
 
LVL 39

Expert Comment

by:Adam314
ID: 19592536
HTTP runs over tcp/ip
0
 

Author Comment

by:munzer
ID: 19592827
yes I knew that. But http is an application that runs on top of tcp/ip what is used to retreive web pages.

For exampl, you use smtp for email, ftp for file transfer, etc.  You do not use http for email or file transfer. It might not work.

0
 
LVL 39

Expert Comment

by:Adam314
ID: 19592947
I was commenting in regards to your last comment:
>>Somone told me that http is goof for retreiving page and not tunneling data and I should check tcp/ip option. <<

If you are trying to talk to a webserver, then you will want to use HTTP.  If you are talking to some other server, you will want to use whatever protocol is needed by that server.
0
 

Author Comment

by:munzer
ID: 19597494
well we have two unix remote servers: one has an oracle database and the other has a file system.
the database machine is behind a firewall. The oracle application server (mod_plsql) which allows you to run oracle stored procedures over the web is outside the firewall (3rd machine in front of the database).

The remote filesystem server basically will send a message (Quey or Update) to the database and then does its file management based on that. For exampl,e if a user flagged a certain book to be deleted in the database,a perl batch job will be sent to the database to perform a query, and the db will send  that book number. perl will delete that book from the filesystem and then send an update message to the database telling it the book has been deleted. The database updates a flag and then send perl on the other server confirmation that "Update OK" or "Update failed'.

This is where I am wondering whether "HTTP" will work or not. The problems that need to be handled are timeouts and errors.
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 19598037
either you have to set up other server processes (for example a web server with CGI)  to handle your various requests, or simply use ssh to start the remote perl scripts
0
 

Author Comment

by:munzer
ID: 19600714
well correct me if i am wrong

if the perl machine wants to only make http calls to that oracle web server then I do not need to have a web server/cgi running on that perl machine. BEcause it simply makes a http request using CURL or LWP and receives a response.

If the database needs to run any perl scripts on the perl machine then YES I must have a web server on the perl machine.
0
 
LVL 39

Expert Comment

by:Adam314
ID: 19601245
That sounds correct.
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In real business world data are crucial and sometimes data are shared among different information systems. Hence, an agreeable file transfer protocol need to be established.
The SignAloud Glove is capable of translating American Sign Language signs into text and audio.
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …
Progress
Suggested Courses
Course of the Month13 days, 8 hours left to enroll

750 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question