Solved

How do POST to another Webpage with CGI

Posted on 2004-04-22
16
608 Views
Last Modified: 2013-12-25
Hello, I am trying to create a PayPal situation where I do not use the standard form but instead have all of the info within a CGI script to be POSTed to the PayPal secure server when the script is accessed.

I am working with what I have learned from www.perldoc.com about the subject but keep getting a 500 Internal Server error.

This is what I have:
---------------------------------------------------------------------------------

#!/usr/bin/perl -w

use LWP::UserAgent;
$ua = LWP::UserAgent->new;

my $req = HTTP::Request->new(POST => 'https://www.paypal.com/cgi-bin/webscr');
$req->content_type('application/x-www-form-urlencoded');
$req->content('cmd=_xclick&business=ziffgone@telus.net&item_name=MASSIVE Webmaster's Resale Package&item_number=MWRP001&amount=69.95&return=http://webmastereseller.com/********&cancel_return=http://webmastereseller.com&no_note=1¤cy_code=USD');

my $res = $ua->request($req);
print $res->as_string;
---------------------------------------------------------------------------------

I have tried escaping all special characters only to get the same result. Any idea why this isn't sending the content to PayPal? Should I be using a different Perl header than "-w"?

Thanks...
0
Comment
Question by:ziffgone
  • 8
  • 7
16 Comments
 
LVL 10

Expert Comment

by:Mercantilum
ID: 10899530
What did you get in your /var/log/httpd/error_log (or equivalent) when the error occured?
0
 
LVL 14

Author Comment

by:ziffgone
ID: 10899686
Hi Mercantilum,

This is what I got:
Premature end of script headers: /home/pwolf/public_html/cgi-bin/test_post.cgi

Thanks...
0
 
LVL 10

Expert Comment

by:Mercantilum
ID: 10899720
This is typically when the famous line
    print "Content-type: text/html\n\n"; # or image or ...
is missing in the CGI script ...

Have a look to http://perlmonks.thepen.com/24447.html too.
0
 
LVL 14

Author Comment

by:ziffgone
ID: 10904837
I had the script print Fatals out to browser now and ends up I had a sintax error.
Fixed this but I also received the following error and don't know how to fix:

This is what I got:

Global symbol "$ua" requires explicit package name at test_post.cgi line 9

Line 9 = "$ua = LWP::UserAgent->new;"

Global symbol "$ua" requires explicit package name at test_post.cgi line 15

Line 15 = "my $res = $ua->request($req);"

What's the explicit package I need to be assigning?

CURRENT SCRIPT:
-----------------------------------------------------------------------------------
#!/usr/bin/perl -w

use strict;
use CGI;

use LWP::UserAgent;
$ua = LWP::UserAgent->new;

my $req = HTTP::Request->new(POST => 'https://www.paypal.com/cgi-bin/webscr');
$req->content_type('application/x-www-form-urlencoded');
$req->content('cmd=_xclick&business=ziffgone\@telus\.net&item_name=MASSIVE Webmaster\&\#39\;s Resale Package&item_number=MWRP001ref=booj\@boogiejack\.com&amount=69\.95&return=http://webmastereseller\.com/*********&cancel_return=http://webmastereseller\.com&no_note=1¤cy_code=USD');

my $res = $ua->request($req);
 print "Content-type: text/html\n\n";
print $res->as_string;
-----------------------------------------------------------------------------------
0
 
LVL 10

Expert Comment

by:Mercantilum
ID: 10905195
If you "use strict" you have to declare locally $ua:

my $ua = LWP::UserAgent->new;
0
 
LVL 14

Author Comment

by:ziffgone
ID: 10906181
Mercantilum, SO CLOSE!!!

Your suggestion of declaring it locally worked, but now I face another problem.

The script generates the PayPal page on my server instead of PayPal's. To see
what I mean, check it out:

http://webmastereseller.com/cgi-bin/test_post.cgi

Is there a way I could "Target" the POSTed data to PayPal's
"https://www.paypal.com/cgi-bin/webscr" page instead of having it generated on my server?

Soooo close, I'm going to have to increase the points for this one!
0
 
LVL 10

Expert Comment

by:Mercantilum
ID: 10907130
Yes, you can automatically simulate a POST of data to paypal.

- I did it with curl, "man curl"  or install it from http://curl.haxx.se/  (curl can be called from Perl)

- In Perl your LWP::UserAgent should be able to do it (just in case: http://www.perldoc.com/perl5.6/lib/LWP/UserAgent.html )

It is actually quite easy when you know the format of the data to be sent (e.g.  user=xxx&password=taratata&... )

If you don't know the format of the post data, and don't know how to get it.
you can simulate a POST using paypal and record all network operations with ethereal http://www.ethereal.com/
This is a fantastic network monitoring tool, it will show you what exactly was sent to paypal when you click the Post button
(You don't need to put actual data for paypal, just to get the format of the post :)

Another thing is about a cookie that could have been set at the beginning of the paypal session, ethereal can tell you that as well
(have to look inside the http frames)
0
DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

 
LVL 14

Author Comment

by:ziffgone
ID: 10907459
Quote Mercantilum: "I did it with curl, "man curl"":

Mercantilum, could you send me a script, based upon the variables in
my script above that will work using the "man curl" funtion? The curl function
works on my server, but I keep getting syntax errors.

<email address deleted by COBOLdinosaur, Page Editor, CGI>

I will think up a just reward if you are willing to do this for me.
I'm fairly new to Perl and web programming in general and what
you indicated above for reference at perldoc.com and ethereal.com
seems to be way above my comprehension to implement.

Thanks...
0
 
LVL 10

Accepted Solution

by:
Mercantilum earned 500 total points
ID: 10910155
"man curl" was for manual curl. On Linux / Unix enter
   man curl
to get the curl help. (or get curl from  http://curl.haxx.se/ )

Extract from the man page:

<<  -d/--data <data>
        (HTTP) Sends the specified data in a POST request to the HTTP server, in a way that  can  emu-
        late  as if a user has filled in a HTML form and pressed the submit button. Note that the data
        is sent exactly as specified with no extra processing (with all newlines cut off).   The  data
        is expected to be "url-encoded". This will cause curl to pass the data to the server using the
        content-type application/x-www-form-urlencoded. Compare to -F.  If  more  than  one  -d/--data
        option  is  used  on  the same command line, the data pieces specified will be merged together
        with a separating &-letter. Thus, using '-d name=daniel -d skill=lousy' would generate a  post
        chunk that looks like 'name=daniel&skill=lousy'.

        If you start the data with the letter @, the rest should be a file name to read the data from,
        or - if you want curl to read the data from stdin.  The contents of the file must  already  be
        url-encoded.  Multiple  files  can  also be specified. Posting data from a file named 'foobar'
        would thus be done with "--data @foobar". >>

So your Perl script can be something like
       ...
      my %data;
      $data{'card'} = "481234567801321";
      $data{'expire'} = "0605";
      $data{'name'} = "Ziff Gone";
      $data{'price'} = "423.28";

Construct the data string

      my $line;
      my @items;
      foreach (keys %data) { push(@items, "$_=$data{$_}"); }
      $line =  join("&", @items);

Output line to a file (in cases of spaces, etc...)

      open F, ">data";
      print F $line;
      close F;

Execute curl and get result in @res

      my @res = `curl -d \@data http://www.paypal.com/something/...`;
      print @res;
0
 
LVL 14

Author Comment

by:ziffgone
ID: 10941203
I can't seem to put this all together, just not learned enough in Perl I guess.

Mercantilum, you get the points for your help.

Thanks.
0
 
LVL 10

Expert Comment

by:Mercantilum
ID: 10941309
What are you getting problem with?
If the perl lines above are the problem it can be fixed.
0
 
LVL 14

Author Comment

by:ziffgone
ID: 10941897
With the following script I'm getting the premature end to script headers again:

#!/usr/bin/perl -w

use strict;
use CGI;


use CGI::Carp qw(fatalsToBrowser);

use LWP::UserAgent;
my $ua = LWP::UserAgent->new;

my %data;
      $data{'cmd'} = "_xclick";
      $data{'business'} = "ziffgone\@telus\.net";
      $data{'item_name'} = "MASSIVE Webmaster\&\#39\;s Resale Package";
      $data{'item_number'} = "MWRP001ref=booj\@boogiejack\.com";
      $data{'amount'} = "69\.95";
      $data{'return'} = "http://webmastereseller\.com/**********\.htm";
      $data{'cancel_return'} = "http://webmastereseller\.com";
      $data{'no_note'} = "1";
      $data{'currency_code'} = "USD";

my $line;
      my @items;
      foreach (keys %data) { push(@items, "$_=$data{$_}"); }
      $line =  join("&", @items);

open F, ">data";
      print F $line;
      close F;

my @res = 'curl -d \@data https://www.paypal.com/cgi-bin/webscr';
      print @res;

If I add the >> print "Content-type: text/html\n\n"; << notation before printing, all it does is output:
"curl -d \@data https://www.paypal.com/cgi-bin/webscr" to the screen.

??
0
 
LVL 10

Expert Comment

by:Mercantilum
ID: 10941996
Oh I see,
1. you need the  print "Content-type: text/html\n\n"
2. and
  my @res = 'curl -d \@data https://www.paypal.com/cgi-bin/webscr';
is actually
  my @res = `curl -d \@data https://www.paypal.com/cgi-bin/webscr`;
using bacquotes `  and not common ' quotes...
0
 
LVL 14

Author Comment

by:ziffgone
ID: 10942069
No there is no errors, but it comes up blank? Sorry for troubling you like this.
0
 
LVL 10

Expert Comment

by:Mercantilum
ID: 10942206
No trouble mate :)
Here to help, and if I'm not in a mood to help, I just close the browser...

Of course your Content-type line is before the print @res...


print "Content-type: text/html\n\n";

# change a bit the options of curl
my @res = 'curl -s -d \@data https://www.paypal.com/cgi-bin/webscr';
print @res;


And try to run the perl script from command line in a shell window, to see what is your output before the web server processed it.
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Preface This article introduces an authentication and authorization system for a website.  It is understood by the author and the project contributors that there is no such thing as a "one size fits all" system.  That being said, there is a certa…
Batch, VBS, and scripts in general are incredibly useful for repetitive tasks.  Some tasks can take a while to complete and it can be annoying to check back only to discover that your script finished 5 minutes ago.  Some scripts may complete nearly …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn the basics of jQuery including how to code hide show and toggles. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now