Solved

SOCKET,BIND,CONNECT-->PROXY-->REMOTE SITE?

Posted on 1998-11-17
5
618 Views
Last Modified: 2012-08-14
I made a program which allows me to connect to a remote website via socket/bind/connect.  It establishes a socket connection and downloads the requested webpage.

Now I would love to actually go thru a proxy server first and have it download the webpage and return it back to me. So normally I have this happening:
SOCKET/BIND/CONNECT-->REMOTE SITE
And now I want this:
SOCKET/BIND/CONNECT-->PROXY SERVER-->REMOTE SITE

This way the remote site sees the proxy server rather than me directly.  How do I go about making a proxy server connection and then to the remote site from my Perl script.  I have the ground work, as I said but I just can't figure out how the proxy server address fits into it.

Maybe I establish a connection not with the remote site but with the proxy server at port 8080.  Then where I normally request for a certain webpage:
GET /index.html HTTP/1.0
Referer: http://me.com

I should place:
GET http://www.mysite.com HTTP/1.0
Referer: http://me.com

Now the proxy server will understand this GET statement and connect to the remote site.  Is this correct thinking?

And another question.  Does the Referer field always get passed by all Proxy servers or only some? It is an important header field that needs to get passed.  And also, the remote site that is receiving the call, does it get the usual ENVironment variables like a normal call or does the Proxy Server insert additional fields (passing info like who I am and it is a proxy server that is calling).  If it is passing additional fields can I somehow turn them off?

Thanks so much.

J.



0
Comment
Question by:ljaques
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
5 Comments
 

Author Comment

by:ljaques
ID: 1206291
Edited text of question
0
 
LVL 5

Expert Comment

by:b2pi
ID: 1206292
Rather than reinventing the wheel, you might want to take a look at the libwww bundle (available on CPAN, current version is 5.36, I think);

That installs a program called GET that does exactly what you want it to do (including parsing the proxy server from the environment, if possible.)
0
 
LVL 2

Accepted Solution

by:
ventolin earned 100 total points
ID: 1206293
#/usr/bin/perl

# This should work for you...
# -ventolin

# QuickAndDirty - web client program for experts exchange

use LWP::UserAgent;
my $email_address = "youknowwhattoputhere";

my $AGENT = LWP::UserAgent->new;
$AGENT->agent("experts-exchange");
$AGENT->from([$email_address]);
$AGENT->proxy(['http', 'ftp'], 'http://proxy.yourserver.com:8080/');

# be sure to set this if you want to use a proxy
# you could also use $AGENT->env_proxy; if a proxy variable has been set

my $REQUEST = HTTP::Request->new('GET', "http://experts-exchange.com/index.html");
my $response = $AGENT->request($REQUEST);
if ($response->is_success)     {                                              
        print $response->content;
       }
else {
        die "$response->code, $response->message";
        }

0
 

Author Comment

by:ljaques
ID: 1206294
Ventolin, thanks so much for the actual code.  Unfortunately I don't have LWP for my Perl in DOS/WIN95.  And everytime I have to go find a new package to include for my Perl there always seesm to be additional pakcages I need to dload to get the first package to work.  ThenI have to worry about setting up this and that file.  I just never can set up these packages for Perl for DOS/WIN95.  Ho Hum...

Though, I already have a script which allows me to connect to other websites.  It's much like the program Linklint.  Unfortunately when I try to connect to a remote site thru a certain proxy server, no response comes back.  Connecting to other proxy servers is fine but not with this one (proxy server: http://128.125.52.62:3128).  I can't figure out whats going wrong.  The script routines to connect to remote sites is flawless and even connecting to certain proxy servers and passing the appropriate request header (GET http://remotesite.com HTTP/1.0) is fine. Just with this one proxy server it seems to just pause and not return anything.  Thru Netscape, setting it up to call the same proxy server and manually connecting to the remote site works just fine, just not in my perl script.  Weird.

Any solutions?  And about the LWP package you mentioned.  I'd love to get it installed for my Perl it's just I wish there's an easy way to do it. Any solutions?  :>

Thanks.
0
 
LVL 2

Expert Comment

by:ventolin
ID: 1206295
Check out:

<a href=http://www.perl.com/CPAN/modules/by-module/LWP>libwww Bundle</a>
<P>
<a href=ftp://cogen.mit.edu/software/perl/libwww-win32-fix.zip>libwww-win32-fix</a>
<P>
This file contains the files necessary for running LWP/libwww under Activeware's port of Perl for Win32 5.003 or higher.

0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
Many time we need to work with multiple files all together. If its windows system then we can use some GUI based editor to accomplish our task. But what if you are on putty or have only CLI(Command Line Interface) as an option to  edit your files. I…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

738 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question