Solved

Perl - WWW::Mechanize with a proxy server

Posted on 2003-11-18
1
1,158 Views
Last Modified: 2012-06-27
Help!

I am using Perl WWW::Mechanize to access some websites.  The goal is to access certain pages on a corporate network and scrape information.  

I am running into a problem in that I can't get past the proxy server.  I have setup the following

my $agent = WWW::Mechanize->new();
$agent->proxy(http => "http://xxx.proxy.xxx.com:8001/");

but when I run

my $response = $agent -> get('http://www.tmng.com/HomePage/homepage.htm/');

I get a 403 error.

Now from IE, to setup the browser to run on the network, under Tools/Internet Options, undr the connection tab, in the LAN settings, you have to click on the Use Auto Configuration script option and the location is "http://xxx.proxy.xxx.com:8001".

How do I get Perl to recognize the auto config setting on the proxy server?  

Thanks
0
Comment
Question by:mattstarbuck
1 Comment
 
LVL 20

Accepted Solution

by:
jmcg earned 400 total points
ID: 9774717
The URL

http://xxx.proxy.xxx.com:8001/

may refer only to the server holding the autoconfiguration file and not itself be a proxy server. You may need to retrieve that URL and examine its content (it will look a lot like JavaScript) to determine what to set as your proxy values. I'm not aware of a way to have an LWP script do this for itself.
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Sending email via Perl on Windows 3 185
Need help with bash and/or perl commands on OS X Terminal 9 110
perl to convert excel to csv 3 260
PERL variable conundrum 9 94
I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question