Solved

Perl - WWW::Mechanize with a proxy server

Posted on 2003-11-18
1
1,178 Views
Last Modified: 2012-06-27
Help!

I am using Perl WWW::Mechanize to access some websites.  The goal is to access certain pages on a corporate network and scrape information.  

I am running into a problem in that I can't get past the proxy server.  I have setup the following

my $agent = WWW::Mechanize->new();
$agent->proxy(http => "http://xxx.proxy.xxx.com:8001/");

but when I run

my $response = $agent -> get('http://www.tmng.com/HomePage/homepage.htm/');

I get a 403 error.

Now from IE, to setup the browser to run on the network, under Tools/Internet Options, undr the connection tab, in the LAN settings, you have to click on the Use Auto Configuration script option and the location is "http://xxx.proxy.xxx.com:8001".

How do I get Perl to recognize the auto config setting on the proxy server?  

Thanks
0
Comment
Question by:mattstarbuck
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
1 Comment
 
LVL 20

Accepted Solution

by:
jmcg earned 400 total points
ID: 9774717
The URL

http://xxx.proxy.xxx.com:8001/

may refer only to the server holding the autoconfiguration file and not itself be a proxy server. You may need to retrieve that URL and examine its content (it will look a lot like JavaScript) to determine what to set as your proxy values. I'm not aware of a way to have an LWP script do this for itself.
0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I've just discovered very important differences between Windows an Unix formats in Perl,at least 5.xx.. MOST IMPORTANT: Use Unix file format while saving Your script. otherwise it will have ^M s or smth likely weird in the EOL, Then DO NOT use m…
On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans

696 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question