Solved

Perl access with Integrated Windows Authentication

Posted on 2004-04-22
19
5,103 Views
Last Modified: 2012-08-13
I have a Perl Script which works when Basic Authentication is used but gives Http Error 401 after the IIS Authentication Method is changed to "Integrated Windows Authentication" on Windows 2000.

The script is:

# non-buffered output to server
 $|=1;

 use LWP::UserAgent;
 use URI::URL;

 local $ua = LWP::UserAgent->new;
 $ua->timeout(20);      # set our BASIS WEBserver timeout to 20 seconds
 $myoutput = "";
 $item = 1;
 do {
    local $req = HTTP::Request->new(GET =>  "http://nztva$dbmv"."DDW?W=$url\&m=$item\&R=Y\&U=1\n\n");
    $req->authorization_basic("b","c");      # uid and password for authorized WWW sources
    $res =$ua->request($req);
    $myoutput = $myoutput . $res->content;
    $item++;
  }  while ($item <= $tmem);      # end do
 $myoutput =~ s/\r/\r~/g;
 print "Content-type: text/html\n";
 print "Content-Length: " . length($myoutput) . "\n\n";
 $myoutput =~ s/\r~/\r/g;
 print $myoutput;

}
exit;

This script concatenantes a series of web pages into 1 page for display. I get a series of 401 authentication errors instead of the data:

HTTP Error 401
401.2 Unauthorized: Logon Failed due to server configuration

How do I need to change the line:

   $req->authorization_basic("b","c");      # uid and password for authorized WWW sources

To allow access when using Integrated Windows Authentication on Windows 2000 and Windows NT Challenge/Response on Windows NT?

I'm sure it's easy to do but I don't know how!

Thanks for your Help.
0
Comment
Question by:beets
  • 9
  • 8
  • 2
19 Comments
 
LVL 3

Expert Comment

by:cjmos
ID: 10895412
beets

Can't you just change

$req->authorization_basic("b","c");

to the user name and password of any user account on the server that has read/write access to the web pages? like yours?
0
 

Author Comment

by:beets
ID: 10895587
I tried this before and it didn't help. Could the fact that the Windows 2000 users login via a Domain cause it not to work in Perl?
0
 

Author Comment

by:beets
ID: 10895633
I used the administrator username and password and they didn't work either.
0
 
LVL 3

Expert Comment

by:cjmos
ID: 10895711
I've just tried to set up a small test based on your example, it doesn't work for me either... without domains. There may be an issue with LWP and Integrated Windows Authentication.

Anyone else get this?
0
 
LVL 20

Expert Comment

by:jmcg
ID: 10905972
You might see whether the module

LWP::Authen::Ntlm

http://search.cpan.org/~gaas/libwww-perl-5.79/lib/LWP/Authen/Ntlm.pm

can be used for your purposes.
0
 

Author Comment

by:beets
ID: 10915002
hello jmcg,

Am trying your suggestion. (Am running Perl v5.6.1)

Am having trouble getting it to work displaying a series of appended html documents as per my posting.

Could you edit and paste the perl in my original posting as a reply getting  an html page more than once? A $ua->credentials line appears to be needed before the GET line and I can't get this working when getting more than 1 document using $req-> credentials instead.

I'm testing on a system with no Domain, I presume this still works by leaving off "MyDomain\\" from the User Name.

Many Thanks.
0
 
LVL 20

Expert Comment

by:jmcg
ID: 10915349
Sorry, I have no experience with the module. I can't set up a test environment to verify that what I'm telling you is likely to be useful....

============

The $ua->credentials line should be inserted after you create the $ua object, but outside the loop. It needs to place the host address in a particular form, since that's how the rest of the module later does lookup. Here's my best guess at how the code should look:

   .
   .
   .
 local $ua = LWP::UserAgent->new;
 $ua->timeout(20);     # set our BASIS WEBserver timeout to 20 seconds

 # set up credential for implicit use by Authen::NTLM
 $ua->credentials( "nztva$dbmw:80", "", "MyDomain\\MyUsername", "MyPassword");

 $myoutput = "";
 $item = 1;
 do {
    local $req = HTTP::Request->new(GET =>  "http://nztva$dbmv"."DDW?W=$url\&m=$item\&R=Y\&U=1\n\n");
   #  $req->authorization_basic("b","c");     # uid and password for authorized WWW sources
    $res =$ua->request($req);
    $myoutput = $myoutput . $res->content;
   .
   .
   .
============

I'm afraid I have no idea of how you have a Windows authentication protocol working without some sort of Workgroup or Domain name as part of the identification.

You must also have downloaded and installed the correct Athen::NTLM module where Perl can find it, as described in the LWP::Authen::Ntlm doc I pointed you to earlier.
0
 

Author Comment

by:beets
ID: 10924170
I am using a Workgroup (called WORKGROUP) instead of a Domain. I tried "WORKGROUP\\Administrator" as the username - which didn't work. Is this the correct format for the credentials line username with a workgroup?

I downloaded the Ntlm.pm file via the Source hyperlink at the top of the page. I could not find where the documentation said to place the file so I placed it in: C:\Perl\site\lib\LWP and in c:\Perl\site\lib\LWP\Authen directories. (is this correct? The Readme file didn't give the directory).

I inserted the $ua->credentials line after creating the $ua object outside the loop as per your last suggestion and commented out the existing authorization_basic line.

If I try to get 1 document, I get Http 401.2 error.

If I try to retrieve 2 or more (i.e. the do loop executes more than once) I get:

CGI Error
The specified CGI application misbehaved by not returning a complete set of HTTP headers. The headers it did return are:


Can't locate object method "authenticate" via package "LWP::Authen::Ntlm" (perhaps you forgot to load "LWP::Authen::Ntlm"?) at C:/Perl/site/lib/LWP/UserAgent.pm line 341.

I'm annoyed that I can't see where to place the Ntlm file. The sample code on the cpan page returns "It didn't work! 401" Access Denied.

If I can get the directory name correct and the Username format I will make some progress. Where is ntlm.pm on your PC?
0
 
LVL 20

Expert Comment

by:jmcg
ID: 10924453
Oh, this error message is very encouraging. It indicates that the LWP package saw an Authenticate header using NTLM and tried to reach for the LWP::Authen::Ntlm module without being told explicitly by you that that's what you wanted to use. I think it means we're on the right track.

If you did not already have a module at

C:\Perl\site\lib\LWP\Authen\Ntlm.pm

it may be necessary to update your entire LWP install. That module should have been there without any need to download it specifically. And I don't understand why the message says it can't find it if you put it there (unless you miscapitalized the name or something like that).

You don't mention downloading and installing the Authen::NTLM module:

http://search.cpan.org/~markbush/NTLM-1.02/

These would presumably go in

C:\Perl\site\lib\Authen\NTLM.pm
C:\Perl\site\lib\Authen\NTLM\DES.pm
C:\Perl\site\lib\Authen\NTLM\MD4.pm

and you need to check on the MIME::Base64 exporting of encode_base64( ) and decode_base64( ) as mentioned in the doc.

=======

I'll repeat: I don't have a system where I can try any of this out. I suspect WORKGROUP\Username works, but I don't have enough experience with managed Windows environments to do more than guess.
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 20

Expert Comment

by:jmcg
ID: 10924478
It looks like you'll need to add this line at about the same place where you have the $ua->credentials call, if you haven't already.

my $ua = new LWP::UserAgent(keep_alive=>1);


And be sure read the LWP::Authen::Ntlm doc for more wrinkles.
0
 

Author Comment

by:beets
ID: 10924600
I already had the UserAgent(keep_alive=>1) line. I hadn't downloaded the Authen::NTLM module. (I have not downloaded anything other than "Ntlm.pm" Script hyperlink and added it to Perl. The LWP package I have is what comes with perl 5.6.1 build 626.

I downloaded the NTLM module and unzipped and compiled it to get the following:


C:\Perl\NTLM-1.02>perl Makefile.PL
Checking if your kit is complete...
Looks good
Writing Makefile for Authen::NTLM::DES
Writing Makefile for Authen::NTLM::MD4
Writing Makefile for Authen::NTLM

C:\Perl\NTLM-1.02>make
MAKE Version 5.2  Copyright (c) 1987, 1998 Inprise Corp.
Fatal makefile 674: No terminator specified for in-line file operator

C:\Perl\NTLM-1.02>

I do not understand the "No terminator ..." error above.

Do I need to upgrade Perl completely for NTLM Authentication to work perhaps?
0
 

Author Comment

by:beets
ID: 10925320
I have always downloaded perl from www.activestate.com with no requirement to compile or make files. I wonder if their latest version will have the NTLM module and others loaded into the correct directories. The CPAN site download files need compiling or is NTLM an "add on" module that doesn't come with the latest version of Perl?
0
 
LVL 20

Expert Comment

by:jmcg
ID: 10927253
The "No terminator" error is most likely caused by using an incompatible version of the 'make' utility.

Yes, packages from ActiveState need no compilation on your machine. The Authen::NTLM module appears to be available for PPM. No Perl distribution includes all of the CPAN modules. Adding on external modules, whether as PPM packages for ActivePerl or from CPAN for UNIX and Linux systems, is usually quite painless.
0
 

Author Comment

by:beets
ID: 10934458
Okay, some progress. I updated Perl 5.8.3 build 809 from Activestate. It already has a copy of Ntlm.pm in C:\Perl\site\lib\LWP\Authen.

I presume I still need to add the Authen::NTLM module as per your instructions for the Windows Authentication to work. This time perl Makefile.PL worked however the next command: make started okay but stopped with the error:

mkdir Authen::NTLM::.: Invalid argument at C:/Perl/lib/ExtUtils/Install.pm line 455

this happens after a screenful of lines of cp messages.

Install.pm line 455 has: mkpath(dirname($to),0,0755);

Is adding the Authen::NTLM module is still necessary with this new version of perl? If so, what is the Invalid Argument about? (I currently get concatenated pages of 401 Authentication failed messages if I try to display 1 or more pages or do loop iterations).
0
 
LVL 20

Expert Comment

by:jmcg
ID: 10934653
Can you check the authorship of the Authen::NTLM package you downloaded?

I would have thought the thing to do was to use PPM to get the package. That should not have required any Makefile.
0
 

Author Comment

by:beets
ID: 10935064
PPM worked and NTLM is finally installed properly! (Will use PPM from now on).

Now I only have the message: HTTP 401.2 - Unauthorized: Logon failed due to server configuration Internet Information Services

This may be due to using a Workgroup (called WORKGROUP) rather than a Domain. I need to get the syntax of the Username correct for a Workgroup connection - perhaps someone else knows this. I have looked on the Internet and all examples use Domains.

The CPAN sample code modified below returns "It didn't work!-> 401

 use LWP::UserAgent;
 use HTTP::Request::Common;
 use LWP::Debug qw(+);

 my $url = "http://acer600/bold.html";

$uid = "WORKGROUP\\Administrator";
$upw = "mypassword";

 # Set up the ntlm client and then the base64 encoded ntlm handshake message
 my $ua = new LWP::UserAgent(keep_alive=>1);
 $ua->credentials('http://acer600:80', '', $uid, $upw);



 $request = GET $url;
 print "--Performing request now...-----------\n";
 $response = $ua->request($request);
 print "--Done with request-------------------\n";



 if ($response->is_success) {print "It worked!->" . $response->code . "\n"}
 else {print "It didn't work!->" . $response->code . "\n"}

This produces the following:

CGI Error
The specified CGI application misbehaved by not returning a complete set of HTTP headers. The headers it did return are:


LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://acer600/bold.html
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 811 bytes
LWP::Protocol::collect: read 3620 bytes
LWP::UserAgent::request: Simple response: Unauthorized
LWP::Authen::Ntlm::authenticate: authenticate() has been called
LWP::Authen::Ntlm::authenticate: No username and password available from get_basic_credentials().  Returning unmodified response object
--Performing request now...-----------
--Done with request-------------------
It didn't work!->401

The URL http://acer600/bold.html works okay if entered direct in IE.

My modified perl script also produces a HTTP 401.2 error message for each concatenated page.

Is this supported using Workgroups? Have tried WORKGROUP\Administrator with 1 slash and others with mixed case - all don't work.
0
 
LVL 20

Expert Comment

by:jmcg
ID: 10935227
It seems odd that there should be a call to "get_basic_credentials" in there. It seems to be falling back to that after failing to find the credentials you supplied. Ah, you have "http://" leading the host name part of the lookup string. Get rid of it. It should look more like:

 $ua->credentials('acer600:80', '', $uid, $upw);

Do you have any ability to snoop on the connection made with IE? That might let you figure out what format will work for the credentials.
0
 

Author Comment

by:beets
ID: 10935475
Thank you, removing the http:// string worked (well spotted!) - at least in my perl script that I originally sent with my question.

The CPAM sample code failed with:

CGI Error
The specified CGI application misbehaved by not returning a complete set of HTTP headers. The headers it did return are:


LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://acer600/bold.html
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 811 bytes
LWP::Protocol::collect: read 3620 bytes
LWP::UserAgent::request: Simple response: Unauthorized
LWP::Authen::Ntlm::authenticate: authenticate() has been called
LWP::Authen::Ntlm::authenticate: In first phase of NTLM authentication
LWP::Authen::Ntlm::authenticate: Returning response object with auth header:
Authorization NTLM TlRMTVNTUAABAAAAB7IAAA0ADQAgAAAACQAJAC0AAABBZG1pbmlzdHJhdG9yV29ya2dyb3Vw
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://acer600/bold.html
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::


But I don't care about that code!

Okay I'd like to award you the points but with the answer spread over all your comments perhaps you could send a final comment with a summary of what was required like below and I'll award them. This may be easier for others to follow. Summarized from above comments:

- Windows Authentication requires installation of the NTLM package.
- Use the perl utility PPM to search for NTLM and then install it (or download it if it is not present):

- A $ua->credentials line should be inserted after you create the $ua object, but before the loop (The Domain name may also be a Workgroup name):

 my $ua = new LWP::UserAgent(keep_alive=>1);
 $ua->credentials('acer600:80','','Domain\\username', 'password');

- Comment out the authorization_basic line

-This will allow Windows Authentication on each concatenated page using NTLM
-------------
Thank you, I appreciate all your help and patience.
0
 
LVL 20

Accepted Solution

by:
jmcg earned 100 total points
ID: 10945584
Wait, are you saying your code now works but the example still doesn't? I don't know why that would be.

It's fairly common for an answer to be spread out over multiple responses. I think you've done a fine job of summarizing.

1. LWP::Authen::Ntlm is now a normally included component of the LWP package

2. The Mark Bush Authen::NTLM module will have to be located and installed. There's a PPM package for it on the default ActiveState repositories.

3. It sounds like you did not have a problem with the base64_encode/decode routines.

4. LWP will automatically use the NTLM authentication scheme if it sees it. It will call on the Authen::NTLM module as long as it can find it via the normal method of looking for it in the @INC list.

5. Turn on keep-alive. You can do this when you create your user agent or set it later (but before you do a request using the UserAgent object).

my $ua = new LWP::UserAgent(keep_alive=>1);

6. Credentials for NTLM are supplied via $ua->credentials call. The first parameter is a lookup key that must have the hostname followed by a colon and port number. There should be no http:// on the front. No Realm value is needed, so a null string can be used. Username can be expressed in the form 'Domain\\Username' or 'Workgroup\\Username'. The password is given as the 4th parameter.

$ua->credentials('acer600:80','','Domain\\username', 'password');


7. A call to basic_credentials is not needed. We've seen that LWP will drop back to attempting Basic authentication if it cannot find credentials appropriate to the host:port pair.

0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now