• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 3079
  • Last Modified:

How to Open HTTP page with digest Authentication using HttpClient and create BufferedReader

I have some generic code for reading file or http page which returns a BufferedReader type to calling function.
For HTTP, I have the following which works for basic authentication.
 
URL url = new URL(inFilePath);

				if (taskdataHandler.SiteCredentialsUserName != null && !taskdataHandler.SiteCredentialsUserName.isEmpty()
						&& taskdataHandler.SiteCredentialsPassword != null && !taskdataHandler.SiteCredentialsPassword.isEmpty())
				{
					HttpURLConnection uc = (HttpURLConnection) url.openConnection();
					String userpassword = taskdataHandler.SiteCredentialsUserName + ":" + taskdataHandler.SiteCredentialsPassword;
					String encodedAuthorization = taskManager.Base64Encode(userpassword);
					uc.setRequestProperty("Authorization", "Basic " + encodedAuthorization);
					in = new BufferedReader(new InputStreamReader(uc.getInputStream()));
				} else
					in = new BufferedReader(new InputStreamReader(url.openStream()));

Open in new window


I'm trying to add logic for digest authentication using HttpClient, and with input parameter of url path, and output type of BufferedReader for the path.

I tried using the following code which is a modified version of code I found on the web, but this gives me an error saying too many redirects.
How can I fix redirect error, and how can I get HttpClient to work with url path as input parameter and BufferedReader as out put parameter?
static public BufferedReader OpenFileForRead_digest(String inFilePath, taskDataHandler taskdataHandler) throws Exception
	{
		BufferedReader in = null;
		String strURL = inFilePath;
		final String username = taskdataHandler.SiteCredentialsUserName;
		final String password = taskdataHandler.SiteCredentialsPassword;

		Authenticator.setDefault(new Authenticator()
		{
			protected PasswordAuthentication getPasswordAuthentication()
			{
				PasswordAuthentication pa = new PasswordAuthentication(username, password.toCharArray());
				return pa;
			}
		});

		try
		{
			URL url = new URL(strURL);
			URLConnection connection = url.openConnection();
			in = new BufferedReader(new InputStreamReader(connection.getInputStream()));

		} catch (java.net.ProtocolException e)
		{
			StringWriter stringWriter = new StringWriter();
			e.printStackTrace(new PrintWriter(stringWriter));
			// get the stackTrace as String...
			String stackTrace = stringWriter.toString();
			String ExceptionError = new String("OpenFileForRead_digest " + " <b>Error</b>ProtocolException: " + e.toString() + "<p>..</p>\n<pre>" + stackTrace
					+ "</pre>");
			cvlog.error(ExceptionError);
		} catch (Exception e)
		{
			StringWriter stringWriter = new StringWriter();
			e.printStackTrace(new PrintWriter(stringWriter));
			// get the stackTrace as String...
			String stackTrace = stringWriter.toString();
			String ExceptionError = new String("OpenFileForRead_digest " + " <b>Error</b>Exception: " + e.toString() + "<p>..</p>\n<pre>" + stackTrace
					+ "</pre>");
			cvlog.error(ExceptionError);
		}

		return in;
	}

Open in new window

0
Axter
Asked:
Axter
  • 21
  • 17
1 Solution
 
for_yanCommented:
maybe I'm missing something, but I don't see if you are
using HttpClient from apache:
http://hc.apache.org/downloads.cgi

Usisng HttpClient I don't think you need to use UrlConnection class directly.

With HttpClient perhpas you can try this code for digest authentication taken from
http://datum-bits.blogspot.com/2011/07/using-apache-commons-httpclient-for.html
which looks pretty simple:

  public static String sendDigestAuthentication(
      String username, Sting password,
      String url, String host,
      int portNumber, String protocol) {

    HttpResponse response = null;
    HttpHost targetHost = new HttpHost(
          host, portNumber, protocol);

    DefaultHttpClient httpclient =
      new DefaultHttpClient();
    try {
      httpclient.getCredentialsProvider().
        setCredentials(
          new AuthScope(targetHost.getHostName(),
                targetHost.getPort()),
          new UsernamePasswordCredentials(username,
                        password));
    
      HttpGet httpget = new HttpGet(url);
      response = httpclient.execute(targetHost,
                    httpget);
    } catch (Exception exc) {
       logger.debug("Exception digesting Authentication");
    } finally {
      // When HttpClient instance is no longer needed,
      // shut down the connection manager to ensure
      // immediate deallocation of all system resources
      httpclient.getConnectionManager().shutdown();
    }
    return response.toString();
  }

Open in new window

0
 
AxterAuthor Commented:
I tried similar code, but it didn't work for me.
But I'm not sure if I'm using it the right way, because I don't know how exactly it ties into my existing code.
Here's my attempt to use the above code.
 
sendDigestAuthentication(taskdataHandler.SiteCredentialsUserName, 
							taskdataHandler.SiteCredentialsPassword,
							inFilePath, "MySiteHere", 80, "http");
					url = new URL(inFilePath);
					HttpURLConnection uc = (HttpURLConnection) url.openConnection();

			uc.setRequestProperty("Authorization", "Basic " + encodedAuthorization);
					in = new BufferedReader(new InputStreamReader(uc.getInputStream()));

Open in new window


The above fails with a redirect error.
IOException: java.net.ProtocolException: Server redirected too many  times (20)
0
 
AxterAuthor Commented:
Correction on the code:
 
sendDigestAuthentication(taskdataHandler.SiteCredentialsUserName, 
							taskdataHandler.SiteCredentialsPassword,
							inFilePath, "myweb", 80, "http");
url = new URL(inFilePath);
HttpURLConnection uc = (HttpURLConnection) url.openConnection();
in = new BufferedReader(new InputStreamReader(uc.getInputStream()));

Open in new window

0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
AxterAuthor Commented:
This is the string I'm getting back from sendDigestAuthentication:
HTTP/1.1 401 Unauthorized [Content-Length: 1656, Content-Type: text/html, Server: Microsoft-IIS/6.0, WWW-Authenticate: Negotiate, WWW-Authenticate: NTLM, WWW-Authenticate: Digest qop="auth",algorithm=MD5-sess,nonce="+Upgraded+......../.",charset=utf-8,realm="Digest", WWW-Authenticate: Basic realm="mywebsite", X-Powered-By: ASP.NET, Date: Tue, 01 Nov 2011 10:56:09 GMT]
0
 
for_yanCommented:
But you are not using HttpClient.
Have you downloaded the zip file I mentioned above and do you have the jar in your classpath ?
0
 
AxterAuthor Commented:
I already have the jar file.
I've tried using the code in the following links, but I couldn't figure out how to get them to work.

http://www.google.com/url?sa=t&rct=j&q=clientpreemptivedigestauthentication&source=web&cd=2&ved=0CCsQFjAB&url=http%3A%2F%2Fhc.apache.org%2Fhttpcomponents-client-ga%2Fhttpclient%2Fexamples%2Forg%2Fapache%2Fhttp%2Fexamples%2Fclient%2FClientPreemptiveDigestAuthentication.java&ei=C_OvTpTtNoWHgwf6xqStAQ&usg=AFQjCNFfeMtlXzec_Ixm8q6vDHlmIH5Wkw&cad=rja

http://www.google.com/url?sa=t&rct=j&q=clientpreemptivebasicauthentication&source=web&cd=1&sqi=2&ved=0CBoQFjAA&url=http%3A%2F%2Fhc.apache.org%2Fhttpcomponents-client-ga%2Fhttpclient%2Fexamples%2Forg%2Fapache%2Fhttp%2Fexamples%2Fclient%2FClientPreemptiveBasicAuthentication.java&ei=yvKvTvuPKYLrgQei3NjMAQ&usg=AFQjCNHOnG5SRlSoY1nZALCH31dWlmPAzg&cad=rja


Here's the code I had with my first attempt:
 
static public BufferedReader OpenFileForRead_digest2(String UrlPath, taskDataHandler taskdataHandler) throws Exception
	{
		cvLog cvlog = new cvLog("taskManagerx", "BufferedReader");
		BufferedReader in = null;
		DefaultHttpClient httpclient = new DefaultHttpClient();

		// Create local execution context
		HttpContext localContext = new BasicHttpContext();

		HttpGet httpget = new HttpGet(UrlPath);
		cvlog.debug("executing request " + httpget.getRequestLine());
		HttpResponse response = httpclient.execute(httpget, localContext);

		cvlog.debug(response.getStatusLine().toString());

		// Consume response content
		HttpEntity entity = response.getEntity();
		if (entity != null)
		{
			entity.consumeContent();
		}

		int sc = response.getStatusLine().getStatusCode();

		AuthState authState = null;
		if (sc == HttpStatus.SC_UNAUTHORIZED)
		{
			// Target host authentication required
			authState = (AuthState) localContext.getAttribute(ClientContext.TARGET_AUTH_STATE);
		}
		if (sc == HttpStatus.SC_PROXY_AUTHENTICATION_REQUIRED)
		{
			// Proxy authentication required
			authState = (AuthState) localContext.getAttribute(ClientContext.PROXY_AUTH_STATE);
		}

		if (authState != null)
		{
			AuthScope authScope = authState.getAuthScope();
			Credentials creds = new UsernamePasswordCredentials(taskdataHandler.SiteCredentialsUserName, taskdataHandler.SiteCredentialsPassword);
			httpclient.getCredentialsProvider().setCredentials(authScope, creds);
			in = new BufferedReader(new InputStreamReader(entity.getContent()));
		}

		return in;
	}

Open in new window


When I tried using above code I always get the following error:
Attempted read from closed stream
0
 
for_yanCommented:
AT which line do you have this closed stream error coming?
Do you see it as an exception?

I would rather start with the snippet I posted above  in http:#37061067 and first check if I get the response in that simple code - just printing the response.
If it works - then go from there to analyze the returned response with getEntity, etc.


0
 
AxterAuthor Commented:
>>I would rather start with the snippet I posted above  in http:#37061067
That's one of the reasons I didn't want to post my original code.  It looked too flaky, and I figured it be easier to work with someone's code that already has been proven to work.

I posted in previous comment that I did try the code in http:#37061067.

This is the string I'm getting back from sendDigestAuthentication:
HTTP/1.1 401 Unauthorized [Content-Length: 1656, Content-Type: text/html, Server: Microsoft-IIS/6.0, WWW-Authenticate: Negotiate, WWW-Authenticate: NTLM, WWW-Authenticate: Digest qop="auth",algorithm=MD5-sess,nonce="+Upgraded+......../.",charset=utf-8,realm="Digest", WWW-Authenticate: Basic realm="mywebsite", X-Powered-By: ASP.NET, Date: Tue, 01 Nov 2011 10:56:09 GMT]

I'm calling it like this:
 
sendDigestAuthentication(taskdataHandler.SiteCredentialsUserName, taskdataHandler.SiteCredentialsPassword, UrlTest, "myweb", 80, "http");
sendDigestAuthentication("\myuser", taskdataHandler.SiteCredentialsPassword, UrlTest, "myweb", 80, "http");
sendDigestAuthentication("mydomain/myuser", taskdataHandler.SiteCredentialsPassword, UrlTest, "myweb", 80, "http");
sendDigestAuthentication("mydomain@myuser", taskdataHandler.SiteCredentialsPassword, UrlTest, "myweb", 80, "http");
sendDigestAuthentication("mydomain\\myuser", taskdataHandler.SiteCredentialsPassword, UrlTest, "myweb", 80, "http");

Open in new window


And sending the output to a log file via following:
cvlog.debug("sendDigestAuthentication = "
            + sendDigestAuthentication("mydomain\\myuser", taskdataHandler.SiteCredentialsPassword, UrlTest, "myweb", 80, "http"));

I get the same error every time.
0
 
AxterAuthor Commented:
Also tried....
sendDigestAuthentication("myuser", taskdataHandler.SiteCredentialsPassword, UrlTest, "myweb", 80, "http");

with same results.
0
 
for_yanCommented:
well, it says - unauthorized - that is another story - not closed stream or anything
Are you sure you have correct user pasword - can you login to that page  manually ?

0
 
AxterAuthor Commented:
I now I have the correct user name and password, because the following code works for basic authentication.
 
static public BufferedReader OpenFileForRead_Org(String inFilePath, taskDataHandler taskdataHandler)
	{
		cvLog cvlog = new cvLog("taskManagerx", "OpenFileForRead");
		BufferedReader in = null;
		try
		{
				// Create a URL for the desired page
				URL url = new URL(inFilePath);

				if (taskdataHandler.SiteCredentialsUserName != null && !taskdataHandler.SiteCredentialsUserName.isEmpty()
						&& taskdataHandler.SiteCredentialsPassword != null && !taskdataHandler.SiteCredentialsPassword.isEmpty())
				{
					cvlog.debug("Will try accessing with credentials");
					HttpURLConnection uc = (HttpURLConnection) url.openConnection();
					String userpassword = taskdataHandler.SiteCredentialsUserName + ":" + taskdataHandler.SiteCredentialsPassword;
					String encodedAuthorization = taskManager.Base64Encode(userpassword);
					uc.setRequestProperty("Authorization", "Basic " + encodedAuthorization);
					in = new BufferedReader(new InputStreamReader(uc.getInputStream()));
					cvlog.detail("in = " + in.toString());
				} else
					in = new BufferedReader(new InputStreamReader(url.openStream()));
		}catch (FileNotFoundException e1)
		{
			in = null;

		} catch (IOException e)
		{
			in = null;
		}

		return in;
	}

Open in new window


I use above function on site which requires credentials, and it works.

I'm passing in the same variables as in the other function using HttpClient.

>>well, it says - unauthorized - that is another story - not closed stream or anything
You're mixing errors with different functions on separate comments.

close stream error is received when using code in function BufferedReader OpenFileForRead_digest2 which I posted with the same comment that states I was getting close stream error.

I get unauthorized error with function sendDigestAuthentication.


0
 
for_yanCommented:
I certainly understand the difference.

where in that function OpenFileForRead_digest2 are you getting this error?
Is it an exception? Does it print a stack trace?
Perhap you need to catch it within the function and print stacktrace - then you coul pinpoint the
the line where it happens
0
 
AxterAuthor Commented:
I get that error in the return function.

BufferedReader in = OpenFileForRead_digest2(MyUrl, taskdataHandler);
String str = null;
while ((str = in.readLine()) != null)  //I get error here, and catch by try-catch
{
      out.println("<!-- str = " + str + " -->");
}

static public BufferedReader OpenFileForRead_digest3(String UrlPath, taskDataHandler taskdataHandler)
	{
		BufferedReader in = null;
		try
		{
			URL url = new URL(UrlPath);
	        DefaultHttpClient httpclient = new DefaultHttpClient();
	        httpclient.getCredentialsProvider().setCredentials(
	                new AuthScope(url.getHost(), url.getPort(), url.getProtocol()), 
	                new UsernamePasswordCredentials(taskdataHandler.SiteCredentialsUserName, taskdataHandler.SiteCredentialsPassword));   
	        HttpGet httpget = new HttpGet(UrlPath);
		    HttpContext localContext = new BasicHttpContext();
			HttpResponse httpresponse = httpclient.execute(httpget, localContext);
	        HttpEntity entity = httpresponse.getEntity();
	        in = new BufferedReader(new InputStreamReader(entity.getContent()));
		}catch (FileNotFoundException e1)
		{
			in = null;

		} catch (IOException e)
		{
			in = null;
		}

		return in;
	}

Open in new window

0
 
AxterAuthor Commented:
Disregard the above function OpenFileForRead_digest3.  I meant to remove it before clicking submit.
0
 
for_yanCommented:
So in what posting you now have the code of this function which is referenced here:

BufferedReader in = OpenFileForRead_digest2(MyUrl, taskdataHandler);
String str = null;
while ((str = in.readLine()) != null)  //I get error here, and catch by try-catch
{
      out.println("<!-- str = " + str + " -->");
}

Or just post it again, as it became confusing
0
 
AxterAuthor Commented:
FYI: If you use Firefox or IE, you can do a keyword search for the function (OpenFileForRead_digest2) and find it in previous post.

See posting ID: 37062698
0
 
for_yanCommented:
I guess this maybe because you already read this entity:

entity.consumeContent();

and now you want to read it again:

in = new BufferedReader(new InputStreamReader(entity.getContent()));
0
 
AxterAuthor Commented:
If that's the case, then how can you use HttpClient to read content via BufferedReader?
0
 
for_yanCommented:
I don't think you need necessarily to use a BufferedReader.
HttpClient returns response, which is  string - if you want to have a reader
you can then open BufferedReader br = new BufferedReader(new StringReader(response)))
something like that or you can dela directly with te string
0
 
AxterAuthor Commented:
>>HttpClient returns response, which is  string - if you want to have a reader

How do I get the response from HttpClient?  Which function get the response?
0
 
for_yanCommented:

if you look at
sendDigestAuthentication

there is where you get the response:

response = httpclient.execute(targetHost,
                    httpget);

well, we can't get through authentication, but that's another story
If you use HttpClient without digest
it also has corresponding method whicvh gets you the response


0
 
AxterAuthor Commented:

static public BufferedReader OpenFileForRead_digest2(String UrlPath, taskDataHandler taskdataHandler) throws Exception
	{
		cvLog cvlog = new cvLog("taskManagerx", "BufferedReader");
		BufferedReader in = null;
		DefaultHttpClient httpclient = new DefaultHttpClient();

		// Create local execution context
		HttpContext localContext = new BasicHttpContext();

		HttpGet httpget = new HttpGet(UrlPath);
		cvlog.debug("executing request " + httpget.getRequestLine());
		HttpResponse response = httpclient.execute(httpget, localContext);

		cvlog.debug(response.getStatusLine().toString());

		// Consume response content
		HttpEntity entity = response.getEntity();
		if (entity != null)
		{
			entity.consumeContent();
		}

		int sc = response.getStatusLine().getStatusCode();

		AuthState authState = null;
		if (sc == HttpStatus.SC_UNAUTHORIZED)
		{
			// Target host authentication required
			authState = (AuthState) localContext.getAttribute(ClientContext.TARGET_AUTH_STATE);
		}
		if (sc == HttpStatus.SC_PROXY_AUTHENTICATION_REQUIRED)
		{
			// Proxy authentication required
			authState = (AuthState) localContext.getAttribute(ClientContext.PROXY_AUTH_STATE);
		}

		if (authState != null)
		{
			AuthScope authScope = authState.getAuthScope();
			Credentials creds = new UsernamePasswordCredentials(taskdataHandler.SiteCredentialsUserName, taskdataHandler.SiteCredentialsPassword);
			httpclient.getCredentialsProvider().setCredentials(authScope, creds);
			
			response = httpclient.execute(httpget, localContext);
			entity = response.getEntity();
			in = new BufferedReader(new InputStreamReader(entity.getContent()));
		}

		return in;
	}

Open in new window

0
 
AxterAuthor Commented:
I modified above function to avoid reusing entity which has been read already, and now I get the access denied error again.
But this error is coming from the data being read from the URL.
HTTP Error 401.2 - Unauthorized: Access is denied due to server configuration.<br>Internet Information Services (IIS)</h2>
0
 
AxterAuthor Commented:
I also tried modifying sendDigestAuthentication to return BufferedReader, but I'm still stuck with the problem that it returns "401 Unauthorized" error.
public static BufferedReader sendDigestAuthentication(String username, String password, String url, String host, int portNumber, String protocol)
	{
		cvLog cvlog = new cvLog("taskManagerx", "sendDigestAuthentication");
		HttpResponse response = null;
		HttpHost targetHost = new HttpHost(host, portNumber, protocol);

		DefaultHttpClient httpclient = new DefaultHttpClient();
		try
		{
			httpclient.getCredentialsProvider().setCredentials(new AuthScope(targetHost.getHostName(), targetHost.getPort()),
					new UsernamePasswordCredentials(username, password));

			HttpGet httpget = new HttpGet(url);
			response = httpclient.execute(targetHost, httpget);
		} catch (Exception exc)
		{
			cvlog.debug("Exception digesting Authentication");
		} finally
		{
			// When HttpClient instance is no longer needed,
			// shut down the connection manager to ensure
			// immediate deallocation of all system resources
			httpclient.getConnectionManager().shutdown();
		}
		
		return new BufferedReader(new StringReader(response.toString())); //response.toString();
	}

Open in new window

0
 
for_yanCommented:
Yes, this is the issue that we cannot get through authorization
0
 
for_yanCommented:

I was reading below in Wikipedia:
http://en.wikipedia.org/wiki/Digest_access_authentication

-------------------------------------------
This typical transaction consists of the following steps.

    The client asks for a page that requires authentication but does not provide a username and password. Typically this is because the user simply entered the address or followed a link to the page.
    The server responds with the 401 "client-error" response code, providing the authentication realm and a randomly-generated, single-use value called a nonce.
    At this point, the client will present the authentication realm (typically a description of the computer or system being accessed) to the user and prompt for a username and password. The user may decide to cancel at this point.
    Once a username and password have been supplied, the client re-sends the same request but adds an authentication header that includes the response code.
    In this example, the server accepts the authentication and the page is returned. If the username is invalid and/or the password is incorrect, the server might return the "401" response code and the client would prompt the user again.

Note: A client may already have the required username and password without needing to prompt the user, e.g. if they have previously been stored by a web browser.
---------------------------------------

But my assumption is that  this sendDigeestAuthorization should handle those steps internally

Are all 100% of  web servers are supposed to accept this digest athorization?

0
 
AxterAuthor Commented:
I have a mix set of servers, with some having Basic, Digest, and anonymous access.
I also have some with NTLM access, but those are all setup to also take Digest.

I can get my original code working on servers with Basic and anonymous access, but I can't get it to work with Digest.

And the HttpClient code doesn't work with basic or digest.
0
 
for_yanCommented:
You see they say in Wikipedia the server should return nonce with 401 message.
Is this

nonce="+Upgraded+......../.",

indeed what it retrurns in your message?
0
 
AxterAuthor Commented:
>>indeed what it retrurns in your message?

How would I change the code to give right response?
0
 
for_yanCommented:
Don't know - it should send nonce if the client sends request without any user/password,
but I guess it should still be there for any case of non-authentication
So, it is strange that it sends this.
Are all the servers which you try sending nonce in this way?
0
 
AxterAuthor Commented:
I only have two servers with digest authentication, and they both return the same error.
0
 
for_yanCommented:


I don't know, I  found one more variant to try with HttpClient from
http://stackoverflow.com/questions/5917356/httpclient-4-1-1-returns-401-when-authenticating-with-ntlm-browsers-work-fine
I modified it a little bit to output the real stuff, but don't know how to test - it looks like to all sites where
I use to use the password it allows me to read stuff without any password :)
Really don't understand how it works.
Thought, that it redirects to login page, but I really can read
the text behind my login without password with this code - maybe there is some caching there going on
Maybe you could try it on your server.

No actually on one local server I really see it redirects me to login page  - but I don't know what should be the host
to authenticate my NT  credentials in this line:
 new NTCredentials("user", "password", "", "localhost"));
I need to talk to our IT folks I guess to understand it.

Also read something the guy was writint in that link on stackoverflow about proxy - below the question
couple of posts later where he explains the solution (rather vaguely).

I used it after I dowbloaded HttpClient and HttpCore from

http://mirror.nyi.net/apache//httpcomponents/httpclient/binary/httpcomponents-client-4.1.2-osgi-bin.zip

and

http://mirror.nyi.net/apache//httpcomponents/httpcore/binary/httpcomponents-core-4.1.3-osgi-bin.zip

and extracted the two jars from them




public class DigestAuthent {
 public static void main(String[] args) {
      
        DefaultHttpClient httpclient = new DefaultHttpClient();
            HttpContext localContext = new BasicHttpContext();
           // HttpGet httpget = new HttpGet("http://www.google.com/");
          HttpGet httpget = new HttpGet("http://www.google.com");
            CredentialsProvider credsProvider = new BasicCredentialsProvider();
            credsProvider.setCredentials(AuthScope.ANY,
                    new NTCredentials("user", "password", "", "localhost"));
            if (!new File(System.getenv("windir") + "\\krb5.ini").exists()) {
                List<String> authtypes = new ArrayList<String>();
               authtypes.add(AuthPolicy.NTLM);
                authtypes.add(AuthPolicy.DIGEST);
               authtypes.add(AuthPolicy.BASIC);
                httpclient.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF,
                        authtypes);
                httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF,
                        authtypes);
            }
            localContext.setAttribute(ClientContext.CREDS_PROVIDER, credsProvider);
           try{
            HttpResponse response = httpclient.execute(httpget, localContext);
            HttpEntity en = response.getEntity();
               System.out.println(en.getContentLength());
               DataInputStream in = new DataInputStream(en.getContent());
               String buff = null;
               while((buff= in.readLine()) != null)System.out.println(buff);
            System.out.println("Response code: " + response.getStatusLine());
           }catch(Exception ex){
               ex.printStackTrace();
           }




    }
}

Open in new window


0
 
for_yanCommented:
But interesting,  no matter how I tried with the above code, - I could not get that 401 error which you saw.
In the worst case (on our local  site which requires authorization) I saw that the actual data was coming from login page , rather than from behind the login, where
my URL was pointing; but not this 401. I guess this becuase of how redirection is working.
0
 
AxterAuthor Commented:
The above function gave me the same error, but there was a piece I took out of it, which help me get the NTLM code to work.
httpclient.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF,  authtypes);
httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF, authtypes);

I added the above lines from that code to the following function:
 
//Code to read URL using NTLM authentication
	public static BufferedReader OpenFileForRead(String UrlPath, taskDataHandler taskdataHandler, String Domain) throws Exception
	{
		BufferedReader in = null;
		DefaultHttpClient httpclient = new DefaultHttpClient();
		URL url = new URL(UrlPath);

		org.apache.http.auth.Credentials  creds = new NTCredentials(taskdataHandler.SiteCredentialsUserName, taskdataHandler.SiteCredentialsPassword, 
				url.getHost(), Domain);
		
		List<String> authPrefs = new ArrayList<String>();
		authPrefs.add(AuthPolicy.NTLM);
		authPrefs.add(AuthPolicy.DIGEST);
		authPrefs.add(AuthPolicy.BASIC);
		httpclient.getParams().setParameter(AuthPolicy.AUTH_SCHEME_PRIORITY, authPrefs);
    	httpclient.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF, authPrefs);
		httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF, authPrefs);
		httpclient.getCredentialsProvider().setCredentials(AuthScope.ANY,  creds);

		HttpHost target = new HttpHost(url.getHost(), url.getPort(), url.getProtocol());

		// Make sure the same context is used to execute logically related requests
		HttpContext localContext = new BasicHttpContext();

		// Execute a cheap method first. This will trigger NTLM authentication
		HttpGet httpget = new HttpGet(url.getPath());
		HttpResponse response1 = httpclient.execute(target, httpget, localContext);
		HttpEntity entity1 = response1.getEntity();
		in = new BufferedReader(new InputStreamReader(entity1.getContent()));
		
		return in;
	}

Open in new window


So now all I need to do is get the Digest version to work.

I'm going to try to modify the Digest version to make it as close as to the NTLM version, and see if that gets it working.

Thanks
0
 
for_yanCommented:

Also read below from the above link in stackoverflow
Thsi is not quite claer to me, and I'm not
sure it is relavant to you, but still maybe it will be of some help:
      
I finally figured it out. Digest authentication requires that if you use a full URL in the request, the proxy also needs to use the full URL. I did not leave the proxy code in the sample, but it was directed to "localhost", which caused it to fail. Changing this to 127.0.0.1 made it work.
 
0
 
AxterAuthor Commented:
Thanks for all the help.
I found out the main problem with getting Digest to work had to do with the machine I was testing from.

I ran the code on another machine, and it work with Digest.
I also found some specific users worked with Digest on the bad machine.
I'm not sure what is causing that issue, but it's not related to the code.

The last version of the code I posted, worked for Basic, Digest, and NTLM.
0
 
for_yanCommented:
Great!
I'm happy your resolved this actually very convoluted issue.
0

Featured Post

New feature and membership benefit!

New feature! Upgrade and increase expert visibility of your issues with Priority Questions.

  • 21
  • 17
Tackle projects and never again get stuck behind a technical roadblock.
Join Now