We help IT Professionals succeed at work.

Retrieval of URL with authorization

rhee asked
Medium Priority
Last Modified: 2008-03-06
The java source at the end retrieves webpage with URL:
but fails for files requesting authorization info.

I'd like to know how to modify the program to handle authorization.
(Of course, I know the ID and password, and I just want to do it
in batch-job rather than doing it in a browser)

Thanks in advance.


Gwangsoo Rhee
Department of Computer Science, SookMyung Women's University
Email       : rhee@cs.sookmyung.ac.kr

---- java source -----------

import java.applet.Applet;
import java.awt.*;
import java.net.*;
import java.io.*;

class urlGet {

    TextField urlField = null;
    TextArea contentArea = null;
    TextField contentType = null;
    TextField contentSize = null;

    public static void main (String args[]) {
        URL target = null;

        try {
            target = new URL ("http://host/path/file");
        } catch( MalformedURLException e ) {
        try {
            URLConnection con = target.openConnection();
            BufferedInputStream in =
                new BufferedInputStream (con.getInputStream(), 2048);
            FileOutputStream out = new FileOutputStream("file");

            byte b[] = new byte[1024];
            int nbytes;
            String content = null;

            while ((nbytes = in.read(b, 0, 1024)) != -1 ) {
                out.write (b, 0, nbytes);
        } catch (java.io.IOException e) {

--- end of source ---
Watch Question

You need to use the URLConnection class rather than the URL class.

For docs and examples on its use see:


Once the page has loaded, look near the begining of the page for the section called: The URLConnection Class

Note: Save this URL to your hard disk since these pages appear and disappear without notice. If you cannot acces it, send me a note to jkelleghan@usa.net and I'll e-mail you the page.


Sorry about your efforts, but it didn't help me at all.
The materials you suggested didn't tell me how to handle
the authentication dialog, which I really need to know.
Take a look at:


for samples on how to establish one and 2 way communication with a URL.

Also look at:


for URLConnection info vis a vis Netscape.

You might also want to see the thread at:


which is currently discussing how to use URLConnection and

 connection.setRequestProperty("Authorization", "Basic " + RFC1421Encode("rod:hanks"));

No actual samples on how to fill the password dialog, but should be enough to get you started on your way.to solve your problem.
Here is another snippet that might help:

How do I retrieve an html page which is on an http server using authentication?

"This needs an encoded password and username (base64) and the ability to support the http authentication protocol extensions."

(Submitted by Ian Goh)

   If you need to provide basic authentication information, you have to
   send the WWW Server the following line (at least!)

   Authorization: Basic auth_info

   where auth_info is the string "username:password" encoded in the
   Base64 Printable Encoding format.

   I rewrote the WWW "HTUU.C" encode function in Java.  I believe you
   do not need any other packages (except java.lang).


   String auth_info = HTUU.encode("username:password");


**      Main code is taken from the HTUU.C distribution, and was originally
**      written by Mark Riordan (riordanmr@clvax1.cl.msu.edu)
** and Ari Luotonen (luotonen@dxcern.cern.ch).
** IG  Ian Goh         ian.goh@jhu.edu
** Converted HTUU.C "HTUU_encode" function into Java (1.0.2): IG 13 July 1996
** -------------------------------------------------------------
**  File contains a routine to convert a buffer
**  of bytes to RFC 1113 printable encoding format.
**  This technique is similar to the familiar Unix uuencode
**  format in that it maps 6 binary bits to one ASCII
**  character (or more aptly, 3 binary bytes to 4 ASCII
**  characters).  However, RFC 1113 does not use the same
**  mapping to printable characters as uuencode.
**  Mark Riordan   12 August 1990 and 17 Feb 1991.
**  This code is hereby placed in the public domain.
** -------------------------------------------------------------

class HTUU {

      static String version = "HTUU Class v1.0 7/13/96";
      // the Base64 printable encoding characters
      static char[] ENC = {
      // function encode takes the "username:password" string and
      // converts it into the printable encoding format
      static String encode(String string) {
            int i, j;
            byte[] byte_array = new byte[3];
            StringBuffer buf_coded = new StringBuffer();

            // get length of input string
             int nbytes = string.length();                              
            for (i = 0; i < nbytes; i+= 3) {

                  // check to make sure we don't run off the end of input string
                  if (i + 3 < nbytes)
                        j = i + 3;
                        j = nbytes;

                  string.getBytes(i, j, byte_array, 0);            // get bytes i..j
                  if (j - i == 1) {
                        // missing last two bytes
                        byte_array[1] = 0;
                        byte_array[2] = 0;
                  if (j - i == 2) {
                        // missing last byte
                        byte_array[2] = 0;
                  // convert the three bytes into four Base64 characters
                  // and append to the buf_coded string buffer
                  buf_coded.append(ENC[byte_array[0] >> 2]);          
                  buf_coded.append(ENC[((byte_array[0] << 4) & 060) | ((byte_array[1] >> 4) & 017)]);
                  buf_coded.append(ENC[((byte_array[1] << 2) & 074) | ((byte_array[2] >> 6) & 03)]);
                  buf_coded.append(ENC[byte_array[2] & 077]);

            } // end for loop

            // If nbytes was not a multiple of 3, then we have encoded too
            // many characters.  Adjust appropriately.

            int buf_length = buf_coded.length();
              if (i == nbytes+1) {
                 /* There were only 2 bytes in that last group */
                 buf_coded.setCharAt(buf_length - 1, '=');
              } else if (i == nbytes+2) {
                 /* There was only 1 byte in that last group */
                 buf_coded.setCharAt(buf_length - 1, '=');
                 buf_coded.setCharAt(buf_length - 2, '=');
            // return the Base64 encoded string
            return buf_coded.toString();
      } // end String encode (String)
} // end Class HTUU
I think we need to start by explaining what the issues are with HTTP, and the limitations in the URL and URLConnection classes.

HTTP/1.0 will return an error status of 401 if you try to access an area that requires authorization.  Your program needs some way to watch for this message.  If 401 is the status, you have to reissue your request with an added field in the HTTP header:

     Authorization:   user   username:userpassword

The word "user" indicates the type of security being used.  It could also be kerberos, or something similar.

If the name and password are correct, you should be able to continue.

Unfortunately, doing this is a little complicated.  URL doesn't allow you to get at most of the HTTP header information.  URLConnection does a better job, but there are still problems.  Apparently, it does not consistently honour setRequestProperty()
which is how you add the Authorization line.  Also (and this just may be my ignorance) I don't know how you can determine the status code returned.  The published API for URLConnection I have doesn't include a getStatus() call or anything similar, nor is there a URLAuthorizationException class.

What you really need to do is to get at the HTTP header information free of interference from the class.  You could use the socket classes and implement your own HTTP class.  That seems a little extreme, especially when many people have already done it.  For example, take a look at:


or even better, at


Both offer source code, so you can do your own thing if they don't offer what you want.  The latter looks far more complete, which may be a good thing or a bad thing depending on your point of view.  Using it, you can actually set authorizations for specific pages and it appears from the API info that it will be used automatically.

On the other hand, if all you want to do is issue very simple HTTP requests with the odd authorization line, maybe it would be easier to write your own subclass of Socket.  Take a look at the documentation for HTTP available through:


You don't actually need to handle that much of the protocol to get a working HTTP client for very simple requests.

Good luck, and add a comment if I can help further.

Not the solution you were looking for? Getting a personalized solution is easy.

Ask the Experts


Thank you, "pedxing", but
I solved my problem thanks to "jpk"'s comments and the class HTUU.

Here's the first successful code (It's for you, "jpk"):

import java.net.*;
import java.io.*;

class ConnectionTest {
    public static void main(String[] args) {
        try {
            URL url = new URL("http://host/path/file");
            URLConnection urlConnection = url.openConnection();              urlConnection.setRequestProperty("Authorization",
                     "Basic " + HTUU.encode("user:password"));
            DataInputStream dis
                = new                  DataInputStream(urlConnection.getInputStream());
            String inputLine;

            while ((inputLine = dis.readLine()) != null) {
        } catch (MalformedURLException me) {
            System.out.println("MalformedURLException: " + me);
        } catch (IOException ioe) {
            System.out.println("IOException: " + ioe);
        }                                                            }        

I'm glad you were able to get your code working.  I should caution you that setRequestProperty() will not work on all machines if they are running 1.02, but perhaps you are on an Intranet (or developing a program for the future) so it doesn't matter.

If you are planning to put your code up on a web server on the Internet soon, though, I suggest you look for alternatives to URLConnection that are less buggy.  For your reference, here is the bug report and its location:
From http://www.javasoft.com/products/jdk/1.0.2/KnownBugs.html#Net

URLConnection - There is a bug in JDK1.0.2 where the methods on URLConnection did not work:


          These are fixed in JDK1.1.


Thank ina, "jpk", and "pedxing".
Access more of Experts Exchange with a free account
Thanks for using Experts Exchange.

Create a free account to continue.

Limited access with a free account allows you to:

  • View three pieces of content (articles, solutions, posts, and videos)
  • Ask the experts questions (counted toward content limit)
  • Customize your dashboard and profile

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.


Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.