Solved

Retrieval of URL with authorization

Posted on 1997-05-05
8
1,178 Views
Last Modified: 2008-03-06
The java source at the end retrieves webpage with URL:
      http://host/path/file,
but fails for files requesting authorization info.

I'd like to know how to modify the program to handle authorization.
(Of course, I know the ID and password, and I just want to do it
in batch-job rather than doing it in a browser)

Thanks in advance.

--

Gwangsoo Rhee
Department of Computer Science, SookMyung Women's University
Email       : rhee@cs.sookmyung.ac.kr

---- java source -----------

import java.applet.Applet;
import java.awt.*;
import java.net.*;
import java.io.*;

class urlGet {

    TextField urlField = null;
    TextArea contentArea = null;
    TextField contentType = null;
    TextField contentSize = null;

    public static void main (String args[]) {
        URL target = null;

        try {
            target = new URL ("http://host/path/file");
        } catch( MalformedURLException e ) {
            return;
        }                                                              
        try {
            URLConnection con = target.openConnection();
            BufferedInputStream in =
                new BufferedInputStream (con.getInputStream(), 2048);
            FileOutputStream out = new FileOutputStream("file");

            byte b[] = new byte[1024];
            int nbytes;
            String content = null;

            while ((nbytes = in.read(b, 0, 1024)) != -1 ) {
                out.write (b, 0, nbytes);
            }
            out.close();
        } catch (java.io.IOException e) {
            return;
        }
    }
}

--- end of source ---
0
Comment
Question by:rhee
  • 3
  • 3
  • 2
8 Comments
 
LVL 6

Expert Comment

by:jpk041897
ID: 1220155
You need to use the URLConnection class rather than the URL class.

For docs and examples on its use see:

http://www.mcp.com/que/et/se_java2e/32javafi.htm

Once the page has loaded, look near the begining of the page for the section called: The URLConnection Class

Note: Save this URL to your hard disk since these pages appear and disappear without notice. If you cannot acces it, send me a note to jkelleghan@usa.net and I'll e-mail you the page.
0
 

Author Comment

by:rhee
ID: 1220156
Sorry about your efforts, but it didn't help me at all.
The materials you suggested didn't tell me how to handle
the authentication dialog, which I really need to know.
0
 
LVL 6

Expert Comment

by:jpk041897
ID: 1220157
Take a look at:

http://www.inf.uni-hohenheim.de/top/java/tutorial/networking/urls/readingWriting.html 

for samples on how to establish one and 2 way communication with a URL.

Also look at:

http://www.io.com/~maus/jnetfaq.html

for URLConnection info vis a vis Netscape.

You might also want to see the thread at:

http://xp9.dejanews.com/getdoc.xp?recnum=104873&server=db97p2&CONTEXT=862904820.13021&hitnum=3

which is currently discussing how to use URLConnection and

 connection.setRequestProperty("Authorization", "Basic " + RFC1421Encode("rod:hanks"));

No actual samples on how to fill the password dialog, but should be enough to get you started on your way.to solve your problem.
0
Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

 
LVL 6

Expert Comment

by:jpk041897
ID: 1220158
Here is another snippet that might help:

How do I retrieve an html page which is on an http server using authentication?

"This needs an encoded password and username (base64) and the ability to support the http authentication protocol extensions."






(Submitted by Ian Goh)


/***************************************************************************
   If you need to provide basic authentication information, you have to
   send the WWW Server the following line (at least!)

   Authorization: Basic auth_info

   where auth_info is the string "username:password" encoded in the
   Base64 Printable Encoding format.

   I rewrote the WWW "HTUU.C" encode function in Java.  I believe you
   do not need any other packages (except java.lang).

   Usage:

   String auth_info = HTUU.encode("username:password");

*****************************************************************************/



/*                                                       
** HTUU.CLASS
**
** ACKNOWLEDGEMENT:
**      Main code is taken from the HTUU.C distribution, and was originally
**      written by Mark Riordan (riordanmr@clvax1.cl.msu.edu)
** and Ari Luotonen (luotonen@dxcern.cern.ch).
**
** AUTHORS:
** IG  Ian Goh         ian.goh@jhu.edu
**
** HISTORY:
** Converted HTUU.C "HTUU_encode" function into Java (1.0.2): IG 13 July 1996
** -------------------------------------------------------------
**  File contains a routine to convert a buffer
**  of bytes to RFC 1113 printable encoding format.
**
**  This technique is similar to the familiar Unix uuencode
**  format in that it maps 6 binary bits to one ASCII
**  character (or more aptly, 3 binary bytes to 4 ASCII
**  characters).  However, RFC 1113 does not use the same
**  mapping to printable characters as uuencode.
**
**  Mark Riordan   12 August 1990 and 17 Feb 1991.
**  This code is hereby placed in the public domain.
** -------------------------------------------------------------
**
**
*/

class HTUU {

      static String version = "HTUU Class v1.0 7/13/96";
      
      // the Base64 printable encoding characters
      static char[] ENC = {
            'A','B','C','D','E','F','G','H','I','J','K','L','M',
            'N','O','P','Q','R','S','T','U','V','W','X','Y','Z',
            'a','b','c','d','e','f','g','h','i','j','k','l','m',
            'n','o','p','q','r','s','t','u','v','w','x','y','z',
            '0','1','2','3','4','5','6','7','8','9','+','/'
      };
            
      // function encode takes the "username:password" string and
      // converts it into the printable encoding format
                  
      static String encode(String string) {
     
            int i, j;
            byte[] byte_array = new byte[3];
            StringBuffer buf_coded = new StringBuffer();

            // get length of input string
             int nbytes = string.length();                              
 
            for (i = 0; i < nbytes; i+= 3) {

                  // check to make sure we don't run off the end of input string
                  if (i + 3 < nbytes)
                        j = i + 3;
                  else
                        j = nbytes;

                  string.getBytes(i, j, byte_array, 0);            // get bytes i..j
                   
                  if (j - i == 1) {
                        // missing last two bytes
                        byte_array[1] = 0;
                        byte_array[2] = 0;
                  }
         
                  if (j - i == 2) {
                        // missing last byte
                        byte_array[2] = 0;
                  }
         
                  // convert the three bytes into four Base64 characters
                  // and append to the buf_coded string buffer
                  buf_coded.append(ENC[byte_array[0] >> 2]);          
                  buf_coded.append(ENC[((byte_array[0] << 4) & 060) | ((byte_array[1] >> 4) & 017)]);
                  buf_coded.append(ENC[((byte_array[1] << 2) & 074) | ((byte_array[2] >> 6) & 03)]);
                  buf_coded.append(ENC[byte_array[2] & 077]);

            } // end for loop

            // If nbytes was not a multiple of 3, then we have encoded too
            // many characters.  Adjust appropriately.

            int buf_length = buf_coded.length();
      
              if (i == nbytes+1) {
                 /* There were only 2 bytes in that last group */
                 buf_coded.setCharAt(buf_length - 1, '=');
              } else if (i == nbytes+2) {
                 /* There was only 1 byte in that last group */
                 buf_coded.setCharAt(buf_length - 1, '=');
                 buf_coded.setCharAt(buf_length - 2, '=');
              }
           
            // return the Base64 encoded string
            return buf_coded.toString();
     
      } // end String encode (String)
   
} // end Class HTUU
0
 

Accepted Solution

by:
pedxing earned 200 total points
ID: 1220159
I think we need to start by explaining what the issues are with HTTP, and the limitations in the URL and URLConnection classes.

HTTP/1.0 will return an error status of 401 if you try to access an area that requires authorization.  Your program needs some way to watch for this message.  If 401 is the status, you have to reissue your request with an added field in the HTTP header:

     Authorization:   user   username:userpassword

The word "user" indicates the type of security being used.  It could also be kerberos, or something similar.

If the name and password are correct, you should be able to continue.

Unfortunately, doing this is a little complicated.  URL doesn't allow you to get at most of the HTTP header information.  URLConnection does a better job, but there are still problems.  Apparently, it does not consistently honour setRequestProperty()
which is how you add the Authorization line.  Also (and this just may be my ignorance) I don't know how you can determine the status code returned.  The published API for URLConnection I have doesn't include a getStatus() call or anything similar, nor is there a URLAuthorizationException class.

What you really need to do is to get at the HTTP header information free of interference from the class.  You could use the socket classes and implement your own HTTP class.  That seems a little extreme, especially when many people have already done it.  For example, take a look at:

    http://www.princeton.edu/~ammulder/java_http.html

or even better, at

http://www.innovation.ch/java/HTTPClient/

Both offer source code, so you can do your own thing if they don't offer what you want.  The latter looks far more complete, which may be a good thing or a bad thing depending on your point of view.  Using it, you can actually set authorizations for specific pages and it appears from the API info that it will be used automatically.

On the other hand, if all you want to do is issue very simple HTTP requests with the odd authorization line, maybe it would be easier to write your own subclass of Socket.  Take a look at the documentation for HTTP available through:

http://www.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/HTTP/Protocol_Specification/

You don't actually need to handle that much of the protocol to get a working HTTP client for very simple requests.

Good luck, and add a comment if I can help further.

0
 

Author Comment

by:rhee
ID: 1220160
Thank you, "pedxing", but
I solved my problem thanks to "jpk"'s comments and the class HTUU.

Here's the first successful code (It's for you, "jpk"):
-----------------------------------------------------------------

import java.net.*;
import java.io.*;

class ConnectionTest {
    public static void main(String[] args) {
        try {
            URL url = new URL("http://host/path/file");
            URLConnection urlConnection = url.openConnection();              urlConnection.setRequestProperty("Authorization",
                     "Basic " + HTUU.encode("user:password"));
            DataInputStream dis
                = new                  DataInputStream(urlConnection.getInputStream());
            String inputLine;

            while ((inputLine = dis.readLine()) != null) {
                System.out.println(inputLine);
            }
            dis.close();
        } catch (MalformedURLException me) {
            System.out.println("MalformedURLException: " + me);
        } catch (IOException ioe) {
            System.out.println("IOException: " + ioe);
        }                                                            }        
}
0
 

Expert Comment

by:pedxing
ID: 1220161
I'm glad you were able to get your code working.  I should caution you that setRequestProperty() will not work on all machines if they are running 1.02, but perhaps you are on an Intranet (or developing a program for the future) so it doesn't matter.

If you are planning to put your code up on a web server on the Internet soon, though, I suggest you look for alternatives to URLConnection that are less buggy.  For your reference, here is the bug report and its location:
From http://www.javasoft.com/products/jdk/1.0.2/KnownBugs.html#Net

URLConnection - There is a bug in JDK1.0.2 where the methods on URLConnection did not work:

          setRequestProperty()
          getRequestProperty()
          getHeaderField()
          getHeaderFieldInt()
          getContentLength()
          getDate()
          etc..

          These are fixed in JDK1.1.

0
 

Author Comment

by:rhee
ID: 1220162
Thank ina, "jpk", and "pedxing".
0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
HSSFWorkbook cannot be resolved error 10 68
even odd program using while loop 3 40
Error in @AspectJ Based AOP with Spring 2 13
going to wrong jsp page 2 20
Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
Viewers learn about the “for” loop and how it works in Java. By comparing it to the while loop learned before, viewers can make the transition easily. You will learn about the formatting of the for loop as we write a program that prints even numbers…
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question