Solved

Retrieval of URL with authorization

Posted on 1997-05-05
8
1,176 Views
Last Modified: 2008-03-06
The java source at the end retrieves webpage with URL:
      http://host/path/file,
but fails for files requesting authorization info.

I'd like to know how to modify the program to handle authorization.
(Of course, I know the ID and password, and I just want to do it
in batch-job rather than doing it in a browser)

Thanks in advance.

--

Gwangsoo Rhee
Department of Computer Science, SookMyung Women's University
Email       : rhee@cs.sookmyung.ac.kr

---- java source -----------

import java.applet.Applet;
import java.awt.*;
import java.net.*;
import java.io.*;

class urlGet {

    TextField urlField = null;
    TextArea contentArea = null;
    TextField contentType = null;
    TextField contentSize = null;

    public static void main (String args[]) {
        URL target = null;

        try {
            target = new URL ("http://host/path/file");
        } catch( MalformedURLException e ) {
            return;
        }                                                              
        try {
            URLConnection con = target.openConnection();
            BufferedInputStream in =
                new BufferedInputStream (con.getInputStream(), 2048);
            FileOutputStream out = new FileOutputStream("file");

            byte b[] = new byte[1024];
            int nbytes;
            String content = null;

            while ((nbytes = in.read(b, 0, 1024)) != -1 ) {
                out.write (b, 0, nbytes);
            }
            out.close();
        } catch (java.io.IOException e) {
            return;
        }
    }
}

--- end of source ---
0
Comment
Question by:rhee
  • 3
  • 3
  • 2
8 Comments
 
LVL 6

Expert Comment

by:jpk041897
ID: 1220155
You need to use the URLConnection class rather than the URL class.

For docs and examples on its use see:

http://www.mcp.com/que/et/se_java2e/32javafi.htm

Once the page has loaded, look near the begining of the page for the section called: The URLConnection Class

Note: Save this URL to your hard disk since these pages appear and disappear without notice. If you cannot acces it, send me a note to jkelleghan@usa.net and I'll e-mail you the page.
0
 

Author Comment

by:rhee
ID: 1220156
Sorry about your efforts, but it didn't help me at all.
The materials you suggested didn't tell me how to handle
the authentication dialog, which I really need to know.
0
 
LVL 6

Expert Comment

by:jpk041897
ID: 1220157
Take a look at:

http://www.inf.uni-hohenheim.de/top/java/tutorial/networking/urls/readingWriting.html

for samples on how to establish one and 2 way communication with a URL.

Also look at:

http://www.io.com/~maus/jnetfaq.html

for URLConnection info vis a vis Netscape.

You might also want to see the thread at:

http://xp9.dejanews.com/getdoc.xp?recnum=104873&server=db97p2&CONTEXT=862904820.13021&hitnum=3

which is currently discussing how to use URLConnection and

 connection.setRequestProperty("Authorization", "Basic " + RFC1421Encode("rod:hanks"));

No actual samples on how to fill the password dialog, but should be enough to get you started on your way.to solve your problem.
0
 
LVL 6

Expert Comment

by:jpk041897
ID: 1220158
Here is another snippet that might help:

How do I retrieve an html page which is on an http server using authentication?

"This needs an encoded password and username (base64) and the ability to support the http authentication protocol extensions."






(Submitted by Ian Goh)


/***************************************************************************
   If you need to provide basic authentication information, you have to
   send the WWW Server the following line (at least!)

   Authorization: Basic auth_info

   where auth_info is the string "username:password" encoded in the
   Base64 Printable Encoding format.

   I rewrote the WWW "HTUU.C" encode function in Java.  I believe you
   do not need any other packages (except java.lang).

   Usage:

   String auth_info = HTUU.encode("username:password");

*****************************************************************************/



/*                                                       
** HTUU.CLASS
**
** ACKNOWLEDGEMENT:
**      Main code is taken from the HTUU.C distribution, and was originally
**      written by Mark Riordan (riordanmr@clvax1.cl.msu.edu)
** and Ari Luotonen (luotonen@dxcern.cern.ch).
**
** AUTHORS:
** IG  Ian Goh         ian.goh@jhu.edu
**
** HISTORY:
** Converted HTUU.C "HTUU_encode" function into Java (1.0.2): IG 13 July 1996
** -------------------------------------------------------------
**  File contains a routine to convert a buffer
**  of bytes to RFC 1113 printable encoding format.
**
**  This technique is similar to the familiar Unix uuencode
**  format in that it maps 6 binary bits to one ASCII
**  character (or more aptly, 3 binary bytes to 4 ASCII
**  characters).  However, RFC 1113 does not use the same
**  mapping to printable characters as uuencode.
**
**  Mark Riordan   12 August 1990 and 17 Feb 1991.
**  This code is hereby placed in the public domain.
** -------------------------------------------------------------
**
**
*/

class HTUU {

      static String version = "HTUU Class v1.0 7/13/96";
      
      // the Base64 printable encoding characters
      static char[] ENC = {
            'A','B','C','D','E','F','G','H','I','J','K','L','M',
            'N','O','P','Q','R','S','T','U','V','W','X','Y','Z',
            'a','b','c','d','e','f','g','h','i','j','k','l','m',
            'n','o','p','q','r','s','t','u','v','w','x','y','z',
            '0','1','2','3','4','5','6','7','8','9','+','/'
      };
            
      // function encode takes the "username:password" string and
      // converts it into the printable encoding format
                  
      static String encode(String string) {
     
            int i, j;
            byte[] byte_array = new byte[3];
            StringBuffer buf_coded = new StringBuffer();

            // get length of input string
             int nbytes = string.length();                              
 
            for (i = 0; i < nbytes; i+= 3) {

                  // check to make sure we don't run off the end of input string
                  if (i + 3 < nbytes)
                        j = i + 3;
                  else
                        j = nbytes;

                  string.getBytes(i, j, byte_array, 0);            // get bytes i..j
                   
                  if (j - i == 1) {
                        // missing last two bytes
                        byte_array[1] = 0;
                        byte_array[2] = 0;
                  }
         
                  if (j - i == 2) {
                        // missing last byte
                        byte_array[2] = 0;
                  }
         
                  // convert the three bytes into four Base64 characters
                  // and append to the buf_coded string buffer
                  buf_coded.append(ENC[byte_array[0] >> 2]);          
                  buf_coded.append(ENC[((byte_array[0] << 4) & 060) | ((byte_array[1] >> 4) & 017)]);
                  buf_coded.append(ENC[((byte_array[1] << 2) & 074) | ((byte_array[2] >> 6) & 03)]);
                  buf_coded.append(ENC[byte_array[2] & 077]);

            } // end for loop

            // If nbytes was not a multiple of 3, then we have encoded too
            // many characters.  Adjust appropriately.

            int buf_length = buf_coded.length();
      
              if (i == nbytes+1) {
                 /* There were only 2 bytes in that last group */
                 buf_coded.setCharAt(buf_length - 1, '=');
              } else if (i == nbytes+2) {
                 /* There was only 1 byte in that last group */
                 buf_coded.setCharAt(buf_length - 1, '=');
                 buf_coded.setCharAt(buf_length - 2, '=');
              }
           
            // return the Base64 encoded string
            return buf_coded.toString();
     
      } // end String encode (String)
   
} // end Class HTUU
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 

Accepted Solution

by:
pedxing earned 200 total points
ID: 1220159
I think we need to start by explaining what the issues are with HTTP, and the limitations in the URL and URLConnection classes.

HTTP/1.0 will return an error status of 401 if you try to access an area that requires authorization.  Your program needs some way to watch for this message.  If 401 is the status, you have to reissue your request with an added field in the HTTP header:

     Authorization:   user   username:userpassword

The word "user" indicates the type of security being used.  It could also be kerberos, or something similar.

If the name and password are correct, you should be able to continue.

Unfortunately, doing this is a little complicated.  URL doesn't allow you to get at most of the HTTP header information.  URLConnection does a better job, but there are still problems.  Apparently, it does not consistently honour setRequestProperty()
which is how you add the Authorization line.  Also (and this just may be my ignorance) I don't know how you can determine the status code returned.  The published API for URLConnection I have doesn't include a getStatus() call or anything similar, nor is there a URLAuthorizationException class.

What you really need to do is to get at the HTTP header information free of interference from the class.  You could use the socket classes and implement your own HTTP class.  That seems a little extreme, especially when many people have already done it.  For example, take a look at:

    http://www.princeton.edu/~ammulder/java_http.html

or even better, at

http://www.innovation.ch/java/HTTPClient/

Both offer source code, so you can do your own thing if they don't offer what you want.  The latter looks far more complete, which may be a good thing or a bad thing depending on your point of view.  Using it, you can actually set authorizations for specific pages and it appears from the API info that it will be used automatically.

On the other hand, if all you want to do is issue very simple HTTP requests with the odd authorization line, maybe it would be easier to write your own subclass of Socket.  Take a look at the documentation for HTTP available through:

http://www.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/HTTP/Protocol_Specification/

You don't actually need to handle that much of the protocol to get a working HTTP client for very simple requests.

Good luck, and add a comment if I can help further.

0
 

Author Comment

by:rhee
ID: 1220160
Thank you, "pedxing", but
I solved my problem thanks to "jpk"'s comments and the class HTUU.

Here's the first successful code (It's for you, "jpk"):
-----------------------------------------------------------------

import java.net.*;
import java.io.*;

class ConnectionTest {
    public static void main(String[] args) {
        try {
            URL url = new URL("http://host/path/file");
            URLConnection urlConnection = url.openConnection();              urlConnection.setRequestProperty("Authorization",
                     "Basic " + HTUU.encode("user:password"));
            DataInputStream dis
                = new                  DataInputStream(urlConnection.getInputStream());
            String inputLine;

            while ((inputLine = dis.readLine()) != null) {
                System.out.println(inputLine);
            }
            dis.close();
        } catch (MalformedURLException me) {
            System.out.println("MalformedURLException: " + me);
        } catch (IOException ioe) {
            System.out.println("IOException: " + ioe);
        }                                                            }        
}
0
 

Expert Comment

by:pedxing
ID: 1220161
I'm glad you were able to get your code working.  I should caution you that setRequestProperty() will not work on all machines if they are running 1.02, but perhaps you are on an Intranet (or developing a program for the future) so it doesn't matter.

If you are planning to put your code up on a web server on the Internet soon, though, I suggest you look for alternatives to URLConnection that are less buggy.  For your reference, here is the bug report and its location:
From http://www.javasoft.com/products/jdk/1.0.2/KnownBugs.html#Net

URLConnection - There is a bug in JDK1.0.2 where the methods on URLConnection did not work:

          setRequestProperty()
          getRequestProperty()
          getHeaderField()
          getHeaderFieldInt()
          getContentLength()
          getDate()
          etc..

          These are fixed in JDK1.1.

0
 

Author Comment

by:rhee
ID: 1220162
Thank ina, "jpk", and "pedxing".
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Suggested Solutions

For customizing the look of your lightweight component and making it look lucid like it was made of glass. Or: how to make your component more Apple-ish ;) This tip assumes your component to be of rectangular shape and completely opaque. (COD…
Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
Viewers learn how to read error messages and identify possible mistakes that could cause hours of frustration. Coding is as much about debugging your code as it is about writing it. Define Error Message: Line Numbers: Type of Error: Break Down…
Viewers will learn how to properly install Eclipse with the necessary JDK, and will take a look at an introductory Java program. Download Eclipse installation zip file: Extract files from zip file: Download and install JDK 8: Open Eclipse and …

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now