• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 336
  • Last Modified:

Challenge: getting directory listing using URLConnection

Hi there!

Some http-servers allow clients to get the directory listing of a specific directory if there's no default html page. The question is easy: how can I get that listing using Java?

It's quite easy getting a page, below you see the code needed for doing that. Just as soon as I try to use a URL which is a directory I get an iContentsize of -1, which is not much, is it?

Below is the code I use in a slightly moderated form. The parts of interest are between "start-" and "end important part". The authorization part is borrowed from Q.10009989 ("Retrieval of URL with Authorization") as is the HTUU-class. Of course, this is not needed for most sites.

Who can/want to help me with this challenge? Maybe it's quite easy, but then you're just lucky getting the points. Any answer which really gets me such a listing will get the credits.

PS: I use SilverStream, in case you were wondering about the AgoInvokedEvent. The method uses "evt.setResult()" for setting data for the calling thread.

Greetz and thanks in advance,
Abel



public void invoked(AgoInvokedEvent evt) throws Exception
{
    Hashtable htResultData = new Hashtable();
    try
    {
        //////////////the important part/////////////////////
        URLConnection ucToGet = (new URL("http://host/path")).openConnection();
        ucToGet.setRequestProperty("Authorization", "Basic " + HTUU.encode("user:password"));
        InputStream is = ucToGet.getInputStream();
        int iContentSize = ucToGet.getContentLength();

        if(iContentSize < 1)         //Can't do anything in this case: a directory
            throw new Exception("Contentlength is smaller then one, can't read content! (Directorylisting)");

        ///////////////end important part////////////////////
       
        byte bData[] = new byte[iContentSize];
        int b = 0, x = 0;
        while ((b = is.read()) != -1 && (x < iContentSize))
        {
                bData[x++]=(byte) b;
        }

        htResultData.put("size", new Integer(iContentSize));
        htResultData.put("data", bData);

        htResultData.put("result", SUCCEED);
        if(DEBUG) System.out.println("SUCCEED gezet");
    }

    catch(Exception __E)
    {
        if(__E instanceof FileNotFoundException)
            htResultData.put("description", "The URL has not been found on the server, " +
                                "couldn't retrieve the URL, please try again with another URL.\n\n" +
                                "Some other reason for this error maybe a malformed loginname/password combination.");
        else
            htResultData.put("description", "");    //means caller should use "error"

        htResultData.put("result", ERROR);
        htResultData.put("error", __E);            //Store error for retrieval for the caller

    }
    finally
    {
        evt.setResult(htResultData);
        return;
    }
}
0
abel
Asked:
abel
  • 2
  • 2
1 Solution
 
heyhey_Commented:
if you can see the directory listing inside your browser, you should be able to get it with your Java app.
conetentLength = -1 means that server does not know the resource length, so you should read from the inputstream, until EOF.

InputStream is = ucToGet.getInputStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream (5000);
byte[] chunk = new byte[5000];

int len = is.read(chunk);
while (len >0)
{
   os.write(chunk, 0, len);
   len = is.read(chunk);
}
os.flush();

byte[] data = bos.toByteArray();
0
 
abelAuthor Commented:
AND IT WORKS INSTANTLY!!

Thanks!

Your solution sounds so logical and is so easy to comprehend and apply that you definitely deserve the points, although you posted as a comment. Do you want to post again as an answer? Then I can grade you well :)

It's probably due to my short time in Java-world that I need to learn these quite logical things.

Once again, thanks for this quick reply!

greetz,
Abel
0
 
heyhey_Commented:
Answer ! :)

note that web-servers usually return content-length = -1 for all the files (so clients usually read until EOF). Content-length is used mainly when client sends HTTP request, because server NEEDS to know how many bytes it must read before starting to generate the answer.
0
 
abelAuthor Commented:
Thanks for the tip :)

Bye!
Abel
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Amazon Web Services - Basic

Are you thinking about creating an Amazon Web Services account for your business? Not sure where to start? In this course you’ll get an overview of the history of AWS and take a tour of their user interface.

  • 2
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now