Challenge: getting directory listing using URLConnection

Hi there!

Some http-servers allow clients to get the directory listing of a specific directory if there's no default html page. The question is easy: how can I get that listing using Java?

It's quite easy getting a page, below you see the code needed for doing that. Just as soon as I try to use a URL which is a directory I get an iContentsize of -1, which is not much, is it?

Below is the code I use in a slightly moderated form. The parts of interest are between "start-" and "end important part". The authorization part is borrowed from Q.10009989 ("Retrieval of URL with Authorization") as is the HTUU-class. Of course, this is not needed for most sites.

Who can/want to help me with this challenge? Maybe it's quite easy, but then you're just lucky getting the points. Any answer which really gets me such a listing will get the credits.

PS: I use SilverStream, in case you were wondering about the AgoInvokedEvent. The method uses "evt.setResult()" for setting data for the calling thread.

Greetz and thanks in advance,
Abel



public void invoked(AgoInvokedEvent evt) throws Exception
{
    Hashtable htResultData = new Hashtable();
    try
    {
        //////////////the important part/////////////////////
        URLConnection ucToGet = (new URL("http://host/path")).openConnection();
        ucToGet.setRequestProperty("Authorization", "Basic " + HTUU.encode("user:password"));
        InputStream is = ucToGet.getInputStream();
        int iContentSize = ucToGet.getContentLength();

        if(iContentSize < 1)         //Can't do anything in this case: a directory
            throw new Exception("Contentlength is smaller then one, can't read content! (Directorylisting)");

        ///////////////end important part////////////////////
       
        byte bData[] = new byte[iContentSize];
        int b = 0, x = 0;
        while ((b = is.read()) != -1 && (x < iContentSize))
        {
                bData[x++]=(byte) b;
        }

        htResultData.put("size", new Integer(iContentSize));
        htResultData.put("data", bData);

        htResultData.put("result", SUCCEED);
        if(DEBUG) System.out.println("SUCCEED gezet");
    }

    catch(Exception __E)
    {
        if(__E instanceof FileNotFoundException)
            htResultData.put("description", "The URL has not been found on the server, " +
                                "couldn't retrieve the URL, please try again with another URL.\n\n" +
                                "Some other reason for this error maybe a malformed loginname/password combination.");
        else
            htResultData.put("description", "");    //means caller should use "error"

        htResultData.put("result", ERROR);
        htResultData.put("error", __E);            //Store error for retrieval for the caller

    }
    finally
    {
        evt.setResult(htResultData);
        return;
    }
}
LVL 39
abelAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

heyhey_Commented:
if you can see the directory listing inside your browser, you should be able to get it with your Java app.
conetentLength = -1 means that server does not know the resource length, so you should read from the inputstream, until EOF.

InputStream is = ucToGet.getInputStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream (5000);
byte[] chunk = new byte[5000];

int len = is.read(chunk);
while (len >0)
{
   os.write(chunk, 0, len);
   len = is.read(chunk);
}
os.flush();

byte[] data = bos.toByteArray();
0
abelAuthor Commented:
AND IT WORKS INSTANTLY!!

Thanks!

Your solution sounds so logical and is so easy to comprehend and apply that you definitely deserve the points, although you posted as a comment. Do you want to post again as an answer? Then I can grade you well :)

It's probably due to my short time in Java-world that I need to learn these quite logical things.

Once again, thanks for this quick reply!

greetz,
Abel
0
heyhey_Commented:
Answer ! :)

note that web-servers usually return content-length = -1 for all the files (so clients usually read until EOF). Content-length is used mainly when client sends HTTP request, because server NEEDS to know how many bytes it must read before starting to generate the answer.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
abelAuthor Commented:
Thanks for the tip :)

Bye!
Abel
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.