asked on

Java Link Checker

Hi,looking to create a java link checker for urls, Want to return the URL if exists or not. If it exists, download its content, examine all the pages it links to, and display broken links. Want to put in place a check for this,

I am new to java , I know I have to use java.net and java.io packages, below is what code I have to work with, I would like to improve on it if possible...

import java.io.*;
import java.net.*;

public class JavaGetUrl {

   public static void main (String[] args) {

              URL u;
      InputStream is = null;
      DataInputStream dis;
      String s;

      try {

                 u = new URL("http://www.msn.com");

                 is = u.openStream();         // throws an IOException

        
         dis = new DataInputStream(new BufferedInputStream(is));

                  

         while ((s = dis.readLine()) != null) {
            System.out.println(s);
         }

      } catch (MalformedURLException mue) {

         System.out.println("Ouch - a MalformedURLException happened.");
         mue.printStackTrace();
         System.exit(1);

      } catch (IOException ioe) {

         System.out.println("Oops- an IOException happened.");
         ioe.printStackTrace();
         System.exit(1);

      } {

                 try {
            is.close();
         } catch (IOException ioe) {
                     }

      } 

   }  

}

Open in new window

Dave Baldwin

Here's a list of open source java web crawlers: http://java-source.net/open-source/crawlers. I'd look at those before I started writing my own.

Indie101

ASKER

Thanks for the advice, looking for any pointers in doing this (with code supplied or more) tbh

Dave Baldwin

That's why I gave you that link. Open Source means the code is available for you to read and use. Some may have a discussion about the code too. At least one was an 'exercise' in doing it.

ASKER CERTIFIED SOLUTION

Venabili

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Indie101

ASKER

Good answer, wasnt exactly what was looking for

Venabili

>Author Comments:
Good answer, wasnt exactly what was looking for

What were you looking for? Unfortunatelly my crystal ball is still stuck at an airport in Europe so I cannot guess if you do not specdify anything.

Anyway - good luck.

Indie101

ASKER

Well I posted code, and gave some pointers in general for what I wanted to do, An A would have been code examples rather than links to read, I appreciate your answer just in my noob java (I work in 3rd level support and doing Java by night) didnt think it was exactly what I was looking for, best of luck :-)

Venabili

The problem is that fixing your code will require reinventing the wheel and as a newbie Java developer, there is no real point doing it :)

No worries for the grade - but you might want to rethink the idea of doing all from scratch - I would usually start from an open source code (or one of the ones in my comment) and change it here and there if I need to :)

Anyway - good luck with your Java.