URLConnection.getLastModified()

well this is bad news indeed.  Let me describe something and perhaps you might have an alternate viewpoint for a different way to go about it.

I'm using the Google api to perform a search....the results are in the neighborhood of 13,000 and I log each url to a database.  This is a news retrieval system for the U.N. and I really don't want to pull down all 13,000 entries each day just to find out if they're different or not...so I was hoping for the last modified date to be useful to me, but it seems that it won't be.

Is there another idea that perhaps is better than what i've got?

Thanks,
David

If the header field Last-Modified in the url does not exists, 0 will be returned. This does not matter whether the page is static or not.

lomidien, you may filter out some URLs by using the following method.

Here is an example:

Calender cal = Calendar.getInstance();
c.add(Calendar.DATE, -1);
URL test = new URL("http://www.hotmail.com");
HttpURLConnection conn = (HttpURLConnection)test.openConnection();

if (conn.getResponseCode() == 304)
   // not modified
else
  // do whatever you want

>>This does not matter whether the page is static or not

That's true in an absolute way but it's generally more likely not to have it if the response is dynamic

>>Calender cal = Calendar.getInstance();

is not used in your code example btw.

lomidien - if you're implementing this in your program, you should have a sound knowledge of the protocol:

http://www.w3.org/Protocols/rfc2616/rfc2616.html

Sorry, typo:

Calender cal = Calendar.getInstance();
cal.add(Calendar.DATE, -1);
UR test = new URL("http://www.hotmail.com");
HttpURLConnection conn = (HttpURLConnection)test.openConnection();
conn.setIfModifiedSince(cal.getTimeInMillis());

Here's a gentle introduction to these matters:

http://www.edginet.org/techie/website/http.html

:-)