Improve company productivity with a Business Account.Sign Up

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 237
  • Last Modified:

Downloading images via HTTP only Last Modified

Hi experts,

I need to download lots of images from a server on a daily basis via HTTP.

I need to only download the images Last Modified on a certain date.

Basically only the new ones.

Is there any way to do this please?

Thanks

ST3VO
0
ST3VO
Asked:
ST3VO
  • 7
  • 4
  • 3
  • +1
1 Solution
 
CodedKCommented:
Do you use a database ? MySQL ?
If so then are those images in a blob field..or do you keep the filenames?
You could query the database if this is the case and get the filenames ...then download.
0
 
ST3VOAuthor Commented:
If I could have access to that it would be perfect but all I get in a list of URL's :o/
0
 
CodedKCommented:
Via HTTP you cant get the creation date of the file/picture.
And you cant rely on the filename.
I cant think a way to make this possible.
Wait for another expert input. :D
0
Upgrade your Question Security!

Your question, your audience. Choose who sees your identity—and your question—with question security.

 
ST3VOAuthor Commented:
OK..Thanks!  :o)
0
 
2266180Commented:
well, making it possible depends solely on the server. does the serevr provide correct last modified header? if so, then you can rely on that. otherwise, you're kind of lost.

basically, what you need to do is mimic a cache. that's all.
so you need to get familiar with the http protocol. caching part here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
as you will see, the modified date is not the only thing to use for caching. there is an etag header and some others if my memory helps me correctly.

you will have to go over the entire http protocol and get the necessary info out and implement that specific behaviour.

OR, if you don't need an application made for this, use an existing cache, like squid (which is a caching proxy).
so a simple way would be to install a caching proxy and get the images through that proxy. if the image was not changed, you will still get it, but from local disk, so it will be very fast and not consuming your internet connection.

anything above this, will have to follow http protocol so ... unless you have a great deal of time on your hands, I say go with squid or another local caching http proxy :)
0
 
Eddie ShipmanAll-around developerCommented:
Using Indy's TidHTTP control, you can use the HEAD command and get just the headers and check the LastModified.

idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(idhttp1.Response.RawHeaders);

0
 
ST3VOAuthor Commented:
My problem is that I have not access to that server. I only get a list of URL's with the links to the images from them.

Having this in mind...can I still work with your suggestions above having just link access to that server?

0
 
ST3VOAuthor Commented:
Hmmm intesting...EddieShipman!

So, i theory I could filter the headers and that the data and time of the last modified stamp of the image and if the local one is older then download it...right?

0
 
ST3VOAuthor Commented:
Is there anyway of just retrieving the Data and time or do I have to filter it?
0
 
Eddie ShipmanAll-around developerCommented:
Get just the last modified like this:
idhttp1.Response.RawHeaders.Values['Last-Modified']

Remember, though that that "timestamp" may have many different formats based on the server where the
images reside. You will need to figure out, from the value returned, what the actual timestamp is.

There is no "standard" way to format that info. Also, note that some servers may not return that value. It is
up to the server admin to set the headers that will be retruend. Oh, and another thing, script generated
images, such as if you return them from PHP's GDI calls, will not have that header.
0
 
ST3VOAuthor Commented:
Could you please give me a same code please as I cannot seem to get the Last Modified in the memo with:

idhttp1.Response.RawHeaders.Values['Last-Modified']

What I'm I doing wrong please?



Idhttp1.Response.RawHeaders.Values['Last-Modified'];
Idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(Idhttp1.Response.RawHeaders);

Open in new window

0
 
Eddie ShipmanAll-around developerCommented:

Idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(Idhttp1.Response.RawHeaders);
ShowMEssage(Idhttp1.Response.RawHeaders.Values['Last-Modified']);

Open in new window

0
 
2266180Commented:
and as I said, keep in mind that last-modified is not the only way to track if a document has changed. there is at least the etag header and maybe some more.
that's why I suggested a caching proxy. it's easier then going over the http protocol.
0
 
ST3VOAuthor Commented:
Great!!! Thanks a million...I'll play around with it and see what I can come up with!!!
0
 
Eddie ShipmanAll-around developerCommented:
caching proxy requires downloading the file whereas the HEAD doesn't
0
 
2266180Commented:
duuh.

look at the question. he WANTS to download stuff, jsut that he doesn't want to doens;d stuff more than once.
so where is the problem? the caching proxy will not download soemthing twice if it was not modified. that's the idea with the "caching".
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 7
  • 4
  • 3
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now