Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Downloading images via HTTP only Last Modified

Posted on 2007-11-23
16
Medium Priority
?
234 Views
Last Modified: 2010-04-21
Hi experts,

I need to download lots of images from a server on a daily basis via HTTP.

I need to only download the images Last Modified on a certain date.

Basically only the new ones.

Is there any way to do this please?

Thanks

ST3VO
0
Comment
Question by:ST3VO
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 4
  • 3
  • +1
16 Comments
 
LVL 16

Expert Comment

by:CodedK
ID: 20338688
Do you use a database ? MySQL ?
If so then are those images in a blob field..or do you keep the filenames?
You could query the database if this is the case and get the filenames ...then download.
0
 

Author Comment

by:ST3VO
ID: 20338738
If I could have access to that it would be perfect but all I get in a list of URL's :o/
0
 
LVL 16

Expert Comment

by:CodedK
ID: 20338749
Via HTTP you cant get the creation date of the file/picture.
And you cant rely on the filename.
I cant think a way to make this possible.
Wait for another expert input. :D
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:ST3VO
ID: 20338799
OK..Thanks!  :o)
0
 
LVL 28

Expert Comment

by:2266180
ID: 20339971
well, making it possible depends solely on the server. does the serevr provide correct last modified header? if so, then you can rely on that. otherwise, you're kind of lost.

basically, what you need to do is mimic a cache. that's all.
so you need to get familiar with the http protocol. caching part here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
as you will see, the modified date is not the only thing to use for caching. there is an etag header and some others if my memory helps me correctly.

you will have to go over the entire http protocol and get the necessary info out and implement that specific behaviour.

OR, if you don't need an application made for this, use an existing cache, like squid (which is a caching proxy).
so a simple way would be to install a caching proxy and get the images through that proxy. if the image was not changed, you will still get it, but from local disk, so it will be very fast and not consuming your internet connection.

anything above this, will have to follow http protocol so ... unless you have a great deal of time on your hands, I say go with squid or another local caching http proxy :)
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 20340145
Using Indy's TidHTTP control, you can use the HEAD command and get just the headers and check the LastModified.

idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(idhttp1.Response.RawHeaders);

0
 

Author Comment

by:ST3VO
ID: 20340154
My problem is that I have not access to that server. I only get a list of URL's with the links to the images from them.

Having this in mind...can I still work with your suggestions above having just link access to that server?

0
 

Author Comment

by:ST3VO
ID: 20340177
Hmmm intesting...EddieShipman!

So, i theory I could filter the headers and that the data and time of the last modified stamp of the image and if the local one is older then download it...right?

0
 

Author Comment

by:ST3VO
ID: 20340182
Is there anyway of just retrieving the Data and time or do I have to filter it?
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 20340218
Get just the last modified like this:
idhttp1.Response.RawHeaders.Values['Last-Modified']

Remember, though that that "timestamp" may have many different formats based on the server where the
images reside. You will need to figure out, from the value returned, what the actual timestamp is.

There is no "standard" way to format that info. Also, note that some servers may not return that value. It is
up to the server admin to set the headers that will be retruend. Oh, and another thing, script generated
images, such as if you return them from PHP's GDI calls, will not have that header.
0
 

Author Comment

by:ST3VO
ID: 20340321
Could you please give me a same code please as I cannot seem to get the Last Modified in the memo with:

idhttp1.Response.RawHeaders.Values['Last-Modified']

What I'm I doing wrong please?



Idhttp1.Response.RawHeaders.Values['Last-Modified'];
Idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(Idhttp1.Response.RawHeaders);

Open in new window

0
 
LVL 26

Accepted Solution

by:
EddieShipman earned 2000 total points
ID: 20340350

Idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(Idhttp1.Response.RawHeaders);
ShowMEssage(Idhttp1.Response.RawHeaders.Values['Last-Modified']);

Open in new window

0
 
LVL 28

Expert Comment

by:2266180
ID: 20340366
and as I said, keep in mind that last-modified is not the only way to track if a document has changed. there is at least the etag header and maybe some more.
that's why I suggested a caching proxy. it's easier then going over the http protocol.
0
 

Author Closing Comment

by:ST3VO
ID: 31410704
Great!!! Thanks a million...I'll play around with it and see what I can come up with!!!
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 20349662
caching proxy requires downloading the file whereas the HEAD doesn't
0
 
LVL 28

Expert Comment

by:2266180
ID: 20349678
duuh.

look at the question. he WANTS to download stuff, jsut that he doesn't want to doens;d stuff more than once.
so where is the problem? the caching proxy will not download soemthing twice if it was not modified. that's the idea with the "caching".
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A lot of questions regard threads in Delphi.   One of the more specific questions is how to show progress of the thread.   Updating a progressbar from inside a thread is a mistake. A solution to this would be to send a synchronized message to the…
Have you ever had your Delphi form/application just hanging while waiting for data to load? This is the article to read if you want to learn some things about adding threads for data loading in the background. First, I'll setup a general applica…
In this video, Percona Director of Solution Engineering Jon Tobin discusses the function and features of Percona Server for MongoDB. How Percona can help Percona can help you determine if Percona Server for MongoDB is the right solution for …
Please read the paragraph below before following the instructions in the video — there are important caveats in the paragraph that I did not mention in the video. If your PaperPort 12 or PaperPort 14 is failing to start, or crashing, or hanging, …
Suggested Courses

604 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question