Solved

Downloading images via HTTP only Last Modified

Posted on 2007-11-23
16
223 Views
Last Modified: 2010-04-21
Hi experts,

I need to download lots of images from a server on a daily basis via HTTP.

I need to only download the images Last Modified on a certain date.

Basically only the new ones.

Is there any way to do this please?

Thanks

ST3VO
0
Comment
Question by:ST3VO
  • 7
  • 4
  • 3
  • +1
16 Comments
 
LVL 16

Expert Comment

by:CodedK
ID: 20338688
Do you use a database ? MySQL ?
If so then are those images in a blob field..or do you keep the filenames?
You could query the database if this is the case and get the filenames ...then download.
0
 

Author Comment

by:ST3VO
ID: 20338738
If I could have access to that it would be perfect but all I get in a list of URL's :o/
0
 
LVL 16

Expert Comment

by:CodedK
ID: 20338749
Via HTTP you cant get the creation date of the file/picture.
And you cant rely on the filename.
I cant think a way to make this possible.
Wait for another expert input. :D
0
 

Author Comment

by:ST3VO
ID: 20338799
OK..Thanks!  :o)
0
 
LVL 28

Expert Comment

by:ciuly
ID: 20339971
well, making it possible depends solely on the server. does the serevr provide correct last modified header? if so, then you can rely on that. otherwise, you're kind of lost.

basically, what you need to do is mimic a cache. that's all.
so you need to get familiar with the http protocol. caching part here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
as you will see, the modified date is not the only thing to use for caching. there is an etag header and some others if my memory helps me correctly.

you will have to go over the entire http protocol and get the necessary info out and implement that specific behaviour.

OR, if you don't need an application made for this, use an existing cache, like squid (which is a caching proxy).
so a simple way would be to install a caching proxy and get the images through that proxy. if the image was not changed, you will still get it, but from local disk, so it will be very fast and not consuming your internet connection.

anything above this, will have to follow http protocol so ... unless you have a great deal of time on your hands, I say go with squid or another local caching http proxy :)
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 20340145
Using Indy's TidHTTP control, you can use the HEAD command and get just the headers and check the LastModified.

idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(idhttp1.Response.RawHeaders);

0
 

Author Comment

by:ST3VO
ID: 20340154
My problem is that I have not access to that server. I only get a list of URL's with the links to the images from them.

Having this in mind...can I still work with your suggestions above having just link access to that server?

0
 

Author Comment

by:ST3VO
ID: 20340177
Hmmm intesting...EddieShipman!

So, i theory I could filter the headers and that the data and time of the last modified stamp of the image and if the local one is older then download it...right?

0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 

Author Comment

by:ST3VO
ID: 20340182
Is there anyway of just retrieving the Data and time or do I have to filter it?
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 20340218
Get just the last modified like this:
idhttp1.Response.RawHeaders.Values['Last-Modified']

Remember, though that that "timestamp" may have many different formats based on the server where the
images reside. You will need to figure out, from the value returned, what the actual timestamp is.

There is no "standard" way to format that info. Also, note that some servers may not return that value. It is
up to the server admin to set the headers that will be retruend. Oh, and another thing, script generated
images, such as if you return them from PHP's GDI calls, will not have that header.
0
 

Author Comment

by:ST3VO
ID: 20340321
Could you please give me a same code please as I cannot seem to get the Last Modified in the memo with:

idhttp1.Response.RawHeaders.Values['Last-Modified']

What I'm I doing wrong please?



Idhttp1.Response.RawHeaders.Values['Last-Modified'];

Idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');

memo1.Lines.Assign(Idhttp1.Response.RawHeaders);

Open in new window

0
 
LVL 26

Accepted Solution

by:
EddieShipman earned 500 total points
ID: 20340350

Idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');

memo1.Lines.Assign(Idhttp1.Response.RawHeaders);

ShowMEssage(Idhttp1.Response.RawHeaders.Values['Last-Modified']);

Open in new window

0
 
LVL 28

Expert Comment

by:ciuly
ID: 20340366
and as I said, keep in mind that last-modified is not the only way to track if a document has changed. there is at least the etag header and maybe some more.
that's why I suggested a caching proxy. it's easier then going over the http protocol.
0
 

Author Closing Comment

by:ST3VO
ID: 31410704
Great!!! Thanks a million...I'll play around with it and see what I can come up with!!!
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 20349662
caching proxy requires downloading the file whereas the HEAD doesn't
0
 
LVL 28

Expert Comment

by:ciuly
ID: 20349678
duuh.

look at the question. he WANTS to download stuff, jsut that he doesn't want to doens;d stuff more than once.
so where is the problem? the caching proxy will not download soemthing twice if it was not modified. that's the idea with the "caching".
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

This article explains how to create forms/units independent of other forms/units object names in a delphi project. Have you ever created a form for user input in a Delphi project and then had the need to have that same form in a other Delphi proj…
Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now