Solved

Downloading images via HTTP only Last Modified

Posted on 2007-11-23
16
225 Views
Last Modified: 2010-04-21
Hi experts,

I need to download lots of images from a server on a daily basis via HTTP.

I need to only download the images Last Modified on a certain date.

Basically only the new ones.

Is there any way to do this please?

Thanks

ST3VO
0
Comment
Question by:ST3VO
  • 7
  • 4
  • 3
  • +1
16 Comments
 
LVL 16

Expert Comment

by:CodedK
ID: 20338688
Do you use a database ? MySQL ?
If so then are those images in a blob field..or do you keep the filenames?
You could query the database if this is the case and get the filenames ...then download.
0
 

Author Comment

by:ST3VO
ID: 20338738
If I could have access to that it would be perfect but all I get in a list of URL's :o/
0
 
LVL 16

Expert Comment

by:CodedK
ID: 20338749
Via HTTP you cant get the creation date of the file/picture.
And you cant rely on the filename.
I cant think a way to make this possible.
Wait for another expert input. :D
0
ScreenConnect 6.0 Free Trial

At ScreenConnect, partner feedback doesn't fall on deaf ears. We collected partner suggestions off of their virtual wish list and transformed them into one game-changing release: ScreenConnect 6.0. Explore all of the extras and enhancements for yourself!

 

Author Comment

by:ST3VO
ID: 20338799
OK..Thanks!  :o)
0
 
LVL 28

Expert Comment

by:2266180
ID: 20339971
well, making it possible depends solely on the server. does the serevr provide correct last modified header? if so, then you can rely on that. otherwise, you're kind of lost.

basically, what you need to do is mimic a cache. that's all.
so you need to get familiar with the http protocol. caching part here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
as you will see, the modified date is not the only thing to use for caching. there is an etag header and some others if my memory helps me correctly.

you will have to go over the entire http protocol and get the necessary info out and implement that specific behaviour.

OR, if you don't need an application made for this, use an existing cache, like squid (which is a caching proxy).
so a simple way would be to install a caching proxy and get the images through that proxy. if the image was not changed, you will still get it, but from local disk, so it will be very fast and not consuming your internet connection.

anything above this, will have to follow http protocol so ... unless you have a great deal of time on your hands, I say go with squid or another local caching http proxy :)
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 20340145
Using Indy's TidHTTP control, you can use the HEAD command and get just the headers and check the LastModified.

idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(idhttp1.Response.RawHeaders);

0
 

Author Comment

by:ST3VO
ID: 20340154
My problem is that I have not access to that server. I only get a list of URL's with the links to the images from them.

Having this in mind...can I still work with your suggestions above having just link access to that server?

0
 

Author Comment

by:ST3VO
ID: 20340177
Hmmm intesting...EddieShipman!

So, i theory I could filter the headers and that the data and time of the last modified stamp of the image and if the local one is older then download it...right?

0
 

Author Comment

by:ST3VO
ID: 20340182
Is there anyway of just retrieving the Data and time or do I have to filter it?
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 20340218
Get just the last modified like this:
idhttp1.Response.RawHeaders.Values['Last-Modified']

Remember, though that that "timestamp" may have many different formats based on the server where the
images reside. You will need to figure out, from the value returned, what the actual timestamp is.

There is no "standard" way to format that info. Also, note that some servers may not return that value. It is
up to the server admin to set the headers that will be retruend. Oh, and another thing, script generated
images, such as if you return them from PHP's GDI calls, will not have that header.
0
 

Author Comment

by:ST3VO
ID: 20340321
Could you please give me a same code please as I cannot seem to get the Last Modified in the memo with:

idhttp1.Response.RawHeaders.Values['Last-Modified']

What I'm I doing wrong please?



Idhttp1.Response.RawHeaders.Values['Last-Modified'];
Idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(Idhttp1.Response.RawHeaders);

Open in new window

0
 
LVL 26

Accepted Solution

by:
EddieShipman earned 500 total points
ID: 20340350

Idhttp1.Head('http://www.experts-exchange.com/expert/images/logo.jpg');
memo1.Lines.Assign(Idhttp1.Response.RawHeaders);
ShowMEssage(Idhttp1.Response.RawHeaders.Values['Last-Modified']);

Open in new window

0
 
LVL 28

Expert Comment

by:2266180
ID: 20340366
and as I said, keep in mind that last-modified is not the only way to track if a document has changed. there is at least the etag header and maybe some more.
that's why I suggested a caching proxy. it's easier then going over the http protocol.
0
 

Author Closing Comment

by:ST3VO
ID: 31410704
Great!!! Thanks a million...I'll play around with it and see what I can come up with!!!
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 20349662
caching proxy requires downloading the file whereas the HEAD doesn't
0
 
LVL 28

Expert Comment

by:2266180
ID: 20349678
duuh.

look at the question. he WANTS to download stuff, jsut that he doesn't want to doens;d stuff more than once.
so where is the problem? the caching proxy will not download soemthing twice if it was not modified. that's the idea with the "caching".
0

Featured Post

Are your AD admin tools letting you down?

Managing Active Directory can get complicated.  Often, the native tools for managing AD are just not up to the task.  The largest Active Directory installations in the world have relied on one tool to manage their day-to-day administration tasks: Hyena. Start your trial today.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Need Help Delphi 2010 CheckBox1 Stored value in memo 13 66
CheckListBox usage 3 58
Wincontrol not (correctly) drawn 15 38
Firemonkey BASS_Init into a thread 17 4
A lot of questions regard threads in Delphi.   One of the more specific questions is how to show progress of the thread.   Updating a progressbar from inside a thread is a mistake. A solution to this would be to send a synchronized message to the…
Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
In a recent question (https://www.experts-exchange.com/questions/28997919/Pagination-in-Adobe-Acrobat.html) here at Experts Exchange, a member asked how to add page numbers to a PDF file using Adobe Acrobat XI Pro. This short video Micro Tutorial sh…
Finds all prime numbers in a range requested and places them in a public primes() array. I've demostrated a template size of 30 (2 * 3 * 5) but larger templates can be built such 210  (2 * 3 * 5 * 7) or 2310  (2 * 3 * 5 * 7 * 11). The larger templa…

773 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question