Solved

How to download multiple files from web server to local folder and access ?

Posted on 2006-11-17
12
1,046 Views
Last Modified: 2010-04-04
Gidday team !

I need my delphi app to download files from one folder on my server.  I can write the code for a single file download but I need to download all files in this server folder into a local folder on the users pc.  For eg:

'http://somewebsite.com/files/'         // this would be my web server
'c:\application\files'                         //  this is the local folder

Now my application needs to check the webserver folder (say every 5 minutes) for the file contents and compare them with the local file contents.  If any file is missing on the server (but resides in the local folder) then application is to delete it from local folder.
At the same time, if any addition file is found on server but not on local machine then it should download it.

This application is a console app that uses an advertising clicker on the bottom of the form.  Ads willd rotate and change every minute.  The above information will help me to regularly check for new adverts to download and/or adds that have been removed.

Lastly, the downloaded files are '.jpg' files that my app will load into an image.  Don't know how to load gif files(moving) yet so I will stick to still adds.  Can this be done without downloading them ?  
Thanks in advance

Elvin
0
Comment
Question by:Mark Brady
  • 7
  • 5
12 Comments
 
LVL 28

Expert Comment

by:ciuly
ID: 17971605
well... no. one way or another they will have to be downloaded.
but: I suggest using a twebbrowser for displaying the adds. and this way you will be able to display gif, flash etc as well.
regarding the directrory download: if the http://somewebsite.com/files/ is not displaying a list of filles but some index.html with some otehr stuff on it, it will be almost impossible to do if you don't have a ftp access to that directory.
0
 
LVL 20

Author Comment

by:Mark Brady
ID: 17972981
Actually I had a play with it and was able to DL some files from a directory by adding a txt file with the actual file URL's to the same folder on my server.  The program downloads the text file and reads into a comboboxx then should run through and download each image listed in the combobox.  Slight problem here.  The text file comes down great and loads into combo.  The first file downloads ok then it crashes with an EInOutError with message 'I/O error 1784'.
Here is my full code so far.  Can you see what is causing this ?


unit Address;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, ExtCtrls,  Menus, jpeg, Wininet;

type
  TForm1 = class(TForm)
    Panel1: TPanel;
    Edit1: TEdit;
    Label1: TLabel;
    Panel2: TPanel;
    MainMenu1: TMainMenu;
    File1: TMenuItem;
    Exit1: TMenuItem;
    Edit2: TMenuItem;
    AddContact1: TMenuItem;
    Help1: TMenuItem;
    About1: TMenuItem;
    Label2: TLabel;
    Image2: TImage;
    Image1: TImage;
    ComboBox1: TComboBox;
    ComboBox2: TComboBox;
    Button1: TButton;
    procedure Exit1Click(Sender: TObject);
    procedure getadds(Sender: TObject);

    private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

function GetInetFile
(const fileURL, FileName: String): boolean;
const BufferSize = 1024;
var
  hSession, hURL: HInternet;
  Buffer: array[1..BufferSize] of Byte;
  BufferLen: DWORD;
  f: File;
  sAppName: string;
begin
 Result:=False;
 sAppName := ExtractFileName(Application.ExeName);
 hSession := InternetOpen(PChar(sAppName),
                INTERNET_OPEN_TYPE_PRECONFIG,
               nil, nil, 0);
 try
  hURL := InternetOpenURL(hSession,
            PChar(fileURL),
            nil,0,0,0);
  try
   AssignFile(f, FileName);
   Rewrite(f,1);
   repeat
    InternetReadFile(hURL, @Buffer,
                     SizeOf(Buffer), BufferLen);
    BlockWrite(f, Buffer, BufferLen);                   // -------------Crashes on this line-----------//
    FlashWindow(Application.Handle,True);
   until BufferLen = 0;
   CloseFile(f);
   Result:=True;
  finally
   InternetCloseHandle(hURL)
  end
 finally
  InternetCloseHandle(hSession)
 end
end;


procedure TForm1.Exit1Click(Sender: TObject);
begin
close;
end;


procedure TForm1.getadds(Sender: TObject);
var FileOnNet, LocalFileName, s: string;
i: integer;
begin
 FileOnNet:=
  'http://www.lfoband.com/adds/adds.txt';
 LocalFileName:='c:\adds\adds.txt';
 if GetInetFile(FileOnNet,LocalFileName)=True then
 combobox1.Items.LoadFromFile(localfilename)
 else
 begin
 ShowMessage('Error in file download');
 exit;
 end;
 combobox1.itemindex := -1;
 for i:= 0 to combobox1.items.count-1 do
 begin
 combobox1.itemindex := combobox1.itemindex + 1;
 FileOnNet:= combobox1.items[combobox1.itemindex];
 LocalFileName:= 'c:\adds\' + (extractfilename(combobox1.items[combobox1.itemindex])) ;

 if GetInetFile(FileOnNet,LocalFileName)=true then
 showmessage('Sucess !')
 else
 ShowMessage('Error in file download');
 end;
end;


end.
0
 
LVL 20

Author Comment

by:Mark Brady
ID: 17973012
One more thing.  When I say the first file downloads ok (meaning the first add which is a jpg) it is visible on my hard drive but is 0 bytes so it is not being written properly.  The txt file is being written propery.  
Here is the contents of the txt file.......


http:\\www.lfoband.com\adds\banner.jpg
http:\\www.lfoband.com\adds\lfo.jpg
http:\\www.lfoband.com\adds\addcf.jpg
0
 
LVL 28

Accepted Solution

by:
ciuly earned 500 total points
ID: 17974056
>Actually I had a play with it and was able to DL some files from a directory by adding a txt file with the actual file URL's to the same folder on my server.
-nice idea, BUT you will have to always update the file ;) but it's a good workaround.

about the error. I did sme modifications to the code and the error is because the hURL is nil. why? because you get an error:
12006 The URL scheme could not be recognized, or is not supported.

why? because it's %^$& microsoft. seriously now, I can't understand why a valid http url is not recognized by the IE engine. beats me.

so use indy instead. it's simpler anyway :)

here is the complete code with your code, modified to spit out some information and the indy way:

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, IdBaseComponent, IdAntiFreezeBase, IdAntiFreeze;

type
  TForm1 = class(TForm)
    Button1: TButton;
    ComboBox1: TComboBox;
    IdAntiFreeze1: TIdAntiFreeze;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
    procedure getadds(Sender: TObject);
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

uses Wininet, idHttp;

function SystemErrorMessage: string;
var
  P: PChar; e:cardinal;
begin
  e:=GetLastError;
  if FormatMessage(Format_Message_Allocate_Buffer OR Format_Message_From_System OR FORMAT_MESSAGE_FROM_HMODULE,
                   pointer(GetModuleHandle('wininet.dll')),
                   e,
                   0,
                   @P,
                   0,
                   nil) <> 0 then
  begin
    Result := inttostr(e)+': '+P;
    LocalFree(Integer(P))
  end
  else
    Result := '';
end;

function GetInetFile
(const fileURL, FileName: String): boolean;
const BufferSize = 1024;
var
  hSession, hURL: HInternet;
  Buffer: array[1..BufferSize] of Byte;
  BufferLen: DWORD;
  f: File;
  sAppName: string;
begin
 Result:=False;
 sAppName := ExtractFileName(Application.ExeName);
 hSession := InternetOpen(PChar(sAppName),
                INTERNET_OPEN_TYPE_PRECONFIG,
               nil, nil, 0);
 if hSession=nil then exit;
 try
  hURL := InternetOpenURL(hSession,
            PChar(fileURL),
            nil,0,0,0);
  if hURL=nil then
  begin
    showmessage(SystemErrorMessage);
    raise exception.Create('hurl is null. cleanup.');
  end;
  try
   AssignFile(f, FileName);
   Rewrite(f,1);
   try
     repeat
       BufferLen:=SizeOf(Buffer);
       InternetReadFile(hURL, @Buffer,
                       SizeOf(Buffer), BufferLen);
      if BufferLen>0 then
        BlockWrite(f, Buffer, BufferLen);                   // -------------Crashes on this line-----------//
      FlashWindow(Application.Handle,True);
     until BufferLen = 0;
     Result:=True;
   except
   end;
   CloseFile(f);
  finally
   InternetCloseHandle(hURL)
  end
 finally
  InternetCloseHandle(hSession)
 end
end;

function getIndyFile(form:TForm; const fileURL, FileName: String): boolean;
var http:TIdHttp;
    s:TFileStream;
    i:integer;
begin
  i:=0;
  while (i<form.ComponentCount) and (form.Components[i].ClassName<>'TIdAntiFreeze') do
    inc(i);
  if i=form.ComponentCount then
    raise exception.Create('You should put an IdAntiFreeze component on your form, otherwise your UI might freeze. Download halted.');
  http:=TIdHTTP.Create(form);
  try
    s:=TFileStream.Create(filename, fmcreate);
    try
      http.Get(fileUrl,s);
      result:=true;
    finally
      s.free;
    end;
  finally
    http.Free;
  end;
end;

procedure TForm1.getadds(Sender: TObject);
var FileOnNet, LocalFileName{ s}: string;
i: integer;
begin
 FileOnNet:='http://www.lfoband.com/adds/adds.txt';
 LocalFileName:=extractfilepath(application.exename)+'adds\adds.txt';
 if {GetInetFile({}getIndyFile(self,FileOnNet,LocalFileName)=True then
   combobox1.Items.LoadFromFile(localfilename)
 else
 begin
   ShowMessage('Error in file download');
   exit;
 end;
 combobox1.itemindex := -1;
 for i:= 0 to combobox1.items.count-1 do
 begin
   combobox1.itemindex := combobox1.itemindex + 1;
   FileOnNet:= combobox1.items[combobox1.itemindex];
   LocalFileName:= extractfilepath(application.exename) +'adds\'+ (extractfilename(combobox1.items[combobox1.itemindex])) ;

   if {GetInetFile({}getIndyFile(self,FileOnNet,LocalFileName)=true then
     showmessage('Sucess !')
   else
     ShowMessage('Error in file download');
 end;
end;

procedure TForm1.Button1Click(Sender: TObject);
begin
  getadds(sender);
end;

end.
0
 
LVL 20

Author Comment

by:Mark Brady
ID: 17974075
Hey thanks ciuly I'll try this code when I wake up !  I can't believe that Microcrap has so many limitations.  Thanks for your help and I'll let you know how it goes.

Elvin
0
 
LVL 20

Author Comment

by:Mark Brady
ID: 17975281
Where do I get IdBaseComponent, IdAntiFreezeBase and  IdAntiFreeze from ?
I'm compiling this under delphi 5 but do have delphi studio.

Elvin
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 28

Expert Comment

by:ciuly
ID: 17975306
0
 
LVL 20

Author Comment

by:Mark Brady
ID: 17977534
I tried that but am still having a freeze problem.  Here is my form.  It takes forever to download the first .jpg.file even though it's only small.  The DL file is useless ( not openable).  I tried increasing the antifreeze timeout to 1000 but it just hangs.


object Form1: TForm1
  Left = 0
  Top = 0
  Width = 362
  Height = 500
  Caption = 'Form1'
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'Tahoma'
  Font.Style = []
  OldCreateOrder = False
  PixelsPerInch = 96
  TextHeight = 13
  object Button1: TButton
    Left = 128
    Top = 24
    Width = 75
    Height = 25
    Caption = 'Button1'
    TabOrder = 0
    OnClick = Button1Click
  end
  object ComboBox1: TComboBox
    Left = 96
    Top = 72
    Width = 145
    Height = 21
    ItemHeight = 13
    TabOrder = 1
    Text = 'ComboBox1'
  end
  object IdAntiFreeze1: TIdAntiFreeze
    Left = 320
  end
end




//  Any thoughts on this or should I go to a webbrowser like you suggested ?

Elvin
0
 
LVL 28

Expert Comment

by:ciuly
ID: 17977564
I never suggested a webbrowser.
and you got it wrong: the time should be decreased not increased. I worst cases I used 50 ms and changed the onlyWhenIdle to false.

this is the dfm for the unit posted by me earlier:

object Form1: TForm1
  Left = 192
  Top = 114
  Width = 870
  Height = 640
  Caption = 'Form1'
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'MS Sans Serif'
  Font.Style = []
  OldCreateOrder = False
  PixelsPerInch = 96
  TextHeight = 13
  object Button1: TButton
    Left = 240
    Top = 96
    Width = 75
    Height = 25
    Caption = 'Button1'
    TabOrder = 0
    OnClick = Button1Click
  end
  object ComboBox1: TComboBox
    Left = 376
    Top = 96
    Width = 145
    Height = 21
    ItemHeight = 13
    TabOrder = 1
    Text = 'ComboBox1'
  end
  object IdAntiFreeze1: TIdAntiFreeze
    Left = 144
    Top = 176
  end
end

just for safety, here is the project as well: http://www.ciuly.com/delphi/DownloadMultipleFiles.zip

but since you are reffering to a hang, I am now considering your machine setup: are you using a proxy to access the internet? if so, then modify:

  http:=TIdHTTP.Create(form);
to
  http:=TIdHTTP.Create(form);
  http.ProxyParams.ProxyPassword:=<your proxy password if any>;
  http.ProxyParams.ProxyPort:=<the proxy server port>;
  http.ProxyParams.ProxyServer:=<the proxy server address>;
  http.ProxyParams.ProxyUsername:=<your proxy username if any>;

to automatically detect IE proxy settings see this: http://www.mydesktophelp.com/tips/ie12.php
0
 
LVL 20

Author Comment

by:Mark Brady
ID: 17977867
Just to clarify the Webbrowser...... //I suggest using a twebbrowser for displaying the adds//   that's where I got that from.  Anyhow I'll try the latest and post back on the results.  Thanks dude !

Elvin
0
 
LVL 28

Expert Comment

by:ciuly
ID: 17978280
oh. that I was reffering just to display, not download. but of course you can also use it for download but then you won't have the issue of download multiple files ;) The reason I suggested webbrowser for displaying is that you will be able to display mostly any type of banner the browser supports.
but in case of using webbrowser then you might think of leaving the multiple download issue out since you will only work with URLs directly (IE caches the files anyway so this will not put an issue in the case of files continuosly being downloaded ;) ) it's a matter of choiuse anyway.
you can still download all files with indy and then load the local file in the webbrowser: works as good.
0
 
LVL 20

Author Comment

by:Mark Brady
ID: 17985855
Thanks Ciurly !

I think I will use the TWebBrowser.  I just used the code to download the text file and loadup the combobox with the urls of the jpg files and have it rotate on a timer.  As far as the firewall issue is concerned, I aren't behind any firewall so I'm not sure on why it is freezing up even when I use IdAntiFreeze component.  I will play with it some more because I would still like to get multiple file downloads working for future reference.  For now, I accept your answer.

Thanks again buddy.

Elvin
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Introduction The parallel port is a very commonly known port, it was widely used to connect a printer to the PC, if you look at the back of your computer, for those who don't have newer computers, there will be a port with 25 pins and a small print…
Introduction Raise your hands if you were as upset with FireMonkey as I was when I discovered that there was no TListview.  I use TListView in almost all of my applications I've written, and I was not going to compromise by resorting to TStringGrid…
Access reports are powerful and flexible. Learn how to create a query and then a grouped report using the wizard. Modify the report design after the wizard is done to make it look better. There will be another video to explain how to put the final p…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now