We help IT Professionals succeed at work.

HttpQueryInfo with WinInet

Richard2000
Richard2000 asked
on
Hi,

I am trying to use the function HttpQueryInfo in WinInet to determine the length of a file in bytes.  (The reason I need to do this is that I need to display a progress bar for the HTTP download and so need to know first the size of the file, so I can determine what percentage has been completed).  I need this data (file size in bytes) as an Integer.

The function looks like this...

BOOL HttpQueryInfo(
    HINTERNET hRequest,
    DWORD dwInfoLevel,
    LPVOID lpvBuffer,
    LPDWORD lpdwBufferLength,
    LPDWORD lpdwIndex
);

The code I've written so far is...

var
  Buffer      : DWORD;
  BufferLength: DWORD;

BufferLength = 4; // 4 bytes

if HttpQueryInfo(hOpenURL, HTTP_QUERY_CONTENT_LENGTH or HTTP_QUERY_FLAG_NUMBER, @Buffer, @BufferSize, nil) = FALSE then // Unsuccessful else success and length will be in Buffer.

When I try to compile though I get an error message "Types of actual and formal var parameters must be identical".  So it looks as though there is an error somewhere.

A couple of other things I am unsure about this function...

* What should I do with the last parameter?  Is it sensible to pass nil or should I be passing something else?

* When used with asynchronous WinInet, will it ever call the callback with an INTERNET_REQUEST_STATUS_COMPLETE_MESSAGE?  The reason I need to know is that any extra messages might have an impact on existing code.

Thanks in Advance,

Richard
Comment
Watch Question

Commented:
I really don't know much about the winInet api's but i did find this usenet article that seems to address the exact same issue:

http://groups.google.com/groups?hl=en&threadm=7vu5b5%24her12%40forums.borland.com&rnum=1&prev=/groups%3Fq%3Ddelphi%2Bhttpqueryinfo%2Bidentical%26hl%3Den%26meta%3D

GL
Mike
Russell LibbySoftware Engineer, Advisory
BRONZE EXPERT
Top Expert 2005

Commented:
Hey there Richard, been awhile:

I forget if your using Delphi3, 4, 5, but I thought it was 5. If this is true, then the WININET.PAS declares HttpQueryInfo as (hRequest: HINTERNET; dwInfoLevel: DWORD;
  lpvBuffer: Pointer; var lpdwBufferLength: DWORD;
  var lpdwReserved: DWORD): BOOL; stdcall;
  {$EXTERNALSYM HttpQueryInfo}

Meaning the lpdwBufferLength needs to be passed "BufferSize" instead of "@BufferSize" and lpdwReserved needs to be passed a DWORD variable instead of nil/pointer.

Example:

var  hOpenURL:      HINTERNET;
     Buffer:        DWORD;
     BufferSize:    DWORD;
     Reserved:      DWORD;
begin

  if HttpQueryInfo(hOpenURL, HTTP_QUERY_CONTENT_LENGTH or HTTP_QUERY_FLAG_NUMBER, @Buffer, BufferSize, Reserved) = False then ...
 
In regards to the other part of the question, MS indicates that only the following can be called async, so I would assume that only these would perform callbacks...

FtpCreateDirectory
FtpDeleteFile
FtpFindFirstFile
FtpGetCurrentDirectory
FtpGetFile
FtpOpenFile
FtpPutFile
FtpRemoveDirectory
FtpRenameFile
FtpSetCurrentDirectory
GopherFindFirstFile
GopherOpenFile
HttpEndRequest
HttpOpenRequest
HttpSendRequestEx
InternetConnect
InternetOpenUrl
InternetReadFileEx

Hope this helps,
Russell



Author

Commented:
Hi Russell,

Great to hear from you again!  I have found the source code that you sent me a few months ago invaluable (the TDownloader component) - I don't know how I would have got on without it.  Thanks!

I managed to get HttpQueryInfo to compile correctly (I have Delphi 4 Pro).  I'm still a little unsure though as to whether it will send INTERNET_STATUS_REQUEST_COMPLETE messages to the callback.  If it did, the problem could be that it might interfere with the reading in the main callback loop, for example it might think it has been made by InternetReadFile, when it has been made by HttpQueryInfo instead.

I noticed that InternetReadFile is not on the above async list, yet according to the TDownloader component (that uses overlapped IO), it calls the callback under 2 scenarios...

1) When InternetReadFile has finished reading the file.
2) It can return FALSE, but later calls the callback when InternetReadFile has finished filling the buffer.

I'm therefore wondering that if InternetRead can call the callback, can HttpQueryInfo?
Russell LibbySoftware Engineer, Advisory
BRONZE EXPERT
Top Expert 2005

Commented:

Richard,

I will verify this for you and get back to you later on today.

Russell

Software Engineer, Advisory
BRONZE EXPERT
Top Expert 2005
Commented:

Richard,

The HttpQueryInfo is totally synchronous. It will not come back until there is enough data from the request (headers are recvd) to determine the results, or an error occurs. It would be fairly easy to add an OnProgress event to the downloader component I sent you...

1.) Add another var to the TRequest record type to track length.
2.) When the url is open (see code in WndProc) perform the HttpQueryInfo and save the results in the new variable in TRequest record.
3.) When data is read directly, or is set by callback, calculate the percentage using the length of the data you currently have.

I will implement this if you would like (and mail it to you), I just was't sure if you had made any mods to the download component.

Russell

Author

Commented:
Russell,

Thanks a lot for looking into this for me.  Actually I have already implemented the OnProgress event, but thanks for offering.  My implementation is almost the same as you mentioned above.  My OnProgress event gives the BytesRead and TotalLength as 2 parameters, therefore the caller can display both the numbers (such as 7,000 of 25,000 bytes received) and/or a percentage.  If you do calculate a percentage by multiplying BytesRead by 100 and dividing by TotalLength, it is important to remember that TotalLength can sometimes be 0 and you need to check for this scenario in order to avoid a division by zero exception.  I also call the OnProgress event with a BytesRead value of 0 just after the file length is determined.  This way, it is easy for the caller of the component to know when reading has started.  So it all seems to be working well :-)

I'm quite pleased that HttpQueryInfo isn't async; if it was it would have complicated things a bit.

I tried the component again today and it seems very fast; it downloaded the homepage of 5 sites in less than 7 seconds using a dial-up connection.

Richard

Commented:
would you be so nice as to post a source code example on this simply because I am facing the same problem. I have wininet doing async'd sockets and am attempting to find a way of getting TProgressBar to show how much of the file is downloaded but dont know where to start. if you can thnx

Author

Commented:
Hi virtex,

Here is some sample code that will return the total length of the file.  Use this after you have opened the file with the InternetOpenUrl() function.

You need to update the progress bar when reading using both the amount of data read so far and the total file length.

// Determine file length

var
  hOpenURL: HINTERNET;
  dwBufferLen, dwTotalLength, dwIndex: DWORD;

dwBufferLen := SizeOf(dwTotalLength);
dwIndex := 0;
if (HttpQueryInfo(hOpenURL, HTTP_QUERY_CONTENT_LENGTH or HTTP_QUERY_FLAG_NUMBER, @dwTotalLength, dwBufferLen, dwIndex) = FALSE) then dwTotalLength := 0;

Hope this helps,

Richard

Commented:
thanks alot. I happened to come across a componant that does the same thing today.

here is the source for it.

very simple to do as well as being async'd

{*************************************************************}
{            HTTPGet component for Delphi 32                  }
{ Version:   1.9                                              }
{ E-Mail:    info@utilmind.com                                }
{ WWW:       http://www.utilmind.com                          }
{ Created:   October  19, 1999                                }
{ Modified:  January 15, 2000                                 }
{ Legal:     Copyright (c) 1999-2000, UtilMind Solutions      }
{*************************************************************}
{ PROPERTIES:                                                 }
{   Agent: String - User Agent                                }
{                                                             }
{*  BinaryData: Boolean - This setting specifies which type   }
{*                        of data will taken from the web.    }
{*                        If you set this property TRUE then  }
{*                        component will determinee the size  }
{*                        of files *before* getting them from }
{*                        the web.                            }
{*                        If this property is FALSE then as we}
{*                        do not knows the file size the      }
{*                        OnProgress event will doesn't work. }
{*                        Also please remember that is you set}
{*                        this property as TRUE you will not  }
{*                        capable to get from the web ASCII   }
{*                        data and ofter got OnError event.   }
{                                                             }
{   FileName: String - Path to local file to store the data   }
{                      taken from the web                     }
{   Password, UserName - set this properties if you trying to }
{                        get data from password protected     }
{                        directories.                         }
{   Referer: String - Additional data about referer document  }
{   URL: String - The url to file or document                 }
{*************************************************************}
{ METHODS:                                                    }
{   GetFile - Get the file from the web specified in the URL  }
{             property and store it to the file specified in  }
{             the FileName property                           }
{   GetString - Get the data from web and return it as usual  }
{               String. You can receive this string hooking   }
{               the OnDoneString event.                       }
{   Abort - Stop the current session                          }
{*************************************************************}
{ EVENTS:                                                     }
{   OnDoneFile - Occurs when the file is downloaded           }
{   OnDoneString - Occurs when the string is received         }
{   OnError - Occurs when error happend                       }
{   OnProgress - Occurs at the receiving of the BINARY DATA   }
{*************************************************************}
{ Please see demo program for more information.               }
{*************************************************************}
{                     IMPORTANT NOTE:                         }
{ This software is provided 'as-is', without any express or   }
{ implied warranty. In no event will the author be held       }
{ liable for any damages arising from the use of this         }
{ software.                                                   }
{ Permission is granted to anyone to use this software for    }
{ any purpose, including commercial applications, and to      }
{ alter it and redistribute it freely, subject to the         }
{ following restrictions:                                     }
{ 1. The origin of this software must not be misrepresented,  }
{    you must not claim that you wrote the original software. }
{    If you use this software in a product, an acknowledgment }
{    in the product documentation would be appreciated but is }
{    not required.                                            }
{ 2. Altered source versions must be plainly marked as such,  }
{    and must not be misrepresented as being the original     }
{    software.                                                }
{ 3. This notice may not be removed or altered from any       }
{    source distribution.                                     }
{*************************************************************}

unit HTTPGet;

interface

uses
  Windows, Messages, SysUtils, Classes, WinInet, Forms;

type
  TOnProgressEvent = procedure(Sender: TObject; TotalSize, Readed: Integer) of object;
  TOnDoneFileEvent = procedure(Sender: TObject; FileName: String; FileSize: Integer) of object;
  TOnDoneStringEvent = procedure(Sender: TObject; Result: String) of object;

  THTTPGetThread = class(TThread)
  private
    FTAgent,
    FTURL,
    FTFileName,
    FTStringResult,
    FTUserName,
    FTPassword,
    FTPostQuery,
    FTReferer: String;
    FTBinaryData: Boolean;

    FTAbort: Boolean;
    FTResult: Boolean;
    FTFileSize: Integer;
    FTToFile: Boolean;

    BytesToRead, BytesReaded: DWord;

    FTProgress: TOnProgressEvent;

    procedure UpdateProgress;
  protected
    procedure Execute; override;
  public
    constructor Create(aAgent, aURL, aFileName, aUserName, aPassword, aPostQuery, aReferer: String;
                       aBinaryData: Boolean; aProgress: TOnProgressEvent; aToFile: Boolean);
  end;

  THTTPGet = class(TComponent)
  private
    FAgent: String;
    FBinaryData: Boolean;
    FURL: String;
    FFileName: String;
    FUserName: String;
    FPassword: String;
    FPostQuery: String;
    FReferer: String;
    FWaitThread: Boolean;

    FThread: THTTPGetThread;
    FError: TNotifyEvent;
    FResult: Boolean;

    FProgress: TOnProgressEvent;
    FDoneFile: TOnDoneFileEvent;
    FDoneString: TOnDoneStringEvent;

    procedure ThreadDone(Sender: TObject);
  public
    constructor Create(aOwner: TComponent); override;

    procedure GetFile;
    procedure GetString;
    procedure Abort;
  published
    property Agent: String read FAgent write FAgent;
    property BinaryData: Boolean read FBinaryData write FBinaryData;
    property URL: String read FURL write FURL;
    property FileName: String read FFileName write FFileName;
    property UserName: String read FUserName write FUserName;
    property Password: String read FPassword write FPassword;
    property PostQuery: String read FPostQuery write FPostQuery;
    property Referer: String read FReferer write FReferer;
    property WaitThread: Boolean read FWaitThread write FWaitThread;

    property OnProgress: TOnProgressEvent read FProgress write FProgress;
    property OnDoneFile: TOnDoneFileEvent read FDoneFile write FDoneFile;
    property OnDoneString: TOnDoneStringEvent read FDoneString write FDoneString;
    property OnError: TNotifyEvent read FError write FError;
  end;

procedure Register;

implementation

//  THTTPGetThread

constructor THTTPGetThread.Create(aAgent, aURL, aFileName, aUserName, aPassword, aPostQuery, aReferer: String;
                                  aBinaryData: Boolean; aProgress: TOnProgressEvent; aToFile: Boolean);
begin
  FreeOnTerminate := True;
  inherited Create(False);

  FTAgent := aAgent;
  FTURL := aURL;
  FTFileName := aFileName;
  FTUserName := aUserName;
  FTPassword := aPassword;
  FTPostQuery := aPostQuery;
  FTReferer := aReferer;
  FTProgress := aProgress;
  FTBinaryData := aBinaryData;

  FTToFile := aToFile;
end;

procedure THTTPGetThread.UpdateProgress;
begin
  FTProgress(Self, FTFileSize, BytesReaded);
end;

procedure THTTPGetThread.Execute;

  procedure ParseURL(URL: String; var HostName, FileName: String);

    procedure ReplaceChar(c1, c2: Char; var St: String);
    var
      p: Integer;
    begin
      while True do
       begin
        p := Pos(c1, St);
        if p = 0 then Break
        else St[p] := c2;
       end;
    end;

  var
    i: Integer;
  begin
    if Pos('http://', LowerCase(URL)) <> 0 then
      System.Delete(URL, 1, 7);

    i := Pos('/', URL);
    HostName := Copy(URL, 1, i);
    FileName := Copy(URL, i, Length(URL) - i + 1);

    if (Length(HostName) > 0) and (HostName[Length(HostName)] = '/') then
      SetLength(HostName, Length(HostName) - 1);
  end;

label Aborted1, Aborted2, Aborted3, Aborted4, Aborted5;

var
  hSession, hConnect, hRequest: hInternet;
  HostName, FileName: String;
  f: File;
  Buf: Pointer;
  dwBufLen, dwIndex: DWord;
  Data: Array[0..$400] of Char;
  TempStr: String;
  RequestMethod, PQ: PChar;
begin
  try
    ParseURL(FTURL, HostName, FileName);

    if FTAgent <> '' then
     hSession := InternetOpen(PChar(FTAgent),
       INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0)
    else
     hSession := InternetOpen(nil,
       INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);

  {} if FTAbort then goto Aborted1;

    hConnect := InternetConnect(hSession, PChar(HostName),
      INTERNET_DEFAULT_HTTP_PORT, PChar(FTUserName), PChar(FTPassword), INTERNET_SERVICE_HTTP, 0, 0);

  {} if FTAbort then goto Aborted2;

    if FTPostQuery = '' then RequestMethod := 'GET'
    else RequestMethod := 'POST';

    hRequest := HttpOpenRequest(hConnect, RequestMethod, PChar(FileName), 'HTTP/1.0',
                PChar(FTReferer), nil, INTERNET_FLAG_RELOAD, 0);

  {} if FTAbort then goto Aborted3;

    if FTPostQuery = '' then
     HttpSendRequest(hRequest, nil, 0, nil, 0)
    else
     begin
      GetMem(PQ, Length(FTPostQuery));
      StrPCopy(PQ, FTPostQuery);
      HttpSendRequest(hRequest, nil, 0, PQ, Length(FTPostQuery));
      FreeMem(PQ);
     end;

  {} if FTAbort then goto Aborted3;

    dwIndex  := 0;
    dwBufLen := 1024;
    GetMem(Buf, dwBufLen);

    FTResult := HttpQueryInfo(hRequest, HTTP_QUERY_CONTENT_LENGTH,
                              Buf, dwBufLen, dwIndex);

 {} if FTAbort then goto Aborted4;

    if FTResult or not FTBinaryData then
     begin
      if FTResult then
        FTFileSize := StrToInt(StrPas(Buf));

      BytesReaded := 0;

      if FTToFile then
       begin
        AssignFile(f, FTFileName);
        Rewrite(f, 1);
       end
      else FTStringResult := '';

      while True do
       begin
        {} if FTAbort then goto Aborted5;
        if not InternetReadFile(hRequest, @Data, SizeOf(Data), BytesToRead) then Break
        else
         if BytesToRead = 0 then Break
         else
          begin
           if FTToFile then
            BlockWrite(f, Data, BytesToRead)
           else
            begin
             TempStr := Data;
             SetLength(TempStr, BytesToRead);
             FTStringResult := FTStringResult + Data;
            end;

           inc(BytesReaded, BytesToRead);
           if Assigned(FTProgress) then
            Synchronize(UpdateProgress);
          end;
       end;

      if FTToFile then
        FTResult := FTFileSize = BytesReaded
      else
       begin
        SetLength(FTStringResult, BytesReaded);
        FTResult := BytesReaded <> 0;
       end;

      Aborted5:
      if FTToFile then CloseFile(f);
     end;

    Aborted4:
    FreeMem(Buf);

    Aborted3:
    InternetCloseHandle(hRequest);
    Aborted2:
    InternetCloseHandle(hConnect);
    Aborted1:
    InternetCloseHandle(hSession);
  except
  end;
end;

// HTTPGet

constructor THTTPGet.Create(aOwner: TComponent);
begin
  inherited Create(aOwner);
  FAgent := 'UtilMind HTTPGet';
end;

procedure THTTPGet.GetFile;
begin
  if not Assigned(FThread) then
   begin
    FThread := THTTPGetThread.Create(FAgent, FURL, FFileName, FUserName, FPassword, FPostQuery, FReferer,
                                     FBinaryData, FProgress, True);
    FThread.FTAbort := False;
    FThread.OnTerminate := ThreadDone;
    if FWaitThread then
      while Assigned(FThread) do Application.ProcessMessages;
   end
end;

procedure THTTPGet.GetString;
begin
  if not Assigned(FThread) then
   begin
    FThread := THTTPGetThread.Create(FAgent, FURL, FFileName, FUserName, FPassword, FPostQuery, FReferer,
                                     FBinaryData, FProgress, False);
    FThread.FTAbort := False;                                
    FThread.OnTerminate := ThreadDone;
    if FWaitThread then
      while Assigned(FThread) do Application.ProcessMessages;
   end
end;

procedure THTTPGet.Abort;
begin
  if Assigned(FThread) then
   begin
    FThread.FTAbort := True;
    FThread.FTResult := True;
   end;
end;

procedure THTTPGet.ThreadDone(Sender: TObject);
begin
  FResult := FThread.FTResult;
  if FResult then
   if FThread.FTToFile then
    if Assigned(FDoneFile) then FDoneFile(Self, FThread.FTFileName, FThread.FTFileSize) else
   else
    if Assigned(FDoneString) then FDoneString(Self, FThread.FTStringResult) else  
  else
   if Assigned(FError) then FError(Self);
  FThread := nil;
end;

procedure Register;
begin
  RegisterComponents('UtilMind', [THTTPGet]);
end;

end.

it may help you guys out as it shows the progress thru a progressbar