Link to home
Start Free TrialLog in
Avatar of IElite
IElite

asked on

Get HTML Page from internet?


I need  a solution to grap the entire HTML from this webpage and place it in a TStringList:
http://www.geocaching.com/seek/cache_details.aspx?guid=ae9b2a75-de78-4bf0-b2d0-5568990b7cf8

IELite

ASKER CERTIFIED SOLUTION
Avatar of Eddie Shipman
Eddie Shipman
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of IElite
IElite

ASKER

Yeah, i figured someone would come back with that -

I get a EReadError with message property AuthProxyRetries does not exist

Ielite
Avatar of IElite

ASKER

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, IdBaseComponent, IdComponent, IdTCPConnection,
  IdTCPClient, IdHTTP;

type
  TForm1 = class(TForm)
    IdHTTP1: TIdHTTP;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;
  SL: TSTringList;

implementation

{$R *.dfm}

procedure TForm1.Button1Click(Sender: TObject);
begin
 SL.Text := idHTTP1.Get('http://www.geocaching.com/seek/cache_details.aspx?guid=ae9b2a75-de78-4bf0-b2d0-5568990b7cf8');
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
 SL:= TSTringList.Create;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
 SL.Free;
end;

end.
Avatar of IElite

ASKER

For what it is worth - Im using Delphi 7 & Indy 10

IELite
Worked just fne for me. Maybe you are behind a PROXY?
Ok, now I see it, you have the wrong Indy BPLs installed somewhere on your system.

"I get a EReadError with message property AuthProxyRetries does not exist"

This tells me that the during form load, it doesn't find this property for
the idHTTP control. MaxAuthRetries.

Reinstall Indy correctly.
Avatar of IElite

ASKER

Yeah, uninstalled and re-installed twice

IELite
Avatar of IElite

ASKER

I dont know th names of them - to delete them

they dont begine with 'Id'

IELite
search for Indy*.bpl

oh, and indy*.dcp
Avatar of IElite

ASKER

Yeah I did - Im working on it - I think there is more problems than I expected....
Cause I also have - Delphi 2005 installed on this machine as well...

IELite
Avatar of IElite

ASKER

I dont know.....

I uninstalled 2005

Removed all indy packages from delphi 7

Deleted all indy folders

removed all DCU's & Bpls that began with 'Id'

removed all paths from delphu 7 that had indy in them

still - getting the error

sholmes

Avatar of IElite

ASKER

OK, at leat i got it to compile and run NOW!

However, im still getting an error (not the same one)

ERangeError with message "Range Check Error"

IELite
Avatar of IElite

ASKER

I added the path to the indy source directly and now i am not getting the ERangeError

however, im back to where i originally started off yesterday before I upgraded to Indy 10

I was getting a error EConvertError with message "invalide Argument to date encode"

now im getting that error again, this time with indy 10


aaaaaaaaaaaaaaaaahhhhhhhhhh - frustration at its best!

IElite
Avatar of IElite

ASKER

Works Fine with just the - 'http://www.geocaching.com"

however, if i try the whole thing i get the error

EConvertError with message "invalide Argument to date encode"

IElite
Well, I am using D6 and Indy10 if it matters to you. I can not explain the trouble you are having.
Ah workaround is:

function TFunctions.DownloadHtml(const sURL, sLocalFileName:string): boolean;
begin
  Result := True;
  with TDownLoadURL.Create(nil) do
  try
    URL := sURL;
    Filename := sLocalFileName;
    try
     // Retrieve page
      ExecuteTarget(nil);
    except
      Result := False;
    end;
  finally
    Free;
  end;
end;

But I suggest you to try and get the Indy components to work.
Avatar of IElite

ASKER

What is a TDownLoadUrl ?

IElite
Oops, Indeed I forgot that part. The full code should be the following. It only works when you Delphi version is 6 or higher.

uses ExtActns;

...

function TFunctions.DownloadHtml(const sURL, sLocalFileName:string): boolean;
begin
  Result := True;
  with TDownLoadURL.Create(nil) do
  try
    URL := sURL;
    Filename := sLocalFileName;
    try
     // Retrieve page
      ExecuteTarget(nil);
    except
      Result := False;
    end;
  finally
    Free;
  end;
end;
Or you could also use WinInet functions, lot's of code but very effective:

procedure TForm1.Button1Click(Sender: TObject);
const
  BUFFERSIZE = 4096;
var
  ms: TMemoryStream;
  hOpenHandle, hConnectHandle, hResourceHandle: HINTERNET;
  lpdwlen, lpdwidx, lpdword: DWORD;
  dwError, dwErrorCode: DWORD;
  Retry: Boolean;
  buf: array[0..BUFFERSIZE-1] of char;
  bufsize: DWORD;
  ServerName, ObjectName: String;
  ptNil: Pointer;
begin
  ms := TMemoryStream.Create;
  try
    ServerName := 'www.austinmetrobaseball.com';
    ObjectName := 'index.html';
    hOpenHandle:=InternetOpen('whatever', INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);
    if hOpenHandle<>nil then
    begin
      hConnectHandle:=InternetConnect(hOpenHandle, PChar(ServerName),
                                      INTERNET_INVALID_PORT_NUMBER, nil,
                                      nil, INTERNET_SERVICE_HTTP,0,0);
      if hConnectHandle<>nil then
      begin
        hResourceHandle:=HttpOpenRequest(hConnectHandle, 'GET',
                                         PChar(ObjectName), nil, nil, nil,
                                         INTERNET_FLAG_KEEP_CONNECTION or
                                         INTERNET_FLAG_NO_CACHE_WRITE, 0);
        Retry:=True;
        while Retry do
        begin
          if HttpSendRequest(hResourceHandle, nil, 0, nil, 0) then
            dwErrorCode:=ERROR_SUCCESS
          else
            dwErrorCode:=GetLastError;
          dwError:=InternetErrorDlg(Self.Handle, hResourceHandle, dwErrorCode,
            FLAGS_ERROR_UI_FILTER_FOR_ERRORS or
            FLAGS_ERROR_UI_FLAGS_CHANGE_OPTIONS or
            FLAGS_ERROR_UI_FLAGS_GENERATE_DATA,
            ptNil);
          Retry:=(dwError=ERROR_INTERNET_FORCE_RETRY);
        end;
        if hResourceHandle<>nil then
        begin
          lpdwlen := 4;
          lpdwidx := 0;
          HttpQueryInfo(hResourceHandle,
            HTTP_QUERY_STATUS_CODE or HTTP_QUERY_FLAG_NUMBER ,
            @lpdword, lpdwlen, lpdwidx);
          if lpdword < 300 then
          begin
            {
            FilePath:=ExtractFilePath(FileName);
            if FilePath<>'' then
              ForceDirectories(FilePath);
            AssignFile(lf, FileName);
            {$i-
            Rewrite(lf,1);
            {$i+
            err:=IOResult;
            if err=0 then
            begin
              lpdword:=0;
              lpdwlen:=4;
              lpdwidx:=0;
            }
              HttpQueryInfo(hResourceHandle, HTTP_QUERY_CONTENT_LENGTH or
                HTTP_QUERY_FLAG_NUMBER, @lpdword, lpdwlen, lpdwidx);
            {
              TotSize:=lpdword;
              fsize := 0;
              tck := GetTickCount;
              }
              bufsize := BUFFERSIZE;
              while (bufsize > 0) do
              begin
                if not InternetReadFile(hResourceHandle, @buf,
                  BUFFERSIZE, bufsize) then Break;
                if (bufsize > 0) and (bufsize <= BUFFERSIZE) then
                  ms.Write(buf,bufsize);
              end;
            end
          end;
          InternetCloseHandle(hResourceHandle);
        end;
        InternetCloseHandle(hConnectHandle);
      end;
      InternetCloseHandle(hOpenHandle);
  finally
    ms.Position := 0;
    Memo1.Lines.LoadFromStream(ms);
    ms.Free;
  end;
end;
Don't forget to put WinInet in your uses.