• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 951
  • Last Modified:

Download a tricky file with Indy in Delphi 2009

If I paste this link into my browser, I eventually get a "download/open" dialog box so I can save the xml file
I want to automate this in Delphi 2009 using Indy
e.g. TidHTTP

The link is
http://download.xmltv.co.nz/archive=none&icons=large&links=remote&provider=freeview/TC-TV1A/TC-TV2S/FV-tv3plus1/TC-TV4/TVGuide.xml

It should be easy, but I get a "page not found" when I try...
note: I can download other files easily enough :-/

0
TheRealLoki
Asked:
TheRealLoki
  • 7
  • 6
  • 4
  • +1
2 Solutions
 
Emmanuel PASQUIERFreelance Project ManagerCommented:
can you provide a sample of the code you are using so that we might start from there to test and possibly fix what's wrong ?
0
 
ThievingSixCommented:
Do you have to use indy?
program Project61;

{$APPTYPE CONSOLE}

uses
  Windows,
  SysUtils,
  WinInet;

function DownloadURL(const aUrl: string): AnsiString;
var
  hSession: HINTERNET;
  hService: HINTERNET;
  lpBuffer: array[0..1024 + 1] of AnsiChar;
  dwBytesRead: DWORD;
begin
  Result := '';
  hSession := InternetOpen('MyApp', INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);
  try
    if Assigned(hSession) then
    begin
      hService := InternetOpenUrl(hSession, PChar(aUrl), nil, 0, 0, 0);
      if Assigned(hService) then
        try
          while True do
          begin
            dwBytesRead := 1024;
            InternetReadFile(hService, @lpBuffer, 1024, dwBytesRead);
            if dwBytesRead = 0 then break;
            lpBuffer[dwBytesRead] := #0;
            Result := Result + lpBuffer;
          end;
        finally
          InternetCloseHandle(hService);
        end;
    end;
  finally
    InternetCloseHandle(hSession);
  end;
end;

var
  Test : String;
begin
  Writeln(String(DownloadURL('http://download.xmltv.co.nz/archive=none&icons=large&links=remote&provider=freeview/TC-TV1A/TC-TV2S/FV-tv3plus1/TC-TV4/TVGuide.xml')));
  readln;
end.

Open in new window

0
 
Emmanuel PASQUIERFreelance Project ManagerCommented:
Nice one ThievingSix !
I'm going to copy this in my toolbox as it is a most easy way.
I'm not too sure about the While True part and break if BytesRead=0, it could end too soon in some cases no ? Will have to check in InternetReadFile how things are handled.
0
Cloud Class® Course: Amazon Web Services - Basic

Are you thinking about creating an Amazon Web Services account for your business? Not sure where to start? In this course you’ll get an overview of the history of AWS and take a tour of their user interface.

 
ThievingSixCommented:
This was just a quick find from http://www.swissdelphicenter.ch/torry/showcode.php?id=412 to see if Loki needed to use indy or if something like this was acceptable.

"To ensure all data is retrieved, an application must continue to call the InternetReadFile function until the function returns TRUE and the lpdwNumberOfBytesRead parameter equals zero."

The code above isn't perfect, but like I said in the beginning I was just probing for info.
0
 
Emmanuel PASQUIERFreelance Project ManagerCommented:
:o) How us poor programmers where doing before google (or altavista) ? Life is almost too easy now
0
 
ThievingSixCommented:
We had bookmarks ;). You know, the real kind!
0
 
Emmanuel PASQUIERFreelance Project ManagerCommented:
ok, I rephrase : How us programmers where doing before Internet ?
Well, I suppose we had no problems with file downloading then :o))
0
 
Ferruccio AccalaiSenior developer, analyst and customer assistance Commented:
This is an example using Indy 10 TIdHttp, wich save your TVGuide.xml in C:\ clicking on Button1

FORM
object Form1: TForm1
  Left = 196
  Top = 92
  Caption = 'Form1'
  ClientHeight = 444
  ClientWidth = 680
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'Tahoma'
  Font.Style = []
  OldCreateOrder = False
  PixelsPerInch = 96
  TextHeight = 13
  object Button1: TButton
    Left = 374
    Top = 64
    Width = 75
    Height = 25
    Caption = 'Button1'
    TabOrder = 0
    OnClick = Button1Click
  end
  object IdHTTP1: TIdHTTP
    AllowCookies = True
    ProxyParams.BasicAuthentication = False
    ProxyParams.ProxyPort = 0
    Request.ContentLength = -1
    Request.Accept = 'text/html, */*'
    Request.BasicAuthentication = False
    Request.UserAgent = 'Mozilla/3.0 (compatible; Indy Library)'
    HTTPOptions = [hoForceEncodeParams]
    OnHeadersAvailable = IdHTTP1HeadersAvailable
    Left = 284
    Top = 74
  end
end
UNIT
unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, IdBaseComponent, IdComponent, IdTCPConnection, IdTCPClient, IdHTTP,
  StdCtrls, IdHeaderList;

type
  TForm1 = class(TForm)
    IdHTTP1: TIdHTTP;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
    procedure IdHTTP1HeadersAvailable(Sender: TObject; AHeaders: TIdHeaderList;
      var VContinue: Boolean);
  private
    { Private declarations }
    FileName: string;
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.Button1Click(Sender: TObject);
var
  MemStream: TMemoryStream;
  FileStream: TFileStream;
begin
  MemStream := TMemoryStream.Create;
  try
    IdHTTP1.Get('http://download.xmltv.co.nz/archive=none&icons=large&links=remote&provider=freeview/TC-TV1A/TC-TV2S/FV-tv3plus1/TC-TV4/TVGuide.xml', MemStream);
    MemStream.Position := 0;
    FileStream := TFileStream.Create(FileName, fmCreate);
    try
      MemStream.SaveToStream(FileStream);
    finally
      FileStream.Free;
    end;
  finally
    MemStream.Free;
  end;
end;

procedure TForm1.IdHTTP1HeadersAvailable(Sender: TObject;
  AHeaders: TIdHeaderList; var VContinue: Boolean);
var
  ContentDisposition: string;
begin
  ContentDisposition := AHeaders.Values['Content-Disposition'];
  ContentDisposition := Copy(ContentDisposition, 1 + Pos(';', ContentDisposition), MaxInt);
  ContentDisposition := StringReplace(ContentDisposition, 'filename=', '', [rfIgnoreCase]);
  ContentDisposition := StringReplace(ContentDisposition, '"', '', [rfReplaceAll]);
  FileName := 'C:\'+ContentDisposition;
end;

end.

Open in new window

0
 
TheRealLokiSenior DeveloperAuthor Commented:
Hi Ferruccio68
Your source is similar to what I already had.

It works for some things, but not for the tvguide

I get an "HTTP/1.2 404 not found" error when I try

here's my modified sample, showing some urls that work, and the tvguide one that doesn't



object Form1: TForm1
  Left = 196
  Top = 92
  Caption = 'Form1'
  ClientHeight = 444
  ClientWidth = 680
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'Tahoma'
  Font.Style = []
  OldCreateOrder = False
  PixelsPerInch = 96
  TextHeight = 13
  object Button1: TButton
    Left = 374
    Top = 64
    Width = 75
    Height = 25
    Caption = 'Button1'
    TabOrder = 0
    OnClick = Button1Click
  end
  object Memo1: TMemo
    Left = 32
    Top = 112
    Width = 473
    Height = 273
    Lines.Strings = (
      'Memo1')
    TabOrder = 1
  end
  object ComboBox1: TComboBox
    Left = 32
    Top = 8
    Width = 473
    Height = 21
    ItemHeight = 13
    TabOrder = 2
    Text = 
      'http://metvuw.com/forecast/2009121718/rain-nzni-2009121718-06.gi' +
      'f'
    Items.Strings = (
      
        'http://metvuw.com/forecast/2009121718/rain-nzni-2009121718-06.gi' +
        'f'
      'http://xmltv.co.nz/uploads/images/donate3.gif'
      
        'http://download.xmltv.co.nz/archive=none&icons=large&links=remot' +
        'e&provider=freeview/TC-TV1A/TC-TV2S/FV-tv3plus1/TC-TV4/TVGuide.x' +
        'ml'
      
        'http://socialmedialawstudent.com/wp-content/uploads/2009/10/vade' +
        'r-fail1.jpg')
  end
  object IdHTTP1: TIdHTTP
    AllowCookies = True
    RedirectMaximum = 100
    ProxyParams.BasicAuthentication = False
    ProxyParams.ProxyPort = 0
    Request.ContentLength = -1
    Request.Accept = 'text/html, */*'
    Request.BasicAuthentication = False
    Request.UserAgent = 'Mozilla/3.0 (compatible; Indy Library)'
    HTTPOptions = [hoForceEncodeParams]
    OnHeadersAvailable = IdHTTP1HeadersAvailable
    Left = 284
    Top = 74
  end
end

UNIT

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, IdBaseComponent, IdComponent, IdTCPConnection, IdTCPClient, IdHTTP,
  StdCtrls, IdHeaderList;

type
  TForm1 = class(TForm)
    IdHTTP1: TIdHTTP;
    Button1: TButton;
    Memo1: TMemo;
    ComboBox1: TComboBox;
    procedure Button1Click(Sender: TObject);
    procedure IdHTTP1HeadersAvailable(Sender: TObject; AHeaders: TIdHeaderList;
      var VContinue: Boolean);
  private
    { Private declarations }
    FileName: string;
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.Button1Click(Sender: TObject);
var
  MemStream: TMemoryStream;
  FileStream: TFileStream;
begin
  IdHTTP1.HandleRedirects := true;

  MemStream := TMemoryStream.Create;
  try
    IdHTTP1.Get(Combobox1.Text, MemStream);
    MemStream.Position := 0;
    FileStream := TFileStream.Create(FileName, fmCreate);
    try
      MemStream.SaveToStream(FileStream);
    finally
      FileStream.Free;
    end;
  finally
    MemStream.Free;
  end;
end;

procedure TForm1.IdHTTP1HeadersAvailable(Sender: TObject;
  AHeaders: TIdHeaderList; var VContinue: Boolean);
var
  ContentDisposition: string;
begin
  ContentDisposition := AHeaders.Values['Content-Disposition'];
  memo1.lines.text := ContentDisposition;
  ContentDisposition := Copy(ContentDisposition, 1 + Pos(';', ContentDisposition), MaxInt);
  ContentDisposition := StringReplace(ContentDisposition, 'filename=', '', [rfIgnoreCase]);
  ContentDisposition := StringReplace(ContentDisposition, '"', '', [rfReplaceAll]);
  if (ContentDisposition <> '') then
    FileName := 'C:\'+ContentDisposition
  else
    Filename := 'C:\file.dat';

end;

end.

Open in new window

0
 
TheRealLokiSenior DeveloperAuthor Commented:
Hi ThievingSix, long time no see...
Your source works just fine.
I would have liked to be able to do it in Indy, as I have proxy stuff and other routines hinging on it
But if an Indy solution can't be found, I'll accept your answer as the solution, and rework my existing code somehow
0
 
TheRealLokiSenior DeveloperAuthor Commented:
typo above btw
I get an "HTTP/1.2 404 not found" error when I try
actual error is (of course)
"HTTP/1.1 404 not found"
0
 
ThievingSixCommented:
Long time no see =O?

I believe you can use a proxy with my example, but after school I'll go ahead and give indy a try. You are using indy 10 correct?
0
 
TheRealLokiSenior DeveloperAuthor Commented:
yep, Delphi 2009, with default Indy 10 except for that SMTP attachment fix (unrelated).
long time, I don't get on here as much as I used to, and used to answer the same questions as you until real life kicked in...
0
 
Ferruccio AccalaiSenior developer, analyst and customer assistance Commented:
Strange thing, I can download it with that code, in fact i got TVGuide.xml in my C:\

I use Indy 10 and Delphi 2010
0
 
TheRealLokiSenior DeveloperAuthor Commented:
maybe it's a Delphi 2009 issue, but unfortunately, that's what I'm using
0
 
ThievingSixCommented:
Hmm, I doubt it's a Delphi 2009 issue, have you tried updating your indy snapshot?
0
 
TheRealLokiSenior DeveloperAuthor Commented:
I guess it must be an Indy bug then.
I work in a shared environment and cant update Indy for a while, so I've modified ThievingSix's sample (to use streams) and it's working except for proxy, which I'll have to learn how to do later.
Points split
0
 
TheRealLokiSenior DeveloperAuthor Commented:
Indy source identical to mine. Issue is with Indy. Used ThievingSix's code as an alternate
0
 
ThievingSixCommented:
hSession: = InternetOpen ( 'MyApp', INTERNET_OPEN_TYPE_PROXY, 'proxy: 8080', nil, 0);

You can use InternetSetOption to set the username/password for the proxy. http://msdn.microsoft.com/en-us/library/aa385384%28VS.85%29.aspx
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Amazon Web Services - Basic

Are you thinking about creating an Amazon Web Services account for your business? Not sure where to start? In this course you’ll get an overview of the history of AWS and take a tour of their user interface.

  • 7
  • 6
  • 4
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now