Indy HTTP component and UserAgent

Posted on 2004-11-15
Last Modified: 2012-06-27
I am using Delphi 5 and my application uses the Indy HTTP component that came with the Pro version.

I am having a problem with the retrieved text from a web page.  I am trying to retrieve a web page at  The problem is that the retrieved data does not match what I get when I use Internet Explorer 6's "View > Source" option.  There are alot of differences between the two text results.

I thought it might have to do with the TIdHTTP's .Request.UserAgent value, which is natively set to 'Mozilla/3.0 (compatible; Indy Library)'.   I have changed this value to:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

...first, but that didn't make any difference, so I set it to...

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; T312461; .NET CLR 1.1.4322)

...(which is the thumbprint that my IE6 leaves behind in a web log on my server).  This too did not make any difference.

Is there something I am overlooking?

Question by:Monroe406
    LVL 17

    Assisted Solution

    by:Wim ten Brink
    Cookies, perhaps?
    LVL 17

    Accepted Solution

    A solution eithout using Indy:

    uses URLMon;

    procedure TForm1.Button1Click(Sender: TObject);
      // replace the URL with yours and the local filename also in the line below
      if URLDownloadToFile(nil, '', 'c:\Experts.html', 0, nil) <> 0 then
        MessageBox(Handle, 'Cannot download the file.', PChar(Application.Title), MB_ICONERROR or MB_OK);

    Regards, Geo

    Author Comment

    No, cookies are not the answer.  I just searched and deleted any cookies I could find containing the string "ebay" (I meant to say, not in my original posting).   Deleting the cookies made no difference.

    Author Comment

    Neither comment worked.  The resulting text from the "GET" still is different from what I get using IE 6 and Firefox 0.9.

    I am beginning to suspect that there is some redirection going on behind the scenes that the browsers can handle, but the Indy and URLDownloadToFile are not responding to.

    To this end, perhaps I need to find a utility that will tell me what GET requests are occuring between my browser and the website?  Does anyone know of any such tools that will tell me what URL is actually being displayed within IE?
    LVL 17

    Expert Comment

    by:Wim ten Brink
    There's one tool that might help but it captures a bit more than you might prefer. Ethereal is a tool that captures all network traffic between you and the rest of the network and I assume you're already familiar with it. If not, download it (it's free) and compare the network traffic of IE with the network traffic of Indy. Then you might notice where things go different.

    Featured Post

    What Is Threat Intelligence?

    Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

    Join & Write a Comment

    Objective: - This article will help user in how to convert their numeric value become words. How to use 1. You can copy this code in your Unit as function 2. than you can perform your function by type this code The Code   (CODE) The Im…
    Introduction Raise your hands if you were as upset with FireMonkey as I was when I discovered that there was no TListview.  I use TListView in almost all of my applications I've written, and I was not going to compromise by resorting to TStringGrid…
    It is a freely distributed piece of software for such tasks as photo retouching, image composition and image authoring. It works on many operating systems, in many languages.
    This video is in connection to the article "The case of a missing mobile phone (". It will help one to understand clearly the steps to track a lost android phone.

    729 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now