Learn how to a build a cloud-first strategyRegister Now


Indy HTTP component and UserAgent

Posted on 2004-11-15
Medium Priority
Last Modified: 2012-06-27
I am using Delphi 5 and my application uses the Indy HTTP component that came with the Pro version.

I am having a problem with the retrieved text from a web page.  I am trying to retrieve a web page at amazon.com.  The problem is that the retrieved data does not match what I get when I use Internet Explorer 6's "View > Source" option.  There are alot of differences between the two text results.

I thought it might have to do with the TIdHTTP's .Request.UserAgent value, which is natively set to 'Mozilla/3.0 (compatible; Indy Library)'.   I have changed this value to:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

...first, but that didn't make any difference, so I set it to...

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; T312461; .NET CLR 1.1.4322)

...(which is the thumbprint that my IE6 leaves behind in a web log on my server).  This too did not make any difference.

Is there something I am overlooking?

Question by:Monroe406
  • 2
  • 2
LVL 17

Assisted Solution

by:Wim ten Brink
Wim ten Brink earned 80 total points
ID: 12592970
Cookies, perhaps?
LVL 17

Accepted Solution

geobul earned 120 total points
ID: 12593132
A solution eithout using Indy:

uses URLMon;

procedure TForm1.Button1Click(Sender: TObject);
  // replace the URL with yours and the local filename also in the line below
  if URLDownloadToFile(nil, 'http://www.experts-exchange.com/Programming/Programming_Languages/Delphi/', 'c:\Experts.html', 0, nil) <> 0 then
    MessageBox(Handle, 'Cannot download the file.', PChar(Application.Title), MB_ICONERROR or MB_OK);

Regards, Geo

Author Comment

ID: 12593140
No, cookies are not the answer.  I just searched and deleted any cookies I could find containing the string "ebay" (I meant to say ebay.com, not amazon.com in my original posting).   Deleting the cookies made no difference.

Author Comment

ID: 12600674
Neither comment worked.  The resulting text from the "GET" still is different from what I get using IE 6 and Firefox 0.9.

I am beginning to suspect that there is some redirection going on behind the scenes that the browsers can handle, but the Indy and URLDownloadToFile are not responding to.

To this end, perhaps I need to find a utility that will tell me what GET requests are occuring between my browser and the website?  Does anyone know of any such tools that will tell me what URL is actually being displayed within IE?
LVL 17

Expert Comment

by:Wim ten Brink
ID: 12602417
There's one tool that might help but it captures a bit more than you might prefer. http://www.ethereal.com/ Ethereal is a tool that captures all network traffic between you and the rest of the network and I assume you're already familiar with it. If not, download it (it's free) and compare the network traffic of IE with the network traffic of Indy. Then you might notice where things go different.

Featured Post

Upgrade your Question Security!

Add Premium security features to your question to ensure its privacy or anonymity. Learn more about your ability to control Question Security today.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains how to create forms/units independent of other forms/units object names in a delphi project. Have you ever created a form for user input in a Delphi project and then had the need to have that same form in a other Delphi proj…
Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
Integration Management Part 2
This lesson discusses how to use a Mainform + Subforms in Microsoft Access to find and enter data for payments on orders. The sample data comes from a custom shop that builds and sells movable storage structures that are delivered to your property. …
Suggested Courses
Course of the Month20 days, 17 hours left to enroll

810 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question