I am using Delphi 5 and my application uses the Indy HTTP component that came with the Pro version.
I am having a problem with the retrieved text from a web page. I am trying to retrieve a web page at amazon.com. The problem is that the retrieved data does not match what I get when I use Internet Explorer 6's "View > Source" option. There are alot of differences between the two text results.
I thought it might have to do with the TIdHTTP's .Request.UserAgent value, which is natively set to 'Mozilla/3.0 (compatible; Indy Library)'. I have changed this value to:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
...first, but that didn't make any difference, so I set it to...
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; T312461; .NET CLR 1.1.4322)
...(which is the thumbprint that my IE6 leaves behind in a web log on my server). This too did not make any difference.
Is there something I am overlooking?