Link to home
Start Free TrialLog in
Avatar of roov
roovFlag for Israel

asked on

HTTP Access

Hi there,

I'm writing some software which in many ways is like a Browser (Netscape, Internet Explorer and co.).
It retrieves HTML pages from HTTP servers.
Mostly it works just fine, yet there are a number of sites that elude me.
Such sites are:
http://www.applelinks.com
http://www.expectingrain.com
When I access these sites from std. browsers, such as those I've mentioned, things work fine.
Yet when my software accesses them, I don't get the same HTML content these browsers get.

More info:
1. I use the std. HTTP 1.0 method of GET (and it works fine in 99% of the cases
2. I tested this on the Internet Sample PowerPlant provides, with the same results.
3. If you try these sites with the Internet Sample, you'll note Expecting Rain returns the HTTP Location header information. I honor that, but still don't get the info. std. browsers get.

The question:
What extra information do Netscape/IE send (and I should send) in order to get the correct HTML info back from these sites?

Thanks,
 Roov
ASKER CERTIFIED SOLUTION
Avatar of boonstra
boonstra

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of roov

ASKER

The answer contained the information required - I didn't need the logs, I actually tried out OTSessionWatcher, and simply compared my logs with those of std. browsers.

The obvious problem showed it's face immediatly: some sites do not send you back all requested data if you don't specify the (optional) "Host" field in the HTTP request header.

Once I realized that, things worked fine...