Where is 'header('Content-Type: text/xml');' actually 'sent' and other http/CGI(?) questions

When I'm creating a response to an HTTP request and php executes a header call,

1) where exactly is that header field sent to?

I suspect it is to the Web Server that is managing the connection... so I imagine the Web Server (e.g. Apache) is assembling/buffering the http response as the php script sends it, possible in pieces.

2) Is that how it works?

3) If so, is there some place to examine the Apache buffer where said response as it is being 'built' by php/html output?

4) What is the event that tells Apache the response is complete and to send it?

Please correct any errors in my wording/naming of things as I try to better understand how this all works.  Thanks--
SAbboushiAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Marco GasiFreelancerCommented:
The header is sento to the client side, not to the server side, that is is sent to the browser so it know what type of data it is receiving.
In Firebug console with Firefox or in the Developers Tool Console with Chrome you can see all data sent and received by the browser
0
SAbboushiAuthor Commented:
Thanks Marco -

The header is sento to the client side, not to the server side
I understand that the header is prepared by the application on the server side for ultimate delivery to the client side, but my understanding is that the application passes the header/response to Apache who in turn sends it to the client.  My question has to do with the interaction between the server side application and Apache.

I'm using Fiddler to monitor requests and responses.  I find that the header is part of the response, and that said header is not sent to the browser (in my case, a client application) until the entire response has been prepared, i.e. it seems it is not until my php script is finished preparing the response that Apache actually sends the response... and until then, Fiddler shows the status of the request/response as "Response is being read from the server" and no response header/response data yet.  

ServerBeginResponse timestamp for the request/response session seems to be stamped as soon as the request has been received (i.e. even before the response has started to be prepared i.e. before php executes the aforementioned header statement).  My findings seem confirmed because GotResponseHeaders timestamp remains empty after php executes the header statement.

So unless you have an alternate explanation for my findings above, my 4 questions still remain open...
0
Marco GasiFreelancerCommented:
I'm sorry, I'll monitor this thread to learn something more... :-)
0
Cloud Class® Course: Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

Dave BaldwinFixer of ProblemsCommented:
Apache passes the page request to the PHP interpreter.  The PHP interpreter processes the page and sends the content back to Apache which then sends it on to the client, whether it is a browser or application program.  I don't exactly what it is but PHP and other such programs have a way to tell Apache that they are done.  

PHP does some buffering itself.  You can use 'obstart' http://php.net/manual/en/function.ob-start.php and related functions to control it.  Different web servers handle buffering of page content differently.  I don't know at the moment what the minimum that Apache will send but IIS supposedly won't send anything until 1024 bytes are available to be sent or the application ends.

As for examining the 'buffer', I doubt that you can do that.  A typical Apache installation has between 4 and 100 threads handling page requests.  In addition, it usually happens very quickly so it is not going to be sitting there for you to examine.

Because Fiddler is running on the client, it only 'sees' things as they pass thru it's proxy set up.  It does not 'see' anything that happens only in the browser or only on the server.

Headers like Content-Type: text/xml have to be sent before any page content because they have to be part of the 'response header' to be recognized.
0
SAbboushiAuthor Commented:
Thanks Dave for sharing what you know.

>> A typical Apache installation has between 4 and 100 threads handling page requests.
I don't think this is applicable to my situation since I'm running Apache on my laptop.

>> In addition, it usually happens very quickly so it is not going to be sitting there for you to examine.
Just to be clear, since I'm talking about a debug scenario, I'm able to keep that request/response session 'open' for as long as I want... so I imagine time isn't the issue here.

Anyone else?
0
gr8gonzoConsultantCommented:
So just to try and fill in some gaps:

1. Apache (or most web servers) can either buffer or not buffer. Most server configurations do automatic Gzip compression these days, so any compression on the web server will require the web server to buffer the entire output first, then compress, then send.

2. There is no good way to output the buffer aside from either customizing the mod_deflate to save the data, or finding some other module that is pre-made that will do this for you. However, you have to understand that this is generally not a good idea, because you might end up dumping TONS of data onto the server, which is going to severely impact performance in multiple ways (I/O bottlenecks galore).

3. As Dave mentioned, PHP has its own buffering system. If you add ob_start() to the beginning of your script, you can add ob_get_clean() (there are several ways to end the buffer) to get the final data in a string variable that you can then tell PHP to write to a file somewhere. That should give you the final output before PHP returns the final data set to Apache.

4. If no buffering and no compression is enabled on either Apache nor on PHP, then PHP is going to return data to the server as soon as it gets it. The server queues up HTTP headers until the first byte of non-HTTP-header content is discovered (e.g. the first bit of whitespace or the first echo/print, etc...), and then the server will flush the HTTP headers to the browser. The web server can then store up to a certain amount of returned content in a small memory buffer and send at its own pace (for performance reasons), and PHP can also issue the flush() command to force the web server to send everything it has queued up at that moment, and then keep on going with more data. If you have a web server buffer turned on, however, a flush() doesn't do very much.

5. There is no customizable "event" where PHP tells Apache it's finished. It's an internal thread signal. Once the PHP child process is done, it returns the data and then closes, and the web server can know that it is finished. This process differs a little based on how you've implemented PHP (e.g. CGI binary, server module, FastCGI, etc), but the principle is the same - once the data stream from PHP is closed, Apache knows it's all done.
0
gr8gonzoConsultantCommented:
All of that said, if you can elaborate on what you're trying to achieve or debug, we might be able to help a bit more. The questions you're asking aren't likely to result in much success in debugging unless you're comfortable building Apache from source and are comfortable with C.
0
Dave BaldwinFixer of ProblemsCommented:
The fact that your Apache is on your laptop is irrelevant, that is a standard feature of Apache.  And the 'session' in Fiddler is on your computer, not on the server.  Fiddler is watching the network traffic.  It can not see what is happening in a buffer on the server.  It only 'sees' the network traffic returned from the server.
0
SAbboushiAuthor Commented:
Thanks for your help guys--

gr8gonzo:
EXCELLENT! EXCELLENT!  Boy did you "fill in some gaps"!

1. Apache (or most web servers) can either buffer or not buffer.
I'm not sure whether I used the right word when I said 'buffering'; what I mean is that when my php script is sending a response, it first sends the header.  I believe Apache receives this header and waits for further data from my php script.  Unless my script is sending an empty response, I suspect that Apache will, as a rule, not send a response to the requesting app/browser with just the header... I suspect it will wait, as a rule, for some response data from my php script.  It is this process of Apache holding the header while waiting for response data that I am referring to when I said 'buffering'.  Is my understanding correct?  And is this different than what you mean re: "[Web Servers] can either buffer or not buffer"?

2. There is no good way to output the buffer aside from either customizing the mod_deflate to save the data
I understand you to be saying that the "buffer" contains the response generated by my php script and that the only way to examine it/save it at a point PRIOR to Apache sending the response would be to hack into Apache somewhere, such as the mod_deflate module which, if the response is going to be compressed, would need to be passed to it for compression.  Is that correct?

3. ...  That should give you the final output before PHP returns the final data set to Apache.
Thanks - my apologies... I should have clarified in my question that I am not trying to solve a problem; I am merely trying to learn more about how php and Apache interact ; )

4. If no buffering and no compression is enabled on either Apache nor on PHP, then PHP is going to return data to the server as soon as it gets it.
I would have thought that buffering/compression on Apache has no bearing on how/when PHP sends response data to the server; I would think that PHP buffering (like what Dave mentioned) would be the sole determinant...?

4. ... The server queues up HTTP headers until the first byte of non-HTTP-header content is discovered (e.g. the first bit of whitespace or the first echo/print, etc...), and then the server will flush the HTTP headers to the browser.
Ah!  If my php script is not buffering, are you saying that Apache (for uncompressed responses) will/can/may send part of the response to the requesting application/browser before Apache receives the entire response from my php script?
0
gr8gonzoConsultantCommented:
So to be clear, buffering is a generic term here that just means that output is not used right away - it is stored in memory and then sent later. There's different types of buffers going on all the time. It may help to explain the response process.

A typical HTTP response basically has two parts - the HTTP response header and the HTTP response body. They are separated by a single blank line. An entire raw HTTP response might look like this:

HTTP/1.1 200 OK
Date: Wed, 04 Mar 2015 23:35:12 GMT
Server: Apache/2.4.7 (Win32) PHP/5.5.8
X-Powered-By: PHP/5.5.8
Content-Length: 38
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html

<html><body>Hello world!</body></html>

Open in new window

So you can see the header block at the top, and then EVERYTHING below the blank line is the content. It doesn't matter how many blank lines there are after that.

When a request first comes in, Apache and PHP work together to figure out what the headers will be. PHP can use the header() command to instruct Apache to use a specific HTTP header. So you could do:
<?php 
header("Foobar: Hello world!");

Open in new window


...and your HTTP header in the response would contain that line:
Foobar: Hello world!

Open in new window


You can even change the value multiple times, and Apache and PHP will work together to make sure that only the latest value gets used:

<?php
header("Foobar: ABC");
header("Foobar: DEF");

Open in new window


...and your HTTP header in the response would contain:
Foobar: DEF

Open in new window


You can have PHP play with headers all day long, but they will not be sent over to the browser until the first byte of non-HTTP-header content is found. So if you had this code:

<?php
header("Foobar: DEF");
echo "Hello world!";

Open in new window


As soon as that first echo was passed back to Apache, the web server would put together the block of content making up all the HTTP header lines, including the custom Foobar HTTP header, and would send it all to the browser, followed by a blank line, and then the phrase "Hello world!"

If you tried this:
<?php
header("Foobar: ABC");
echo "Hello world!";
header("Foobar: DEF");

Open in new window


...then you would get an HTTP header with "Foobar: ABC" and then "Hello world" and then an error message saying that you cannot modify the headers because headers have already been sent - because you tried to send the second header() line after the content was sent.

If you use PHP's output buffer (ob_start, etc...), then PHP will hold EVERYTHING in memory, and it will not send ANY contents to Apache until the buffer is flushed. That means you could do the above code without getting that same error. PHP would be smart enough to realize that the buffer is active, and would allow you to modify the headers at any time. Once the buffer is "flushed", all that content is sent to Apache, and it's up to Apache to send it to the browser.

Now, if Apache is configured to use compression (for example, the mod_deflate Apache module), then Apache will use its own buffer, too. This is a completely separate buffer from PHP. The reason for this is that in order to compress any content, you have to know what all of the content IS first. So if Apache is using its own buffer, then it will not return a response to the browser until PHP finishes.

It may help to imagine a restaurant where a table of people (the web browser) give their food orders to a waiter (the web server). The waiter takes the orders back to the chef in the kitchen (PHP), who prepares the plates of food (the response). The chef doesn't really care about the waiter - he only cares about doing his own job and filling the orders. When he's finished, he leaves the plates of food on a counter, and the waiter can pick them up and take it back to the diner.

Now, a restaurant can have its own policies on how to bring out the food. For example, a restaurant might say, "Don't bring out the plates one-by-one. Instead, wait until the ENTIRE order is completed, and then take all the plates back to the table at once." That is basically the same thing as web server buffering. PHP might be doing its own thing, but Apache is ultimately the one that deals with the browser, so if Apache is told to do anything that requires buffering, then it will wait for the entire response to be ready before it sends anything to the browser, regardless of how PHP does its own thing.

Likewise, the chef might decide not to put each plate of food out on the counter one-by-one. The chef might decide to create all the food, and once all the plates are ready, he puts them ALL out on the counter at once, so the waiter gets them all at once. That is the chef's decision, and that's like PHP's output buffering.

The different buffers have nothing to do with each other, though. The waiter's choice to bring everything out at once has nothing to do with the chef's choice to put all the plates out at once.

If you WANT the fastest response possible, then the way to do it is to ensure that your chef is putting out plates as they are ready, and that the waiter is taking plates to the table as soon as they are given to him. Translated, that means that if PHP does not use output buffering, then it will be sending the data to Apache as soon as it gets it. Then, if Apache does not use compression or any other modules that require buffering, then Apache will send it to the browser as soon as it gets data from PHP. So the browser might start receiving data even before PHP is done with the script.

The downside to that is that by not using buffering, you lose the benefits that come with it (e.g. PHP's ability to modify headers after content is outputted, or Apache's ability to compress the page). So sometimes it's a matter of deciding what benefit is better. For example, if you're regularly sending large, multi-megabyte text files, then it would probably be better to allow Apache to compress the output so that the download of the final data is faster.

I understand you to be saying that the "buffer" contains the response generated by my php script and that the only way to examine it/save it at a point PRIOR to Apache sending the response would be to hack into Apache somewhere, such as the mod_deflate module which, if the response is going to be compressed, would need to be passed to it for compression.  Is that correct?
Correct.

I would have thought that buffering/compression on Apache has no bearing on how/when PHP sends response data to the server;
That is correct. Sorry - I must have been thinking one thing and typing another. Buffering/compression on Apache only determines when the response will be sent to the browser. Buffering on PHP determines when the data will be sent to Apache.

Ah!  If my php script is not buffering, are you saying that Apache (for uncompressed responses) will/can/may send part of the response to the requesting application/browser before Apache receives the entire response from my php script?
Correct. That said, some browsers deal with rendering content in their own ways. For example, Internet Explorer sometimes will not start rendering anything it receives until it has at least 256 bytes of data (or something like that - it's been a while since I looked at the specifications). So browsers can use their own buffers when rendering pages in order to render them properly or faster. For example, some browsers can have problems with rendering table tags in HTML until the table definition is finished and closed, at which point, they'll know how wide the table will be, all the content inside, etc..., and THEN it will appear on the page. So if you're experimenting with things and you want to see "streaming" content (uncompressed data coming back from the server while PHP is still going at it), then you should take those rendering quirks into consideration.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Dave BaldwinFixer of ProblemsCommented:
That's a good description by @gr8gonzo.  The only thing I would add is that Fiddler is like an observer standing at your table watching what happens between you and the waiter.  Fiddler can't see into the kitchen area (server) at all.  Fiddler only sees what passes between you and the waiter.
0
SAbboushiAuthor Commented:
gr8gonzo
Thanks for the thorough response and answers to my question.  Much appreciated.

Thanks also for your input Dave-

With Regards-
Sam
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Web Servers

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.