Web Server vs. CGIs

Hi Experts again.
This one is quite special. How do web servers pass the parameters to any side applications (CGI, PHP etc.) and how does the sever catch the result from the CGIs? I know the CGIs simply write to console and the server must catch this. I also know that the CGIs read the system variables for the params from the server but it really seems strange to me. What if the cgi receives a file? Please, tell me if you have any experience.

Thank you,
Jakub
jakubklosAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

simonetCommented:
Since we're in the Delphi area, I'll consider Delphi-written CGIs.

Delphi CGI's to everything on the TWebResponse and TWebRequest objects. In your specific question, the object that does the job of receiving information from the Web server (such as IIS), is the TWebRequest object.

If you want to read, for example, a parameter of name USER from the server, you use the following syntax:

var
  usr : string;
begin
  usr := Request.ContentFields.Values['USER'];

The TWebModule, present in the C/S and Enterprise versions of Delphi, encapsulate a lot of what would be otherwise hard work.

In order to send data back to the server, Delphi web applications (either CGI or ISAPI), use the TWebResponse object, and that can be anything, form text to files.

Receiving a file using CGI is not as complicated as it seems. Rather than explaining how it's done, I've posted some code on how it's done using the TWebModule application.

Yours,

Alex


****************

program DownloadCGI;

{$APPTYPE CONSOLE}

uses
  WebBroker,
  CGIApp,
  DownloadUnit in 'DownloadUnit.pas' {WebModule1: TWebModule};

{$R *.RES}

begin
  Application.Initialize;
  Application.CreateForm(TWebModule1, WebModule1);
  Application.Run;
end.

******************

unit DownloadUnit;

interface

uses
  Windows, Messages, SysUtils, Classes, HTTPApp;

type
  TWebModule1 = class(TWebModule)
    procedure WebModule1WebActionItem1Action(Sender: TObject;
      Request: TWebRequest; Response: TWebResponse; var Handled: Boolean);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  WebModule1: TWebModule1;

implementation

{$R *.DFM}

uses
  Controls;

procedure TWebModule1.WebModule1WebActionItem1Action(Sender: TObject;
  Request: TWebRequest; Response: TWebResponse; var Handled: Boolean);
var
  Path, FileName,
  Counter         : String;
  Today, LastEver,
  Ever, LastToday : Integer;
  LastDate        : TDate;
  FS              : TFileStream;
  fp              : TextFile;
begin
  Path := ExtractFilePath(ParamStr(0));
  Path := Copy(Path,1,Length(Path)-8); // Matches both 'cgi-bin\' and 'scripts\'
  with Request.QueryFields do begin
    FileName := Path+'downloads\'+Values['FileName'];
    Counter := Path+'counters\Download'+ChangeFileExt(ExtractFileName(FileName),'')+'.txt';
  end;
  try
    // Write a new empty counter file if it doesn't exist
    if not FileExists(Counter) then begin
      AssignFile(fp,Counter);
      Rewrite(fp);
      WriteLn(fp,0);
      WriteLn(fp,Date);
      WriteLn(fp,0);
      CloseFile(fp);
    end;

    // Read the old counter values
    AssignFile(fp,Counter);
    Reset(fp);
    ReadLn(fp,LastEver);
    Ever := LastEver+1;
    ReadLn(fp,LastDate);
    ReadLn(fp,LastToday);
    if Date = LastDate then
      Today := LastToday+1
    else
      Today := 1;
    CloseFile(fp);

    // Write the new counter values
    AssignFile(fp,Counter);
    Rewrite(fp);
    WriteLn(fp,Ever);
    WriteLn(fp,Date);
    WriteLn(fp,Today);
    CloseFile(fp);

    with Response do begin
      FS := TFileStream.Create(FileName,fmOpenRead);
      CustomHeaders.Add('Content-Disposition=; filename='+ExtractFileName(FileName));
      ContentType := 'application/x-zip-compressed; name='+ExtractFileName(FileName);
      ContentLength := FS.Size;
      FS.Position := 0;
      SendResponse;
      SendStream(FS);
    end;
  except
    on E: Exception do
      Response.Content := E.Message;
  end;
  Handled := True;
end;

end.

0
sburckCommented:
listening...
0
intheCommented:
listening..
0
Cloud Class® Course: SQL Server Core 2016

This course will introduce you to SQL Server Core 2016, as well as teach you about SSMS, data tools, installation, server configuration, using Management Studio, and writing and executing queries.

jakubklosAuthor Commented:
Yes, thank you but I asked about the server's side of the story. I know how the CGIs work but not how the server posts and gets the data back from the CGI.
0
BigRatCommented:
When the http request comes in the server spawns a new process. This new process is the cgi script, which in all events is a program which will get executed. (A script is executed by the Shell program). On both Windows and Unix the new process inherits all resources of the parent process. Therefore passing environment variables is easy.
   On Unix systems the programs output is piped back into a server's file handle. The server reads bytes from this handle (until it is closed) and writes them into the socket obtained when the http connection was made. This action takes place in the spawned program, since the latter inherits all the resources of its parent (the Listner for incoming requests).
   On Windows the CreateProcess API procedure allows you to specify handles for standard input (std_in) and standard output (std_out), so when the script thinks it is writing to the console it isn't, but back into the server. On Windows file handles and socket handles are not equivalent, but that doesn't matter.
   
0
philipleighsCommented:
Hi,

Say you have an HTML form with a text box and a submit button.

If the form action is POST then the content of the text box becomes an environment variable in the CGI script.

If the form action is GET then the content of the text box is appended to the URL. .../cgi-bin/script.cgi?EditBox=WhatYouTyped

This is how it is for CGI scripts written in Perl at least. A typical CGI script in Perl will either put all environment variables into a hash, or will parse the URL into a hash. (A hash is a Perl type for a lookup table). I can give you Perl code if you want.

If you're passing a file, then use POST. The entire content becomes an environment variable. GET uses the URL which can have a limit on the length.

Is this what you mean? I don't know how the server sets up the environment variables.

Cheers,
Phil.
0
jakubklosAuthor Commented:
Yes, we're almost there. What I wonder is how will Windows handle a system variable which might be 2MB long for example. You know, the file uploading. Do you really think windows can handle that without any problem? If yes, there's nothing to solve :). I hope I'll be able to read from the handle. Any detailed description how to do that? Thank you so much for your effort and time. I think this might be useful also for other developers ;).
0
philipleighsCommented:
Hmm, I don't know too much about capacities and limits. Prehaps you should look in the Perl/CGI groups at EE. Maybe someone has asked a similiar question.

Cheers,
Phil.
0
simonetCommented:
CGI variable have a limit of 1024 bytes (1 kb)

Now, streams have no size limit.
0
MadshiCommented:
listening...
0
jakubklosAuthor Commented:
Ok, then. I've found out how it works. It's really no problem. You can pass the environment to the CreateProcess command and it works like a charm. If you somebody wants to know how, I'll tell you. The last thing I need is how to read from the console or how to redirect the handles. Please, help. Thanks.
0
philipleighsCommented:
Hi again,

I don't know if this will help, but it is a function that executes a console program and captures stdout and stderr into two string lists.

Cheers,
Phil.

>>>>>>>>>>>>>>>>>>>>
unit ConsolePipe;

interface

uses Windows, Classes, SysUtils;

procedure LaunchConsoleAndWait(ExeName: string; Args: string; WorkingDir: string; Output: TStringList; Errors: TStringList);

implementation

procedure LaunchConsoleAndWait(ExeName: string; Args: string; WorkingDir: string; Output: TStringList; Errors: TStringList);
  var sa: TSECURITYATTRIBUTES;
      si: TSTARTUPINFO;
      pi: TPROCESSINFORMATION;
      hPipeOutputRead: THANDLE;
      hPipeOutputWrite: THANDLE;
      hPipeErrorsRead: THANDLE;
      hPipeErrorsWrite: THANDLE;

      Res: Boolean;
      env: array [0..100] of Char;
      pWorkingDir: PChar;

      bTest: Boolean;
      dwNumberOfBytesRead: DWORD;
      szBuffer: array [0..256] of Char;
      Stream: TMemoryStream;

      CRCRLFPos: Integer;
  begin
    sa.nLength := sizeof(sa);
    sa.bInheritHandle := true;
    sa.lpSecurityDescriptor := nil;

    CreatePipe(hPipeOutputRead,
               hPipeOutputWrite,
               @sa,
               0);
    CreatePipe(hPipeErrorsRead,
               hPipeErrorsWrite,
               @sa,
               0);

    ZeroMemory(@env, SizeOf(env));
    ZeroMemory(@si, SizeOf(si));
    ZeroMemory(@pi, SizeOf(pi));
    si.cb := SizeOf(si);
    si.dwFlags := STARTF_USESHOWWINDOW or STARTF_USESTDHANDLES;
    si.wShowWindow := SW_HIDE;
    si.hStdInput   := 0;
    si.hStdOutput  := hPipeOutputWrite;
    si.hStdError   := hPipeErrorsWrite;
    pWorkingDir := nil;
    if WorkingDir <> '' then
      pWorkingDir := PChar(WorkingDir);

    if ExeName <> '' then
      Res := CreateProcess(
          PChar(ExeName),
          PChar(' ' + Args),
          nil,
          nil,
          true,
          CREATE_NEW_CONSOLE or NORMAL_PRIORITY_CLASS,
          @env,
          pWorkingDir,
          si,
          pi)
    else
      Res := CreateProcess(
          nil,
          PChar(Args),
          nil,
          nil,
          true,
          CREATE_NEW_CONSOLE or NORMAL_PRIORITY_CLASS,
          @env,
          pWorkingDir,
          si,
          pi);
    if not Res then
      begin
        CloseHandle (hPipeOutputRead);
        CloseHandle(hPipeOutputWrite);
        CloseHandle (hPipeErrorsRead);
        CloseHandle(hPipeErrorsWrite);
        raise Exception.Create('Unable to execute ' + ExeName);
        exit;
      end;

    CloseHandle(hPipeOutputWrite);
    CloseHandle(hPipeErrorsWrite);

    //Read output pipe
    Stream := TMemoryStream.Create;
    try
      while true do
        begin
          bTest:=ReadFile(
                hPipeOutputRead,
                szBuffer,
                256,
                dwNumberOfBytesRead,
                nil);
          if not bTest then
            begin
              break;
            end;
          CRCRLFPos := Pos(#13#13#10, szBuffer);
          while CRCRLFPos > 0 do
            begin
              szBuffer[CRCRLFPos - 1] := ' ';
              CRCRLFPos := Pos(#13#13#10, szBuffer);
            end;
          Stream.Write(szBuffer, dwNumberOfBytesRead);
        end;
      Stream.Position := 0;
      Output.LoadFromStream(Stream);
    finally
      Stream.Free;
    end;

    //Read error pipe
    Stream := TMemoryStream.Create;
    try
      while true do
        begin
          bTest:=ReadFile(
                hPipeErrorsRead,
                szBuffer,
                256,
                dwNumberOfBytesRead,
                nil);
          if not bTest then
            begin
              break;
            end;
          CRCRLFPos := Pos(#13#13#10, szBuffer);
          while CRCRLFPos > 0 do
            begin
              szBuffer[CRCRLFPos - 1] := ' ';
              CRCRLFPos := Pos(#13#13#10, szBuffer);
            end;
          Stream.Write(szBuffer, dwNumberOfBytesRead);
        end;
      Stream.Position := 0;
      Errors.LoadFromStream(Stream);
    finally
      Stream.Free;
    end;

    WaitForSingleObject (pi.hProcess, INFINITE);
    CloseHandle (pi.hThread);
    CloseHandle (pi.hProcess);
    CloseHandle (hPipeOutputRead);
    CloseHandle (hPipeErrorsRead);
  end;

end.
<<<<<<<<<<<<<<<<<<<
0
jakubklosAuthor Commented:
Wow, this is definitely IT. I didn't test it but it looks it's the one :). Thank you so much Phill.
0
philipleighsCommented:
OK great, let me know how you get on and if it does the trick.

Cheers,
Phil.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
jakubklosAuthor Commented:
Yes, it's completely working now. I've developed my own multi threaded web server which can process PHP and CGI now :). Thanks so much.
0
jakubklosAuthor Commented:
Tha :)
0
philipleighsCommented:
Excellent and thanks for the massive 2000 points!  :-)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Web Languages and Standards

From novice to tech pro — start learning today.