We help IT Professionals succeed at work.

How do I get response from SendReqest?

hshliang
hshliang asked
on
Medium Priority
2,099 Views
Last Modified: 2013-11-20
I want to learn how to get the Response after sending a Post by SendRequest
I have a segment of Perl example below that I would like to convert to VC++ and MFC wininet functions:

---
#Start Utility functions
#-------------------------------------------------------------------------
sub processRequest
{
    my ($request) = @_;
    my $retVal = $ERROR;
    if (-e $DIR . $RESPONSE)
    {#if debugging on remove response file before processing new request
         unlink($DIR . $RESPONSE);
    }
    my $response = $UA->request($request,$DIR . $RESPONSE);
    if ($response->is_success)    
    {
         $retVal = processResponse();
    }
    else
    {
        if ($DEBUG == 1)
        {
            open(FL1,">$DIR$RESPONSE") || die "unable to open $DIR$RESPONSE for writing!\n";
            print FL1 $response->as_string;
            close(FL1);
        }
        $Messages = "Invalid or no response from server";
    }
    return $retVal;
}

sub processResponse
{
    my($line,$lline);
    my $retVal = $ERROR;
    open(FL1,"<$DIR$RESPONSE") || die "unable to open $DIR$RESPONSE for reading!\n";
    open(FL2,">$DIR$TMPFILE") || die "unable to open $DIR$TMPFILE for writing!\n";
    binmode(FL1);
    binmode(FL2);
    $line = <FL1>;
    while ($line)
    {
        if (!eof(FL1))
        {
            print FL2 $line;
        }
        $lline = $line;
        $line = <FL1>;
    }
    close(FL1);
    close(FL2);
    $retVal = processFooter($lline);
    if ($retVal == $SUCCESS)
    {
        if ($Filename eq "")
        {
            unlink($DIR . $TMPFILE);
        }
        else
        {
            rename($DIR . $TMPFILE,$DIR . $Filename);
        }
    }
    else
    {
        unlink($DIR . $TMPFILE);
        $Messages = "Invalid response from server";
    }
    if ($DEBUG != 1)
    {
       unlink($DIR . $RESPONSE);
    }
    return $retVal;
}

Please suggest someways I can read the RESPONSE. Do I use cookie or read the header returned?

Comment
Watch Question

Commented:
#include <afxinet.h>
//assumes server, port and URL names have been initialized
CInternetSession session("My Session");
CHttpConnection* pServer = NULL;
CHttpFile* pFile = NULL;
LPCTSTR pstrURL;
DWORD& dwServiceType;
CString& strServer;
CString& strObject;
INTERNET_PORT nPort;
try
{
   CString strServerName;
   INTERNET_PORT nPort;
AfxParseURL( pstrURL, dwServiceType, strServer, strObject, nPort );
   pServer = session.GetHttpConnection(strServerName, nPort);
   pFile = pServer->OpenRequest(CHttpConnection::HTTP_VERB_GET, strObject);
   pFile->AddRequestHeaders(szHeaders);

/*
pstrHeaders

A pointer to a string containing the header or headers to append to the request. Each header must be terminated by a CR/LF pair.

dwFlags

Modifies the semantics of the new headers. Can be one of the following:

HTTP_ADDREQ_FLAG_COALESCE   Merges headers of the same name, using the flag to add the first header found to the subsequent header. For example, "Accept: text/*" followed by "Accept: audio/*" results in the formation of the single header "Accept: text/*, audio/*". It is up to the calling application to ensure a cohesive scheme with respect to data received by requests sent with coalesced or separate headers.


HTTP_ADDREQ_FLAG_REPLACE   Performs a remove and add to replace the current header. The header name will be used to remove the current header, and the full value will be used to add the new header. If the header-value is empty and the header is found, it is removed. If not empty, the header-value is replaced.


HTTP_ADDREQ_FLAG_ADD_IF_NEW   Only adds the header if it does not already exist. If one exists, an error is returned.


HTTP_ADDREQ_FLAG_ADD   Used with REPLACE. Adds the header if it doesn’t exist.
dwHeadersLen

The length, in characters, of pstrHeaders. If this is -1L, then pstrHeaders is assumed to be zero-terminated and the length is computed.

str

A reference to a CString object containing the request header or headers to be added
*/

   pFile->SendRequest();
   pFile->QueryInfoStatusCode(dwRet);
   if (dwRet == HTTP_STATUS_OK)
   {
       UINT nRead = pFile->Read(szBuff, 1023);
       while (nRead > 0)
       {
           //read file...
       }
   }
   delete pFile;
   delete pServer;
}
catch (CInternetException* pEx)
{
   //catch errors from WinInet
}
session.Close();



regards jobs
CERTIFIED EXPERT
Author of the Year 2009

Commented:
The term 'response' can mean one of two things.  There is a response code supplied by the server.  For instance 200 means OK, 404 means page not found, etc.

But the real response from a POST is the body of the data.  In most cases, this is a HTML page -- tat is, easy toi read test.

Taking up where we left off in your previous question:

BOOL fRet;
DWORD dwRet;

fRet= pFile->SendRequest(
   gsHeaders, (LPVOID)(LPCTSTR)gsFormData, nLen );

pFile->QueryInfoStatusCode( dwRet );

//----- this line will help debug.  It shows all of the
//----- text from the headers returned by the host:

CString sInfoFromHost;
pFile->QueryInfo( HTTP_QUERY_RAW_HEADERS_CRLF, sInfoFromHost );

//--------------- next read the body text

CString sBodyText;

int nRead= 1024;
char szBuf[ 1025 ];
while ( nRead > 0 ) {
     nRead= pFile->Read(szBuf, 1024);  
     szBuf[nRead] = '\0';
     sBodyText += szBuf;
}

//---- now you can look at sBodyText in the debugger to see what you got

-==-=-=-=-=-=-=-
PERL is such an ugly language, I have no idea what it is doing.  It looks like it is saving the response to a temporary file than the renaming it to something.  But I can't tell.

Do you need a way to save the contents of the page to a file?  If so, do this:

CFile cFileOut( "c:\\temp\\test.htm", CFile::modeCreate  );
cFileOut.Write( (LPCSTR)sBodyText, sBodyText.GetLength() );
cFileOut.Close();

Now you can open c:\temp\test.htm in a browser or a text editor to look at it.

-- Dan

Author

Commented:
Thank you for all your suggestions.

Please try to enter this manually in Internet Explorer:

https://tlpd1.moh.hnet.bc.ca/TeleplanBroker?ExternalAction=AsignOn&username=user&password=password
(as one line of course)

You will get this LINE as the return page:

#TID=null;Result=FAILURE:user;Filename=;Msgs=LDAP logon failure: user is not authorized;

This is what I call the 'RESPONSE'.
I have tried the GetHttpConnection with [tlpd1.moh.hnet.bc.ca] to Open the URL
Note [https://tlpd1.moh.hnet.bc.ca] will fail.
//Get Pass the Certificate Authorization request(see below for the segment to by-pass CA request)
//The try to call the request as POST
phttpSFile = m_pInetConnect->OpenRequest(CHttpConnection::HTTP_VERB_POST,
     m_strPath,
     NULL, 1, NULL, NULL,
     INTERNET_FLAG_DONT_CACHE |
     INTERNET_FLAG_IGNORE_CERT_CN_INVALID |
     INTERNET_FLAG_IGNORE_CERT_DATE_INVALID |
     INTERNET_FLAG_SECURE           //for SSL security
      );
Then set
CString sHeader = "Accept: text/*\r\n"
     "User-Agent: B2001API 1.0\r\n"
     "Cookie: cookies.txt\r\n"
     "Content-Type: Application/x-www-form-urlencode\r\n";
and set
CString Action = "ExternalAction=AsignOn&username=name&password=password";
I send the request as
bSendRequest = phttpSFile->SendRequest(sHeader,(LPVOID)(LPCTSTR)Action,Action.GetLength());
I get the return code as
phttpSFile->QueryInfoStatusCode(dwStatusCode)
(It returns OK with 200)
Then I read the Header which is:
POST /TeleplanBroker HTTP/1.1
Accept: text/*
User-Agent: B2001API 1.0
Content-Type: Application/x-www-form-urlencode
Host: tlpd1.moh.hnet.bc.ca
Content-Length: 61
Cache-Control: no-cache
Then I read the body with
phttpSFile->Read()
But I got a html web page with a message telling me : ExternalAction not recognised!!!
Question is : How do I get the line
#TID=null;Result=FAILURE:user;Filename=;Msgs=LDAP logon failure: user is not authorized;
as I have seen on IE????????? As simple as that!!!
That is what I want to find out. Please tell me HOW
Try the manual entry yourself and you will see the line. Somehow I cannot get it with MFC functions.
Please help!!
Henry

////// Addtional Information to explain how to bypass CA request /////
First OpenRequest with a "Get" then phttpSFile->SendRequest(NULL, 0, 0, 0);
an Error will be thrown to show
ERROR_INTERNET_INVALID_CA or
ERROR_INTERNET_CLIENT_AUTH_CERT_NEEDED
Get the INTERNET_OPTION_SECURITY_FLAGS with  InternetQueryOption
then add SECURITY_FLAG_IGNORE_UNKNOWN_CA
and use InternetSetOption to set the option

CERTIFIED EXPERT
Author of the Year 2009
Commented:
This code will obtain the desired results.  I use the pFile->ErrorDlg( ) fn, but you should be able to patch in the SECURITY_FLAG_IGNORE_UNKNOWN_CA as you described.

#include <afxinet.h>

void CDlgApp03::OnPbTest()
{
     CString sURL, sInfoFromHost,sBodyText,sHeaders, sFormData;
     sHeaders=
          "Accept: text/*\r\n"
          "User-Agent: Hack-o-Matic ver 0.01\r\n"
          "Content-Type: application/x-www-form-urlencoded\r\n"
     ;

     sFormData="ExternalAction=AsignOn&username=user&password=password";

     sURL=  "https://tlpd1.moh.hnet.bc.ca/TeleplanBroker";

     DWORD   dwServiceType;
     CString sObjectName, sServerName;
     USHORT nPort;

     AfxParseURL( sURL, dwServiceType, sServerName, sObjectName, nPort );

     CInternetSession session("My Agent" );

     CHttpConnection* pConnection =
          session.GetHttpConnection(
               sServerName, nPort,
               0,0 //m_sUser, m_sPswd
     );

     CHttpFile* pFile= pConnection->OpenRequest(
          "POST", // CHttpConnection::HTTP_VERB_POST,
          sObjectName,
          NULL, 1, NULL, NULL,
          INTERNET_FLAG_DONT_CACHE |
          INTERNET_FLAG_IGNORE_CERT_CN_INVALID |
          INTERNET_FLAG_IGNORE_CERT_DATE_INVALID |
          INTERNET_FLAG_SECURE           //for SSL security
     );

     DWORD dwErr;
     BOOL fRet;
     int nLen= sFormData.GetLength();

     try{
          BOOL fResult= pFile->SendRequest( sHeaders, (LPVOID)(LPCSTR)sFormData, nLen );
     }
     catch( CInternetException* e ) {
          dwErr= e->m_dwError;
          // MessageBox( GetInetErrText(dwErr) ); // errors are in WININET.H
     }
     //------------------------------------------- you are getting this error
     if ( dwErr == ERROR_INTERNET_INVALID_CA ) {
          DWORD dwRet= pFile->ErrorDlg(
               this,
               ERROR_INTERNET_INVALID_CA,
               FLAGS_ERROR_UI_FLAGS_GENERATE_DATA | FLAGS_ERROR_UI_FLAGS_CHANGE_OPTIONS,
               NULL
          );
          dwErr= 0;;
          fRet= pFile->SendRequest( sHeaders, (LPVOID)(LPCTSTR)sFormData, nLen );
             pFile->QueryInfoStatusCode( dwRet );
     }
     DWORD dwRet;
     pFile->QueryInfoStatusCode( dwRet );
     //------------ TBD:  check dwRet... should be 200
     pFile->QueryInfo( HTTP_QUERY_RAW_HEADERS_CRLF, sInfoFromHost );

     int nRead= 1024;
     char szBuf[ 1025 ];
     while ( nRead > 0 ) {
          nRead= pFile->Read(szBuf, 1024);  
          szBuf[nRead] = '\0';
          sBodyText += szBuf;
     }

     CFile cFileOut( "c:\\temp\\test.htm", CFile::modeCreate | CFile::modeWrite    );
     cFileOut.Write( (LPCSTR)sBodyText, sBodyText.GetLength() );
     cFileOut.Close();
}

Author

Commented:
Dear Dan,
You know what, I am really surprised, your program works so well.
I try to imitate everything you did, my stupid program still doesn't work!
I can easily tell by the header returned. My program only return the same things I send. Yours return the right stuff.
The only thing I can think of that is different is that my program is written in separate threads (to prepare to become a dll).

The GetHttpConnection, the bypass CA, and the final Post (OpenRequest and SendRequest) are in separate calls.

Could there be a problem to that???

I am going to give you credit anyway, but if you can think of the reason (especially because of my program is in separate threads) please drop me a line.
I am going to redo my whole program (Gee!) using your program, but keep it in a separate thread and see what's wrong.

Thank you for showing me that it is my program that is in trouble.

Henry
CERTIFIED EXPERT
Author of the Year 2009

Commented:
I went through a number of false starts before I got that to work.  The clue is in the dwErr code that you get back in the exception catch().  The error code is usually pretty self-explanitory (or you can look it up in MSDN for some details).   One error indicated an internal problem with the SSL layer.  I kept getting that error until I rebooted and then never got it again.

>>are in separate calls.  Could there be a problem to that???

There should be no problem, as long as you instantiate and use the session and connection objects correctly.

But as you can see, there is no need to break the sequence into separate calls since the code is pretty small.  In my production code, I have written a "GetFile" function... I just pass in some relevant info and it returns the response text.  That function looks a lot like the sample code I provided in my last post.  

Incidently, I often run up to 9 separate queries, each on its own thread.  There is no problem multi-threading as long as you avoid global variables and never directly interact with the U/I (just post progress messages, etc.).

-- Dan