Solved

Sending and getting data from a remote web service with request and response objects

Posted on 2011-09-29
20
552 Views
Last Modified: 2012-05-12
I have been reading about how to use HttpWebRequest and HttpWebResponse with vb.net, but all that is getting me is confused.

Perhaps someone can explain it to me from a both a high and more detailed level?  My goal is for my vb.net web service to send an account number to a vendor's web service and get back the demographics on that account.

(Cannot believe in this day and age I cannot just a set a reference to it, but unfortunately the vendors site does not work that way.)

Thanks!

0
Comment
Question by:codefinger
  • 9
  • 8
  • 3
20 Comments
 
LVL 2

Expert Comment

by:EndersDev
ID: 36818658

HttpWebRequest is asking a remote service for something. There may or may not be credentials, encryption etc. But your software uses it to ask the other server to do something.

HttpWebResponse is what the remote server gives back. It may be saying "Take a hike", it may be as simple as a success code or it may return data. That all depends on the request.

If you're building a request, you need to include things like Method (e.g.: POST) and Content Type (e.g.: url encoded form data) etc. If the service doesn't recognize these things, it might not know what to do and could return any number of failure codes.

Let's say I want to talk to PayPal... (This is C#, sorry I don't have time to convert)


 
// Create a secure request object to PayPal
            HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://www.paypal.com/cgi-bin/webscr");

            req.Method = "POST";
            req.ContentType = "application/x-www-form-urlencoded";
            byte[] param = Request.BinaryRead(HttpContext.Current.Request.ContentLength);
            string strRequest = Encoding.ASCII.GetString(param);
            strRequest += "&cmd=_notify-validate";
            req.ContentLength = strRequest.Length;

   

            //Send the request to PayPal and get the response
            StreamWriter streamOut = new StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII);
            streamOut.Write(strRequest);
            streamOut.Close();

            // Read the response
            StreamReader streamIn = new StreamReader(req.GetResponse().GetResponseStream());
            string strResponse = streamIn.ReadToEnd();
            streamIn.Close();

Open in new window


So, we created a request and configured it. Then we sent it out and closed the session. Finally we read the response in and closed that session. What we do with what is returned depends wholly on the content.
0
 
LVL 2

Expert Comment

by:EndersDev
ID: 36818668
Converted to VB.NET (I don't have time for a manual conversion so I hope this works. The C# is tested though so VB shouldn't be too far from working if it isn't already)
' Create a secure request object to PayPal
Dim req As HttpWebRequest = DirectCast(WebRequest.Create("https://www.paypal.com/cgi-bin/webscr"), HttpWebRequest)

req.Method = "POST"
req.ContentType = "application/x-www-form-urlencoded"
Dim param As Byte() = Request.BinaryRead(HttpContext.Current.Request.ContentLength)
Dim strRequest As String = Encoding.ASCII.GetString(param)
strRequest += "&cmd=_notify-validate"
req.ContentLength = strRequest.Length



'Send the request to PayPal and get the response
Dim streamOut As New StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII)
streamOut.Write(strRequest)
streamOut.Close()

' Read the response
Dim streamIn As New StreamReader(req.GetResponse().GetResponseStream())
Dim strResponse As String = streamIn.ReadToEnd()
streamIn.Close()

Open in new window

0
 

Author Comment

by:codefinger
ID: 36818785
Okay, this is exactly like what I see on the NET when I Google and its also what I am just not getting.  

Lets say you want to send your userid and password to PayPal  and PayPal will respond by telling you whether or not the userid is for a valid PayPal user.  Lets say it will say something like "User is not a registered PayPal user" or "User is a valid PayPal user" or "Invalid Password".

Where exactly in the request object do you provide your userid and password?  Where exactly in the response object do you find PayPal's response?  And if nothing works properly at all, where in the response object do you find "failure codes"?

0
 
LVL 2

Expert Comment

by:EndersDev
ID: 36818867

The reason you don't see anything as specific as that is it all depends on the request you are performing. I did include what you are asking for, but perhaps I should have elaborated. It's fairly simple.


strRequest = strRequest & "&cmd=_notify-validate"

What I'm doing there is adding to the url. Depending on the request I might add any number of parameters. This is PayPal specific of course, and I'm building the URL that the request will be made to.

Remember, the url in this case is given here:
WebRequest.Create("https://www.paypal.com/cgi-bin/webscr" ...
I encode, then add my parameters and shoot it off.


The response is xml. It is PayPal specific and PayPal tells me what the parameters are.I have a method to parse the xml response and do whatever the app needs to do, but this is all specific to what I am doing, so it won't help you much.




0
 

Author Comment

by:codefinger
ID: 36869636
I suppose it is my unfamiliarity with html and form interaction that is making this difficult for me.

Perhaps if you could provide the same information in the form of a function:

Function IsPayPalUser(str_userid as string,str_password as string,byref str_errmsg as string) as Boolean



End Function

(BTW, please don't say its simple.  I am feeling pretty stupid already...<idotici grin>)
0
 
LVL 83

Expert Comment

by:CodeCruiser
ID: 36891228
HTTPWebRequest may not work with paypal this way as paypal uses SSL. Paypal has its API though.

I use the following code to login to a page programmatically.

 
Dim req As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://example.com/login.aspx?ReturnUrl=%2f")
        Dim postdata As String = "__LASTFOCUS=&__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=%2FwEPDwUKMTA3NDI3MzAyNA9kFgJmD2QWAmYPZBYCAgMPZBYCAgcPZBYCAgMPZBYCAgEPZBYCZg9kFgICDQ8QDxYCHgdDaGVja2VkaGRkZGQYAQUeX19Db250cm9sc1JlcXVpcmVQb3N0QmFja0tleV9fFgEFJmN0bDAwJFBsYWNlSG9sZGVyTWFpbiRsb2dpbiRSZW1lbWJlck1lvPiyr2LE1B28XNKgcFf5rSQWNyk%3D&__EVENTVALIDATION=%2FwEWBgKei97VCwLdyMakAQLE96mtBQLLtsPBAgLkkP7MCgK%2FlZyyB%2BablFFsF1i79MXnSdLBcM8QsHpo&ctl00%24PlaceHolderMain%24login%24UserName=username&ctl00%24PlaceHolderMain%24login%24password=password&ctl00%24PlaceHolderMain%24login%24login=Sign+In&__spDummyText1=&__spDummyText2="
        req.Method = "POST"
        req.ContentType = "application/x-www-form-urlencoded"
        Dim bytedata As Byte() = System.Text.Encoding.UTF8.GetBytes(postdata)
        req.ContentLength = bytedata.Length
        req.CookieContainer = New CookieContainer
        req.AllowAutoRedirect = True

        Dim str As System.IO.Stream = req.GetRequestStream
        str.Write(bytedata, 0, bytedata.Length)
        str.Close()

        Dim res As HttpWebResponse = req.GetResponse

Open in new window



Once I run this code, I can use the cookie container of the request object and attached it to all the subsequent requests to other pages and I will be authenticated.
0
 
LVL 2

Accepted Solution

by:
EndersDev earned 500 total points
ID: 36892756
@CodeCruiser > Since we use HttpWebRequest, I can promise it does work with PayPal. :)

@codefinger > If I gave you a function for user information using PayPal as an example it would not help with anything other than PP. First, understand that the URL is going to point to the service and method you want to access, then you assign parameters according to the contract of that method. When PayPal gets your POST, they will look for the parameters that concern them and ignore everything they don't recognize. So you could add other parameters as well, that mean something to you, and get that on the return for your own processing needs. For instance if you had an out of process request and you needed it to return the orderID for your system... but I digress. I think your confusion is simply due to anticipating the process being more complicated than it is. You're building a url with parameters, you have to encode that for the post operation and write out to the stream appropriately, but that code won't change much for basic stuff. So you're just setting up the request with the URL addressing the service and method you wish to access, then strRequest adds necessary params.


Let's try another example and perhaps you'll make more sense of it learning by difference.

 string url = "http://localhost:63806/WebRequests/YahooCheckInventory.aspx";
        HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
        req.Method = "POST";
        req.ContentType = "application/x-www-form-urlencoded";
        string strRequest = ".catalog=yhst-15098436415377&.id=_fake_yahoo_item_&.code=_fake_yahoo_code_";
        req.ContentLength = strRequest.Length;
        StreamWriter streamOut = new StreamWriter(req.GetRequestStream());
        streamOut.Write(strRequest);
        streamOut.Close();
        StreamReader streamIn = new StreamReader(req.GetResponse().GetResponseStream());
        string strResponse = streamIn.ReadToEnd();
        streamIn.Close();

So, this is actually test code. I have a test service on localhost. But the parameters mimic Yahoo.
".code" is a Yahoo param. I assign the value (in this case a bogus one) as a url, because it IS a url.

http://www.experts-exchange.com/Programming/Languages/.NET/Visual_Basic.NET/Q_27362498.html?cid=1573#a36891228

The page I'm on right now is assigning a "cid", when I talk to Yahoo I'm assigning a ".code". If there was a Yahoo function to get a user's name, and I wanted that in the response, I could add that parameter with the appropriate value. But Yahoo may call that parameter "userid" and accept an order Id as input (e.g.: ...&userid=3836202...) while PayPal may call it "username" and expect an email address to identify which user I want info about.

So... your function... in hypothetical terms (because it all depends on what methods a service has available and what they expect as parameters)

Function IsPayPalUser(str_userid as string,str_password as string,byref str_errmsg as string) as Boolean

' Create a secure request object to PayPal
Dim req As HttpWebRequest = DirectCast(WebRequest.Create("https://www.paypal.com/cgi-bin/webscr"), HttpWebRequest)

req.Method = "POST"
req.ContentType = "application/x-www-form-urlencoded"
Dim param As Byte() = Request.BinaryRead(HttpContext.Current.Request.ContentLength)
Dim strRequest As String = Encoding.ASCII.GetString(param)
strRequest += "&userid=" & _userid & "&password=" & _password
req.ContentLength = strRequest.Length



'Send the request to PayPal and get the response
Dim streamOut As New StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII)
streamOut.Write(strRequest)
streamOut.Close()

' Read the response
Dim streamIn As New StreamReader(req.GetResponse().GetResponseStream())
Dim strResponse As String = streamIn.ReadToEnd()
streamIn.Close()

' Send xml response to our method to determine if PayPal acknowledges this user
Return IsAuthenticated(strResponse)

End Function



Notice the strRequest string. That's where I set up any variables PayPal has outlined for making requests or inputing data.
0
 

Author Comment

by:codefinger
ID: 36893848
Getting closer, but some of the code you provided does not cooperate if called from another web service.

Dim param As Byte() = Request.BinaryRead(HttpContext.Current.Request.ContentLength)

"Request is not declared"

I have tried alternatives, but so far, no joy.

0
 

Author Comment

by:codefinger
ID: 36894054
Never mind that last comment...got this working:

Dim param As Byte() = HttpContext.Current.Request.BinaryRead(HttpContext.Current.Request.ContentLength)

0
 
LVL 2

Expert Comment

by:EndersDev
ID: 36894069
Yes, depending on where you place the code you may have to get the context. My apologies.
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 

Author Comment

by:codefinger
ID: 36894493
Based on what you have told me so far, I was able to put together the following function:
 
Public Function PatientIsRegisteredWithEpic(ByRef ptnt As DOBJ.InputParameters.PatientInputParms) As Boolean
            Dim retval As Boolean = True

            Dim str_epic_key As String = Nothing
            

            Dim str_req_bldr As StringBuilder = Nothing

            'epiID = "E110"

            'epiType = "0"

            'retType = "-1"

            'lppID = "132006"


            Dim epiID As String = "E110"
            Dim epiType As String = "0"
            Dim retType As String = "-1"
            Dim lppID As String = "132006"


            Try

                str_epic_key = ConfigurationManager.AppSettings("EPIC_WEB_KEY")

                'Prepare web request
                Dim hrreq As HttpWebRequest = DirectCast(WebRequest.Create("https://mychartn.ochsner.org/poc/webservice.asp"), HttpWebRequest)
                hrreq.Method = "POST"
                hrreq.ContentType = "application/x-www-form-urlencoded"
                Dim param As Byte() = HttpContext.Current.Request.BinaryRead(HttpContext.Current.Request.ContentLength)
                Dim strRequest As String = Encoding.ASCII.GetString(param)

                hrreq.ContentLength = strRequest.Length
                hrreq.KeepAlive = False
                hrreq.ReadWriteTimeout = 600000
                hrreq.Timeout = 600000



                strRequest = strRequest & "<?xml version=""1.0""?><epicRequest version=""1.0""><key>test3</key><request name=""ProgPoint""><params><ppoint id=""" & lppID & """><input>" & epiID & "</input><input>" & epiType & "</input><input>" & retType & "</input></ppoint></params></request></epicRequest>"

                'Send the request to vendor and get the response
                Dim streamOut As New StreamWriter(hrreq.GetRequestStream(), System.Text.Encoding.ASCII)
                streamOut.Write(strRequest)
                streamOut.Close()

                ' Read the response
                Dim streamIn As New StreamReader(hrreq.GetResponse().GetResponseStream())
                Dim strResponse As String = streamIn.ReadToEnd()
                streamIn.Close()





            Catch ex As Exception

                Me.LastException.MESSAGE = ex.Message
                If Not ex.InnerException Is Nothing Then
                    Me.LastException.DETAIL = ex.InnerException.Message
                End If
                retval = False


            End Try

            Return retval


        End Function

Open in new window


It is incomplete, but at least I have a starting point, and I am starting to catch on.

There is an exception thrown from this line:
 streamOut.Close()

The exception is: The request was aborted.  The request was cancelled.
The inner exception is "Cannot close stream until all bytes are written.", thus
the additional code I added to extend the timeout values, but it did not fix the
problem.

When I examined the strrequest string in the debugger just before it was sent,  I saw that it included data fields and values from the calling page after the binary code and before the xml values I was trying to send.  Is that normal?  (To be clear, I am not trying to submit a form, since to do so would mean nothing to the vendor.)

Please advise.

Thanks in advance.
0
 
LVL 2

Expert Comment

by:EndersDev
ID: 36894512

First make sure you are correctly calculating your content length. If certain that is correct, streamOut.Flush() before Close()

You can try setting the AllowWriteStreamBuffering prop to true

0
 

Author Comment

by:codefinger
ID: 36894639
Yep, that was that problem.   Content length was determined before I had finished setting up my string.
Moved if below the final string and got all the way to

Dim streamIn As New StreamReader(hrreq.GetResponse().GetResponseStream())

before the next exception.   In this case, the exception is:

"The remote server returned an error: (500) Internal Server Error."

However, I get the exact same error if I try to browse to the site, so I will have to do some more research to determine if there is actually something wrong with the site or there is something wrong with my code.

Your help has gotten me much further than I could have ever figured out on my own.  This  is one of those times it really irks me that I cannot award more than 500 pts.

0
 
LVL 2

Expert Comment

by:EndersDev
ID: 36894645
Glad I can help. This community has helped me out a lot so I like to give back when I can.
0
 
LVL 83

Expert Comment

by:CodeCruiser
ID: 36894685
@EndersDev
You mean you are able to paste username and password to paypal site in plain text?
0
 
LVL 2

Expert Comment

by:EndersDev
ID: 36894691
@CodeCruiser No, but as I wrote those fields are hypothetical. I meant, you can certainly access ssl using HttpWebRequest etc
0
 
LVL 83

Expert Comment

by:CodeCruiser
ID: 36894719
>I meant, you can certainly access ssl using HttpWebRequest etc
How does it work? Do you download the certificate and encrypt the form data yourself before sending out?
0
 
LVL 2

Expert Comment

by:EndersDev
ID: 36894745
No, you its just a normal post. But the object is aware of the secure pipe. I'm just saying its transparent to hit a secure server. In some cases you're sending credentials and retrieving a session cookie or something similar, but you don't need to do anything special. We have a secure server and devs accessing remote services don't need to know that. We expose the methods. But that is a whole separate topic.
0
 

Author Comment

by:codefinger
ID: 36911776
Just to let you know this question is not inactive...I am still working with the server administrators to determine why my post request is getting this response...still not sure whether the problem is the web service or my code (though I strongly suspect it is the former).

.When I am sure of the answer to that I will come back to this question and award the points.
0
 

Author Closing Comment

by:codefinger
ID: 36999980
Ender's comments were all on target.  Remaining problems were with the server set up.

Thanks!
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
Why do we like using grid based layouts in website design? Let's look at the live examples of websites and compare them to grid based WordPress themes.
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now