?
Solved

get the http response stream in c#

Posted on 2007-03-19
5
Medium Priority
?
772 Views
Last Modified: 2011-10-03
Hi all,

heres what ive been asked to do, I need to read the http responses of a website into a database, im working in c# and not to sure how to go about this one.

0
Comment
Question by:StephenLiversidge
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
5 Comments
 
LVL 29

Accepted Solution

by:
Gautham Janardhan earned 2000 total points
ID: 18747225
WebRequest req = null;
                  WebResponse rsp = null;
Stream rspStream = null,reqStream = null;
req = WebRequest.Create(strPath);
rsp = req.GetResponse();
                                    rspStream = rsp.GetResponseStream();      
if (rspStream != null)
                        {
                              // Load the XmlTextReader from the stream                                                      
                              XmlTextReader xmlread = null;
                              xmlread = new XmlTextReader (rspStream);

                              //SAVE THE RESPONSE XML assumed
                              XmlDocument doc = new XmlDocument();
                              doc.Load(xmlread);
0
 
LVL 29

Expert Comment

by:Gautham Janardhan
ID: 18747229
str path is the url and assumed t5hat the respnse contains an xml u can cahnge it to ur requirement...
0
 
LVL 143

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 18747268
here is a class I use in my project(s):

#region HTTP Request
      public class HTTPRequest
      {
            private string serverIp = "";
            private int serverPort = 0;
            private string serverUsername = "";
            private string serverPassword = "";
            
            public HTTPRequest(string ip,int port,string username,string password) {
                  serverIp = ip;
                  serverPort = port;
                  serverUsername = username;
                  serverPassword = password;
            }

            public string Submit(string servlet, string param)
            {                  
                  return Submit("/"+servlet+"?"+param);
            }

            public string Submit(string url)
            {
                  int tries = 0;
                  string s = "";

                              TcpClient tcp = new TcpClient();
                              tcp.Connect(serverIp,serverPort);
                              tcp.ReceiveTimeout = 20000;
                              s = SubmitTcp(tcp,Header(url));
                              tcp.Close();
                  return s;
            }

            private string Header(string url) {
                  string header = "";
                  UTF8Encoding Uuserpass = new UTF8Encoding();
                  string userpass = Convert.ToBase64String(Uuserpass.GetBytes(serverUsername+":"+serverPassword));
                  header = "GET "+url+" HTTP/1.1\r\nAuthorization: Basic "+userpass+"\r\n\r\n";
                  header += "Content-type: application/x-www-form-urlencoded\r\n";
                  header += "Content Length: "+url.Length+"\r\n";
                  header += "Connection: close\r\n\r\n";
                  return header;
            }

            private string SubmitTcp(TcpClient tcp,string header) {
                  string response = "";

                  ASCIIEncoding encode = new ASCIIEncoding();
                  byte[] b = encode.GetBytes(header);
                  Stream stream = tcp.GetStream();
                  stream.Write(b,0,b.Length);

                  byte[] ret = new byte[1000];
                  int k = 0;
                  while ((k=stream.Read(ret,0,1000)) > 0) {
                        for (int i=0;i<k;i++) response += (Convert.ToChar(ret[i]));
                  }
                  stream.Close();
                  return response;
            }
      }
      #endregion
0
 

Author Comment

by:StephenLiversidge
ID: 18747445


Hi, Im getting this message at the min, very frustrating

The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF

The url im trying to grab is www2.url.com though, would this make a difference?
0
 

Author Comment

by:StephenLiversidge
ID: 18755725
Thanks, everyone

All sorted now
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Having just graduated from college and entered the workforce, I don’t find myself always using the tools and programs I grew accustomed to over the past four years. However, there is one program I continually find myself reverting back to…R.   So …
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
Suggested Courses

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question