Using the WebClient to post a form and screen scrape some data (not working)

Hi,

I'm trying to build an application that accepts a user name and password for MySpace and scrapes some of the user meta data from the user account. The intended use of this program is legal and legitimate.

Ideally I need to write the code in ASP.NET (C#), but if you have something like this you've done in PHP or some other language I could use that as a sample and convert it to asp.net.

.NET has a couple namespaces that should easily accomplish this. They are the WebRequest and WebClient classes (System.Net.HttpWebRequest, System.Net.WebClient). I tried using this code,but when the form submits, even though the user credentials are correct it returns a page that says "You must login!". Its obviously not working:

        string email = "jenni@myspace.com";
        string password = "secretPassword";
        string formUrl = "http://login.myspace.com/index.cfm?fuseaction=login.process";
       
         WebClient client = new WebClient();
        Stream stream;
        StreamReader streamReader;
        System.Collections.Specialized.NameValueCollection collection =
            new System.Collections.Specialized.NameValueCollection();

        collection.Add("email",email);
        collection.Add("password",password);
        collection.Add("loginbutton", "loginbutton");

        client.QueryString = collection;
        stream = client.OpenRead(formUrl);
        streamReader = new StreamReader(stream);

        Response.Write(streamReader.ReadToEnd());

        streamReader.Close();
        stream.Close();

Maybe I'm missing a parameter or maybe its the method of post I'm using? I dunno how to change it form post/get , or if that would even make difference.

Thanks!Start Free Trial