Data scraping from the website

Posted on 2007-10-16
Last Modified: 2008-01-09
Hi Experts, I wanted to scrape the data from the website I want to extract the data from the Today's Calendar section of the following columns: Date, Time, Currency, New description, high impact to the Excel spreadsheet.

Please advice me on to code this in VB.NET.

Question by:bpyeo
    LVL 12

    Expert Comment

    This need some homework. but if you have have enough time you can do yourself. Just study the HttpWebRequest and HttpWebRespose classes which are defined in System.Net.
    There is lot of articles in internet, best of luck.

    Author Comment

    Any website that provides an example code for the purpose? Please let me know.
    LVL 12

    Accepted Solution

    Exact source code you may not get, but you can search EE and google for articles on HttpWebRequest and HttpWebResponse classes. They used send requests to websites and get responses from them. Once you got website html code into your application, you have to play with it to get data as u required.

    For examle here is (C# - easy to convert) function will get html string from your site.

    private String PostData(string url)
                String result = "";
                String strPost = "If any Paramerets add here";
                StreamWriter myWriter = null;

                HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);
                objRequest.Method = "POST";
                objRequest.ContentLength = strPost.Length;
                objRequest.ContentType = "application/x-www-form-urlencoded";

                    myWriter = new StreamWriter(objRequest.GetRequestStream());
                catch (Exception e)
                    return e.Message;

                HttpWebResponse objResponse = (HttpWebResponse)objRequest.GetResponse();
                using (StreamReader sr =
                   new StreamReader(objResponse.GetResponseStream()))
                    result = sr.ReadToEnd();

                    // Close and clean up the StreamReader
                return result;
    LVL 1

    Expert Comment

    Forced accept.

    EE Admin

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    How to improve team productivity

    Quip adds documents, spreadsheets, and tasklists to your Slack experience
    - Elevate ideas to Quip docs
    - Share Quip docs in Slack
    - Get notified of changes to your docs
    - Available on iOS/Android/Desktop/Web
    - Online/Offline

    Well, all of us have seen the multiple EXCEL.EXE's in task manager that won't die even if you call the .close, .dispose methods. Try this method to kill any excels in memory. You can copy the kill function to create a check function and replace the …
    Microsoft Reports are based on a report definition, which is an XML file that describes data and layout for the report, with a different extension. You can create a client-side report definition language (*.rdlc) file with Visual Studio, and build g…
    Need more eyes on your posted question? Go ahead and follow the quick steps in this video to learn how to Request Attention to your question. *Log into your Experts Exchange account *Find the question you want to Request Attention for *Go to the e…
    Hi everyone! This is Experts Exchange customer support.  This quick video will show you how to change your primary email address.  If you have any questions, then please Write a Comment below!

    761 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    7 Experts available now in Live!

    Get 1:1 Help Now