Code to access data from website

Posted on 2010-01-06
Last Modified: 2013-11-19
Hi Experts.  I currently log onto a website where I collect data relating to usage of a data line.
I log on with a username and password, click a menu item and the data is presented on the web page.  All very standard stuff and the site does not have a security certificate.  I'm looking for code samples or ways to access this data automatically.   I'm thinking of an automated process of logging in with the username and password, getting to the data and then downloading it, automatically and at regular time intervals.  
Question by:PNRT
    LVL 12

    Expert Comment

    several tutorials exist on the internet. check them:
    LVL 13

    Expert Comment

    try this
    static void Main(string[] args)
                var request = WebRequest.Create("");
                request.Credentials = new NetworkCredential("username", "password");
                var response = request.GetResponse();
                var buffer = new byte[1024];
                var htmlPage = string.Empty;
                using (var dataStream = response.GetResponseStream())
                    using (var reader = new StreamReader (dataStream))
                        htmlPage = reader.ReadToEnd();

    Open in new window

    LVL 1

    Author Comment

    Damn it.  My fault entirelly, I forget to mention, would you perhaps have the VB.Net2 version.
    LVL 13

    Accepted Solution


    Sub DownloadWebPage()
    Dim htmlPage As String
    Dim website As String = ""
    Dim uid As String = "username"
    Dim pwd As String = "password"
    Dim fileSave As String = "c:\file.html"
    Dim request As WebRequest = WebRequest.Create(website)
    request.Credentials = New NetworkCredential(uid, pwd)
    Dim dataStream As Stream = request.GetResponse.GetResponseStream()
    Dim reader As StreamReader = New StreamReader(dataStream)
    htmlPage = reader.ReadToEnd
    Dim fs As FileStream = New FileStream(fileSave, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None)
    Dim writer As StreamWriter = New StreamWriter(fs)
    End Sub
    LVL 1

    Author Closing Comment

    A very helpful expert.   Thank you

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Find Ransomware Secrets With All-Source Analysis

    Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

    Have you tried to learn about Unicode, UTF-8, and multibyte text encoding and all the articles are just too "academic" or too technical? This article aims to make the whole topic easy for just about anyone to understand.
    Why do we like using grid based layouts in website design? Let's look at the live examples of websites and compare them to grid based WordPress themes.
    The viewer will learn how to count occurrences of each item in an array.
    The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.

    737 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    20 Experts available now in Live!

    Get 1:1 Help Now