Extract the Page Source of URL as a String using VBA

Posted on 2011-05-01
Last Modified: 2012-05-11

I am trying to extract the HTML Page Source of a given URL as a String. I found the code below which works but for some reason doesn't work all the time (it gives various different errors sporadically).

Public Function gsGetPageSource(URLToCrawl As String) As String
Dim ie As Object

Set ie = CreateObject("InternetExplorer.Application")

With ie
.Navigate URLToCrawl
Do While .Busy And Not .ReadyState = 4

gsGetPageSource = IIf(StrComp(.Document.Title, "Cannot find server", vbTextCompare) = 0, vbNullString, .Document.Body.outerHTML)
End With

Set ie = Nothing

End Function

Open in new window

Question by:xyobi
    LVL 22

    Expert Comment

    by:Nico Bontenbal
    See comment 35465535 in this question:
    You'll need the function GetInternetResult in the Postalcode module of the attached file. You can call this function for example like this:

    LVL 11

    Accepted Solution

    try the attached;
    Sub tester()
        Set objFSO = CreateObject("Scripting.FileSystemObject")
        Set objHTTP = CreateObject("Msxml2.XMLHTTP")
        objHTTP.Open "GET", "", False
        strPageText = objHTTP.responseText
        MsgBox strPageText
    End Sub

    Open in new window


    Author Comment

    thydzik... you are a legend, thanks so much for your help

    nicobo... thanks for your solution, I tried it out and it looked interesting but im not sure if it does exactly as i want as i dont have the skills required to make the necessary changes

    Author Closing Comment


    Featured Post

    Why You Should Analyze Threat Actor TTPs

    After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

    Join & Write a Comment

    Displaying an arrayList in a listView using the default adapter is rarely the best solution. To get full control of your display data, and to be able to refresh it after editing, requires the use of a custom adapter.
    Entering a date in Microsoft Access can be tricky. A typo can cause month and day to be shuffled, entering the day only causes an error, as does entering, say, day 31 in June. This article shows how an inputmask supported by code can help the user a…
    This Micro Tutorial demonstrates in Microsoft Excel how to consolidate your marketing data by creating an interactive charts using form controls. This creates cool drop-downs for viewers of your chart to choose from.
    In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …

    734 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    23 Experts available now in Live!

    Get 1:1 Help Now