Having a program log into a website

Posted on 2006-05-16
Last Modified: 2010-04-23

I'm writing a personal news aggregator that pulls press releases from some of my favorite news websites, but I've come across a problem: some websites require you to log in before accessing their content., for example, will tell me the URL of a story via their RSS feed, but when I go to the address I will only see the story if I'm logged in.  Otherwise, I see some generic login page.

I have a function GetHTML(url) that grabs the HTML from any url, but if I try to do that on a businesswire page, I just get the HTML from the login page.  Is there any way I can have my program "log in" so that when it retrieves the HTML it actually gets page containing the article I'm looking for and not the login page?  I'm assuming the "logged in" status is done using cookies... can the VB internet functions have their own cookies?

Here's a sample URL from businesswire that will only show you the story if you're logged in:

Question by:quamusa
    1 Comment
    LVL 3

    Accepted Solution


    Using HTTPPost method, you can POST the login credentials to that page and mimick user behaviour.

    Hope this helps


    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    IT, Stop Being Called Into Every Meeting

    Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

    1.0 - Introduction Converting Visual Basic 6.0 (VB6) to Visual Basic 2008+ (VB.NET). If ever there was a subject full of murkiness and bad decisions, it is this one!   The first problem seems to be that people considering this task of converting…
    The ECB site provides FX rates for major currencies since its inception in 1999 in the form of an XML feed. The files have the following format (reducted for brevity) (CODE) There are three files available HERE (…
    In this sixth video of the Xpdf series, we discuss and demonstrate the PDFtoPNG utility, which converts a multi-page PDF file to separate color, grayscale, or monochrome PNG files, creating one PNG file for each page in the PDF. It does this via a c…
    In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor ( If you're interested in additional methods for monitoring bandwidt…

    794 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    17 Experts available now in Live!

    Get 1:1 Help Now