Having a vb.net program log into a website


I'm writing a personal news aggregator that pulls press releases from some of my favorite news websites, but I've come across a problem: some websites require you to log in before accessing their content.

BusinessWire.com, for example, will tell me the URL of a story via their RSS feed, but when I go to the address I will only see the story if I'm logged in.  Otherwise, I see some generic login page.

I have a function GetHTML(url) that grabs the HTML from any url, but if I try to do that on a businesswire page, I just get the HTML from the login page.  Is there any way I can have my program "log in" so that when it retrieves the HTML it actually gets page containing the article I'm looking for and not the login page?  I'm assuming the "logged in" status is done using cookies... can the VB internet functions have their own cookies?

Here's a sample URL from businesswire that will only show you the story if you're logged in:

Who is Participating?
wildcard76Connect With a Mentor Commented:

Using HTTPPost method, you can POST the login credentials to that page and mimick user behaviour.

Hope this helps

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.