Solved

Programmatically download the first page of a list of websites

Posted on 2008-09-30
2
208 Views
Last Modified: 2012-05-05
I have a list of 400 website URLs and site names.  I want to automatically download just the home page and name the folder by the site name.  Is there a tool or a way in VB.NET?
0
Comment
Question by:Whah
  • 2
2 Comments
 
LVL 4

Accepted Solution

by:
alaran earned 500 total points
ID: 22607336
You could use the HttpWebRequest class.
Loop though each of the URLs, make the request using the HttpWebRequest.Create method.
Then get the response and the response stream. You can then write the response stream fo file or whatever you want to do with the downloaded contents.

C# example, but should be easy to port:

foreach(string theUrl in myUrlList)
{
    HttpWebRequest request = (HttpWebRequest) HttpWebRequest.Create(theUrl);
    WebResponse response = request.GetResponse();
    System.IO.Stream contentStream = response.GetResponseStream();
    // Write the contents to file or something...
}
0
 
LVL 4

Expert Comment

by:alaran
ID: 22607345
You need to import the System.Net namespace by the way.
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

This article explains how to create and use a custom WaterMark textbox class.  The custom WaterMark textbox class allows you to set the WaterMark Background Color and WaterMark text at design time.   IMAGE OF WATERMARKS STEPS Create VB …
1.0 - Introduction Converting Visual Basic 6.0 (VB6) to Visual Basic 2008+ (VB.NET). If ever there was a subject full of murkiness and bad decisions, it is this one!   The first problem seems to be that people considering this task of converting…
Along with being a a promotional video for my three-day Annielytics Dashboard Seminor, this Micro Tutorial is an intro to Google Analytics API data.
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question