Programmatically download the first page of a list of websites

Posted on 2008-09-30
Medium Priority
Last Modified: 2012-05-05
I have a list of 400 website URLs and site names.  I want to automatically download just the home page and name the folder by the site name.  Is there a tool or a way in VB.NET?
Question by:Whah
  • 2

Accepted Solution

alaran earned 1500 total points
ID: 22607336
You could use the HttpWebRequest class.
Loop though each of the URLs, make the request using the HttpWebRequest.Create method.
Then get the response and the response stream. You can then write the response stream fo file or whatever you want to do with the downloaded contents.

C# example, but should be easy to port:

foreach(string theUrl in myUrlList)
    HttpWebRequest request = (HttpWebRequest) HttpWebRequest.Create(theUrl);
    WebResponse response = request.GetResponse();
    System.IO.Stream contentStream = response.GetResponseStream();
    // Write the contents to file or something...

Expert Comment

ID: 22607345
You need to import the System.Net namespace by the way.

Featured Post

Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

This tutorial demonstrates one way to create an application that runs without any Forms but still has a GUI presence via an Icon in the System Tray. The magic lies in Inheriting from the ApplicationContext Class and passing that to Application.Ru…
Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
This video tutorial shows you the steps to go through to set up what I believe to be the best email app on the android platform to read Exchange mail.  Get the app on your phone: The first step is to make sure you have the Samsung Email app on your …
From store locators to asset tracking and route optimization, learn how leading companies are using Google Maps APIs throughout the customer journey to increase checkout conversions, boost user engagement, and optimize order fulfillment. Powered …

597 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question