Solved

Efficiently downloading large number of webpages using Visual Basic 2008

Posted on 2007-11-27
1
156 Views
Last Modified: 2010-04-30
Im currently using this code to download binary data from about 2000 uri's. The code works fine, however, once it gets near the 600th request it starts to slow down considerably. Any help with optomizing my current method or a point towards a different one is greatly appreciated thanks.
Private Function getpiece(ByVal hash As String, ByVal piece As String)
 
        Dim txturi As String = (removed)
        Dim pieceuri As New Uri(txturi)
        Dim piecerequest As System.Net.WebRequest = System.Net.WebRequest.Create(pieceuri)
        Dim pieceresponse As System.Net.WebResponse = piecerequest.GetResponse()
        ' Get the stream containing content returned by the server.
        Dim dataStream As Stream = pieceresponse.GetResponseStream()
        ' Open the stream using a StreamReader for easy access.
        Dim reader As New System.IO.BinaryReader(dataStream)
        ' Read the content.
        Dim responseFromServer() As Byte = reader.ReadBytes(264144)
        Application.DoEvents()
        reader.Close()
        dataStream.Close()
        pieceresponse.Close()
        Return responseFromServer
 
    End Function

Open in new window

0
Comment
Question by:dead1ne
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
1 Comment
 
LVL 38

Accepted Solution

by:
PaulHews earned 500 total points
ID: 20360930
The only thing I can think of changing is to use

dataStream.Dispose()

instead of closing the stream.  But I'll see if anyone else has anything to add.
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction While answering a recent question (http://www.experts-exchange.com/Q_27402310.html) in the VB classic zone, I wrote some VB code in the (Office) VBA environment, rather than fire up my older PC.  I didn't post completely correct code o…
This article describes how to use a set of graphical playing cards to create a Draw Poker game in Excel or VB6.
Get people started with the process of using Access VBA to control Outlook using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Microsoft Outlook. Using automation, an Access applic…
This lesson covers basic error handling code in Microsoft Excel using VBA. This is the first lesson in a 3-part series that uses code to loop through an Excel spreadsheet in VBA and then fix errors, taking advantage of error handling code. This l…
Suggested Courses
Course of the Month10 days, 4 hours left to enroll

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question