Solved

Efficiently downloading large number of webpages using Visual Basic 2008

Posted on 2007-11-27
1
150 Views
Last Modified: 2010-04-30
Im currently using this code to download binary data from about 2000 uri's. The code works fine, however, once it gets near the 600th request it starts to slow down considerably. Any help with optomizing my current method or a point towards a different one is greatly appreciated thanks.
Private Function getpiece(ByVal hash As String, ByVal piece As String)
 

        Dim txturi As String = (removed)

        Dim pieceuri As New Uri(txturi)

        Dim piecerequest As System.Net.WebRequest = System.Net.WebRequest.Create(pieceuri)

        Dim pieceresponse As System.Net.WebResponse = piecerequest.GetResponse()

        ' Get the stream containing content returned by the server.

        Dim dataStream As Stream = pieceresponse.GetResponseStream()

        ' Open the stream using a StreamReader for easy access.

        Dim reader As New System.IO.BinaryReader(dataStream)

        ' Read the content.

        Dim responseFromServer() As Byte = reader.ReadBytes(264144)

        Application.DoEvents()

        reader.Close()

        dataStream.Close()

        pieceresponse.Close()

        Return responseFromServer
 

    End Function

Open in new window

0
Comment
Question by:dead1ne
1 Comment
 
LVL 38

Accepted Solution

by:
PaulHews earned 500 total points
ID: 20360930
The only thing I can think of changing is to use

dataStream.Dispose()

instead of closing the stream.  But I'll see if anyone else has anything to add.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

There are many ways to remove duplicate entries in an SQL or Access database. Most make you temporarily insert an ID field, make a temp table and copy data back and forth, and/or are slow. Here is an easy way in VB6 using ADO to remove duplicate row…
The debugging module of the VB 6 IDE can be accessed by way of the Debug menu item. That menu item can normally be found in the IDE's main menu line as shown in this picture.   There is also a companion Debug Toolbar that looks like the followin…
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Get people started with the utilization of class modules. Class modules can be a powerful tool in Microsoft Access. They allow you to create self-contained objects that encapsulate functionality. They can easily hide the complexity of a process from…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now