Solved

Read buffer

Posted on 2015-01-03
3
80 Views
Last Modified: 2015-01-19
I'm trying to read a file in segments and pass it into the function uploadFile(), however when I try and read large files I get memory issues:-

        Dim uploadBytesSize As Integer = 200000

        Dim sr As New IO.StreamReader("C:\A Large File.mp4")
        Dim fileLength As Long = sr.BaseStream.Length
        Dim bytesRemaining As Long = fileLength
        Dim bufferSize As Long
        Dim bytesUploaded As Long = 0

        Do
            If bytesRemaining <= uploadBytesSize Then
                buffersize = bytesRemaining
            Else
                bufferSize = uploadBytesSize
            End If

            Dim buffer(bufferSize - 1) As Byte
            sr.BaseStream.Position = bytesUploaded
            sr.BaseStream.Read(buffer, 0, bufferSize)

            bytesUploaded += bufferSize

            uploadFile(buffer)

        Loop Until bytesRemaining < uploadBytesSize

Open in new window


Seems to work fine for files for a few Mb, but anything over 300Mb it seems to fall over when I try to access the file buffer. I only need to access a small portion of the file at a time (defined by uploadBytesSize)

Does anyone have any ideas how I can access a file in segments?
0
Comment
Question by:tonelm54
3 Comments
 
LVL 26

Expert Comment

by:Shaun Kline
ID: 40530107
It does not appear that the condition for your loop will be satisfied if the file size is larger than 200000 because you are not refreshing the value of bytesRemaining. You should add a line of code after
bytesUploaded += bufferSize

Open in new window

with
bytesRemaining -= bufferSize

Open in new window

0
 
LVL 32

Accepted Solution

by:
Robberbaron (robr) earned 500 total points
ID: 40531287
your code doesnt make complete sense to me as BytesRemaining never changes.

msdn seems to indicate a slightly different method, http://msdn.microsoft.com/en-us/library/9kstw824(v=vs.110).aspx?cs-save-lang=1&cs-lang=vb#code-snippet-2


note that .Read reads up to the buffer size, so no need to change buffersize.
        Dim sr As New IO.StreamReader("C:\A Large File.mp4")
        Dim fileLength As Long = sr.BaseStream.Length

        Dim bufferSize As Integer = 4096 * 50  '4k block size most efficient
        Dim bytesUploaded As Long = 0
        Dim thisreadsize as integer

        Do While sr.Peek() => 0

            Dim buffer(bufferSize) As Byte
            'reads MAX buffer bytes.  returns less if end of file is reached
            thisreadsize = sr.BaseStream.Read(buffer, 0, buffer.Length)

            bytesUploaded += thisreadsize

            uploadFile(buffer)

        Loop 

Open in new window

0
 

Author Closing Comment

by:tonelm54
ID: 40558135
Modified the MSDN solution and works good now :--)
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Article by: Kraeven
Introduction Remote Share is a simple remote sharing tool, enabling you to see, add and remove remote or local shares. The application is written in VB.NET targeting the .NET framework 2.0. The source code and the compiled programs have been in…
Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.

929 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now