Solved

Read buffer

Posted on 2015-01-03
3
81 Views
Last Modified: 2015-01-19
I'm trying to read a file in segments and pass it into the function uploadFile(), however when I try and read large files I get memory issues:-

        Dim uploadBytesSize As Integer = 200000

        Dim sr As New IO.StreamReader("C:\A Large File.mp4")
        Dim fileLength As Long = sr.BaseStream.Length
        Dim bytesRemaining As Long = fileLength
        Dim bufferSize As Long
        Dim bytesUploaded As Long = 0

        Do
            If bytesRemaining <= uploadBytesSize Then
                buffersize = bytesRemaining
            Else
                bufferSize = uploadBytesSize
            End If

            Dim buffer(bufferSize - 1) As Byte
            sr.BaseStream.Position = bytesUploaded
            sr.BaseStream.Read(buffer, 0, bufferSize)

            bytesUploaded += bufferSize

            uploadFile(buffer)

        Loop Until bytesRemaining < uploadBytesSize

Open in new window


Seems to work fine for files for a few Mb, but anything over 300Mb it seems to fall over when I try to access the file buffer. I only need to access a small portion of the file at a time (defined by uploadBytesSize)

Does anyone have any ideas how I can access a file in segments?
0
Comment
Question by:tonelm54
3 Comments
 
LVL 26

Expert Comment

by:Shaun Kline
ID: 40530107
It does not appear that the condition for your loop will be satisfied if the file size is larger than 200000 because you are not refreshing the value of bytesRemaining. You should add a line of code after
bytesUploaded += bufferSize

Open in new window

with
bytesRemaining -= bufferSize

Open in new window

0
 
LVL 32

Accepted Solution

by:
Robberbaron (robr) earned 500 total points
ID: 40531287
your code doesnt make complete sense to me as BytesRemaining never changes.

msdn seems to indicate a slightly different method, http://msdn.microsoft.com/en-us/library/9kstw824(v=vs.110).aspx?cs-save-lang=1&cs-lang=vb#code-snippet-2


note that .Read reads up to the buffer size, so no need to change buffersize.
        Dim sr As New IO.StreamReader("C:\A Large File.mp4")
        Dim fileLength As Long = sr.BaseStream.Length

        Dim bufferSize As Integer = 4096 * 50  '4k block size most efficient
        Dim bytesUploaded As Long = 0
        Dim thisreadsize as integer

        Do While sr.Peek() => 0

            Dim buffer(bufferSize) As Byte
            'reads MAX buffer bytes.  returns less if end of file is reached
            thisreadsize = sr.BaseStream.Read(buffer, 0, buffer.Length)

            bytesUploaded += thisreadsize

            uploadFile(buffer)

        Loop 

Open in new window

0
 

Author Closing Comment

by:tonelm54
ID: 40558135
Modified the MSDN solution and works good now :--)
0

Featured Post

Announcing the Most Valuable Experts of 2016

MVEs are more concerned with the satisfaction of those they help than with the considerable points they can earn. They are the types of people you feel privileged to call colleagues. Join us in honoring this amazing group of Experts.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

A while ago, I was working on a Windows Forms application and I needed a special label control with reflection (glass) effect to show some titles in a stylish way. I've always enjoyed working with graphics, but it's never too clever to re-invent …
1.0 - Introduction Converting Visual Basic 6.0 (VB6) to Visual Basic 2008+ (VB.NET). If ever there was a subject full of murkiness and bad decisions, it is this one!   The first problem seems to be that people considering this task of converting…
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …
This Micro Tutorial demonstrates using Microsoft Excel pivot tables, how to reverse engineer competitors' marketing strategies through backlinks.

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now