Solved

Read buffer

Posted on 2015-01-03
3
78 Views
Last Modified: 2015-01-19
I'm trying to read a file in segments and pass it into the function uploadFile(), however when I try and read large files I get memory issues:-

        Dim uploadBytesSize As Integer = 200000

        Dim sr As New IO.StreamReader("C:\A Large File.mp4")
        Dim fileLength As Long = sr.BaseStream.Length
        Dim bytesRemaining As Long = fileLength
        Dim bufferSize As Long
        Dim bytesUploaded As Long = 0

        Do
            If bytesRemaining <= uploadBytesSize Then
                buffersize = bytesRemaining
            Else
                bufferSize = uploadBytesSize
            End If

            Dim buffer(bufferSize - 1) As Byte
            sr.BaseStream.Position = bytesUploaded
            sr.BaseStream.Read(buffer, 0, bufferSize)

            bytesUploaded += bufferSize

            uploadFile(buffer)

        Loop Until bytesRemaining < uploadBytesSize

Open in new window


Seems to work fine for files for a few Mb, but anything over 300Mb it seems to fall over when I try to access the file buffer. I only need to access a small portion of the file at a time (defined by uploadBytesSize)

Does anyone have any ideas how I can access a file in segments?
0
Comment
Question by:tonelm54
3 Comments
 
LVL 26

Expert Comment

by:Shaun Kline
ID: 40530107
It does not appear that the condition for your loop will be satisfied if the file size is larger than 200000 because you are not refreshing the value of bytesRemaining. You should add a line of code after
bytesUploaded += bufferSize

Open in new window

with
bytesRemaining -= bufferSize

Open in new window

0
 
LVL 32

Accepted Solution

by:
Robberbaron (robr) earned 500 total points
ID: 40531287
your code doesnt make complete sense to me as BytesRemaining never changes.

msdn seems to indicate a slightly different method, http://msdn.microsoft.com/en-us/library/9kstw824(v=vs.110).aspx?cs-save-lang=1&cs-lang=vb#code-snippet-2


note that .Read reads up to the buffer size, so no need to change buffersize.
        Dim sr As New IO.StreamReader("C:\A Large File.mp4")
        Dim fileLength As Long = sr.BaseStream.Length

        Dim bufferSize As Integer = 4096 * 50  '4k block size most efficient
        Dim bytesUploaded As Long = 0
        Dim thisreadsize as integer

        Do While sr.Peek() => 0

            Dim buffer(bufferSize) As Byte
            'reads MAX buffer bytes.  returns less if end of file is reached
            thisreadsize = sr.BaseStream.Read(buffer, 0, buffer.Length)

            bytesUploaded += thisreadsize

            uploadFile(buffer)

        Loop 

Open in new window

0
 

Author Closing Comment

by:tonelm54
ID: 40558135
Modified the MSDN solution and works good now :--)
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

A while ago, I was working on a Windows Forms application and I needed a special label control with reflection (glass) effect to show some titles in a stylish way. I've always enjoyed working with graphics, but it's never too clever to re-invent …
Creating an analog clock UserControl seems fairly straight forward.  It is, after all, essentially just a circle with several lines in it!  Two common approaches for rendering an analog clock typically involve either manually calculating points with…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now