Solved

Read buffer

Posted on 2015-01-03
3
85 Views
Last Modified: 2015-01-19
I'm trying to read a file in segments and pass it into the function uploadFile(), however when I try and read large files I get memory issues:-

        Dim uploadBytesSize As Integer = 200000

        Dim sr As New IO.StreamReader("C:\A Large File.mp4")
        Dim fileLength As Long = sr.BaseStream.Length
        Dim bytesRemaining As Long = fileLength
        Dim bufferSize As Long
        Dim bytesUploaded As Long = 0

        Do
            If bytesRemaining <= uploadBytesSize Then
                buffersize = bytesRemaining
            Else
                bufferSize = uploadBytesSize
            End If

            Dim buffer(bufferSize - 1) As Byte
            sr.BaseStream.Position = bytesUploaded
            sr.BaseStream.Read(buffer, 0, bufferSize)

            bytesUploaded += bufferSize

            uploadFile(buffer)

        Loop Until bytesRemaining < uploadBytesSize

Open in new window


Seems to work fine for files for a few Mb, but anything over 300Mb it seems to fall over when I try to access the file buffer. I only need to access a small portion of the file at a time (defined by uploadBytesSize)

Does anyone have any ideas how I can access a file in segments?
0
Comment
Question by:tonelm54
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
3 Comments
 
LVL 26

Expert Comment

by:Shaun Kline
ID: 40530107
It does not appear that the condition for your loop will be satisfied if the file size is larger than 200000 because you are not refreshing the value of bytesRemaining. You should add a line of code after
bytesUploaded += bufferSize

Open in new window

with
bytesRemaining -= bufferSize

Open in new window

0
 
LVL 32

Accepted Solution

by:
Robberbaron (robr) earned 500 total points
ID: 40531287
your code doesnt make complete sense to me as BytesRemaining never changes.

msdn seems to indicate a slightly different method, http://msdn.microsoft.com/en-us/library/9kstw824(v=vs.110).aspx?cs-save-lang=1&cs-lang=vb#code-snippet-2


note that .Read reads up to the buffer size, so no need to change buffersize.
        Dim sr As New IO.StreamReader("C:\A Large File.mp4")
        Dim fileLength As Long = sr.BaseStream.Length

        Dim bufferSize As Integer = 4096 * 50  '4k block size most efficient
        Dim bytesUploaded As Long = 0
        Dim thisreadsize as integer

        Do While sr.Peek() => 0

            Dim buffer(bufferSize) As Byte
            'reads MAX buffer bytes.  returns less if end of file is reached
            thisreadsize = sr.BaseStream.Read(buffer, 0, buffer.Length)

            bytesUploaded += thisreadsize

            uploadFile(buffer)

        Loop 

Open in new window

0
 

Author Closing Comment

by:tonelm54
ID: 40558135
Modified the MSDN solution and works good now :--)
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Microsoft Reports are based on a report definition, which is an XML file that describes data and layout for the report, with a different extension. You can create a client-side report definition language (*.rdlc) file with Visual Studio, and build g…
Creating an analog clock UserControl seems fairly straight forward.  It is, after all, essentially just a circle with several lines in it!  Two common approaches for rendering an analog clock typically involve either manually calculating points with…
Come and listen to Percona CEO Peter Zaitsev discuss what’s new in Percona open source software, including Percona Server for MySQL (https://www.percona.com/software/mysql-database/percona-server) and MongoDB (https://www.percona.com/software/mongo-…
Michael from AdRem Software outlines event notifications and Automatic Corrective Actions in network monitoring. Automatic Corrective Actions are scripts, which can automatically run upon discovery of a certain undesirable condition in your network.…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question