?
Solved

Problem with ADODB.Stream and large files

Posted on 2005-05-12
17
Medium Priority
?
15,463 Views
Last Modified: 2009-03-17
I'm working on a system that will allow users to check files in & out by uploading & downloading using a web browser, & I'm running into a problem with larger files.  Smaller files download fine, but when I try to use the system with files in the 8-9 mb range I get an error message stating that IE "was not able to open this Internet site".

The download script I'm using is as follows:

      Response.Buffer = True
      Dim Path, file, strFilePath, strFileSize, strFileName, objStream, strFileType, ContentType
      Path = Request("Path")
      File = Request("File")

      Const adTypeBinary = 1

      strFilePath = Server.MapPath(Path) & "\" & file
      strFileName = file

      strFileType = lcase(Right(strFileName, 4))

      ' Feel Free to Add Your Own Content-Types Here
      Select Case strFileType
      Case ".asf"
      ContentType = "video/x-ms-asf"
      Case ".avi"
      ContentType = "video/avi"
      Case ".doc"
      ContentType = "application/msword"
      Case ".zip"
      ContentType = "application/zip"
      Case ".xls"
      ContentType = "application/vnd.ms-excel"
      Case ".gif"
      ContentType = "image/gif"
      Case ".jpg", "jpeg"
      ContentType = "image/jpeg"
      Case ".wav"
      ContentType = "audio/wav"
      Case ".mp3"
      ContentType = "audio/mpeg3"
      Case ".mpg", "mpeg", "mpga"
      ContentType = "video/mpeg"
      Case ".rtf"
      ContentType = "application/rtf"
      Case ".htm", "html"
      ContentType = "text/html"
      Case ".asp"
      ContentType = "text/asp"
      Case Else
      'Handle All Other Files
      ContentType = "application/octet-stream"
      End Select

      Response.AddHeader "Content-Disposition", "attachment; filename=" & strFileName
      Response.AddHeader "Content-Length", strFileSize
      ' In a Perfect World, Your Client would also have UTF-8 as the default
      ' In Their Browser
      Response.Charset = "UTF-8"
      Response.ContentType = ContentType
      
      Set objStream = Server.CreateObject("ADODB.Stream")
      objStream.Open
      objStream.Type = adTypeBinary
      objStream.LoadFromFile strFilePath

      Response.BinaryWrite objStream.Read

      objStream.Close
      Set objStream = Nothing

      Response.Flush

Any ideas?  Thanks
0
Comment
Question by:eusebius
  • 7
  • 6
  • 3
  • +1
17 Comments
 
LVL 15

Expert Comment

by:deighc
ID: 13994012
Maybe it's a simple script timeout? It could a bit of time to stream out a 8mb file.

Try setting the script timeout to something like 5 mins.
0
 
LVL 1

Author Comment

by:eusebius
ID: 13995547
Tried setting the timeout to 600, then 1200, then 30000... no success yet.  Also have "Response.Buffer = False".
0
 
LVL 15

Assisted Solution

by:deighc
deighc earned 750 total points
ID: 13995722
Hmmmmmm, I'd be interested to know if the script is executing properly. That is, it would be interesting to know if the problem lies with the streaming script or the browser.

Have you tried other browsers?

Maybe you could try writing to a file or sending an email at various points in the script and see if the script executes completely.

As far as I understand it, the Read method of the Stream object reads the binary data into the ADO buffer, and the Close method flushes all content from the buffer. So it'd be interesting to know if your script gets past the Read method. It could be that your server chokes on reading the entire file into memory ????
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
LVL 10

Accepted Solution

by:
thefritterfatboy earned 750 total points
ID: 13995725
This sounds like a known microsoft issue in Server 2003, that limits the download size to 20MB no matter what you set the buffering limit to. The good news is there is a hot fix that resolves the issue:

More info: http://support.microsoft.com/default.aspx?scid=kb;en-us;826756
0
 
LVL 1

Author Comment

by:eusebius
ID: 13996463
Just tried this in Firefox.  It downloads smaller files (provided there's no space in the filename) just fine.  When attempting to download larger files, the downloaded file contains this error message:

Response object error 'ASP 0251 : 80004005'
Response Buffer Limit Exceeded
FileCheckout.asp, line 70
Execution of the ASP page caused the Response Buffer to exceed its configured limit.

Line 70 is:
Response.BinaryWrite objStream.Read

Just realized the same thing was happening with IE as well.  Even for (some) larger files, I do get prompted with the File Download box, which gives me the option to open or save.  If I choose open (hadn't tried this before), it takes me to FileCheckout.asp, and gives me the same 'ASP 0251 : 80004005' error message.

It does appear the script is executing properly for smaller files.  When I execute the script on a text file, it writes any errors to the file, and no errors are currently being written to smaller downloaded text files.
0
 
LVL 15

Expert Comment

by:deighc
ID: 13996489
> Response Buffer Limit Exceeded

Sounds like the exact problem that thefritterfatboy mentioned.
0
 
LVL 1

Author Comment

by:eusebius
ID: 13996567
Strange thing is, I get the same error message for files that are only 8 or 9 MB.  Any chance this limit would apply to them?
0
 
LVL 15

Expert Comment

by:deighc
ID: 13996719
I'm not sure what the default maximum size is for IIS (probably varies with different versions) but maybe it's less than 20 mB.

You can change many settings in IIS (maximum request size etc) but it seems like the response buffer hits a limit at 20 mb, though it may default to less than that value. Perhaps you can bump up the max output size in the IIS metabase. So long as your files are less than 20mB you might be OK.

The bug is actually quite weird: IIS can only make a maximum of 10000 chunk writes per request and the default chunk size is 2kB. Strange!!
0
 
LVL 10

Expert Comment

by:thefritterfatboy
ID: 13996788
I think the 20MB figure is purely theoretical as I have never managed to get to 20MB! I think that although the chunk size is 2KB some smaller chunks are being sent. Or maybe when the TCP/IP re-sends a lost or unacknowledged packet this adds another chunk.
0
 
LVL 15

Expert Comment

by:deighc
ID: 13996811
...and I had a look at a metabase file for IIS6. There's no obvious value there that specifies an max output size. So maybe thefritterfatboy is right. You may already be over the limit with a 8 or 9MB file.
0
 
LVL 1

Author Comment

by:eusebius
ID: 13996903
Unfortunately there may be some who use this system, who don't have access to apply hot fixes or make settings changes on the server the system is hosted on.  I'm trying to figure out if I can create a workaround by splitting the file into chunks, using something like:

      iSz = objStream.Size
      Response.AddHeader "Content-Length", iSz
      For i = 1 To iSz \ chunk
            If Not Response.IsClientConnected Then Exit For
            Response.BinaryWrite objStream.Read(chunk)
      Next

      If iSz Mod chunk > 0 Then
            If Response.IsClientConnected Then
                  Response.BinaryWrite objStream.Read(iSz Mod chunk)
            End If
      End If

      objStream.Close
      Set objStream = Nothing

      Response.End

But my knowledge of the inner workings of IIS is very sketchy.  Does this look like it would land me in the same situation as I'm already in?
0
 
LVL 15

Expert Comment

by:deighc
ID: 13996945
> Does this look like it would land me in the same situation as I'm already in?

As I understand it yes. Because the bug lies with the final writing of data from the Response buffer to the client.

What you're proposing will only change the way in which the buffer is filled, not the way in which the buffer writes to the client. This is something that you have no control over.

But, hey, what do I know. Give a try...
0
 
LVL 10

Expert Comment

by:thefritterfatboy
ID: 13996984
Splitting the file into chunks won't work. The bug in IIS is due to a file which is a little further into the system than the ASP layer.

If you don't have access to apply hot fixes, maybe your hosting comapny will if you explain that you're having a problem. (Also - send the link regarding the bug)
0
 
LVL 1

Author Comment

by:eusebius
ID: 13997254
Believe it or not, it does work.  At least for 8 or 9MB files - haven't tried it with anything really large.

I used "chunk = 2048", as per the code I found at http://aspfaq.com/show.asp?id=2161.

Just to make things fun, if either of you (deighc or thefritterfatboy) can tell me how to get Firefox to properly handle file names with spaces in them (instead of cutting off the file name at the first space), I'll award you the points.  Otherwise I'll just split them between you.

Thanks
0
 
LVL 15

Expert Comment

by:deighc
ID: 13997298
Maybe try URL encoding the file name:

Response.AddHeader "Content-Disposition", "attachment; filename=" & Server.URLEncode(strFileName)

Or try replacing spaces with %20
0
 
LVL 1

Author Comment

by:eusebius
ID: 13997500
Tried that, which makes sense, but Firefox doesn't re-translate the encoding back into spaces, dots, etc.  So unfortunately it doesn't get my very far beyond where I was before (with a truncated file name).

Really not a big deal just thought I'd give it a shot.

Thanks very much for your input.
0
 
LVL 46

Expert Comment

by:fritz_the_blank
ID: 13998440
One thing that might help is to clear the buffer periodically. Consider the following:

      for i = 0 to objStream.size
            i = i + 128000
            Response.BinaryWrite(objStream.Read(128000))
            Response.Flush
      next

What I am doing here is outputting the file but periodically clearing the buffer. This has helped me to overcome issues with large files.

FtB
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I recently decide that I needed a way to make my pages scream on the net.   While searching around how I can accomplish this I stumbled across a great article that stated "minimize the server requests." I got to thinking, hey, I use more than one…
I would like to start this tip/trick by saying Thank You, to all who said that this could not be done, as it forced me to make sure that it could be accomplished. :) To start, I want to make sure everyone understands the importance of utilizing p…
Screencast - Getting to Know the Pipeline
Whether it be Exchange Server Crash Issues, Dirty Shutdown Errors or Failed to mount error, Stellar Phoenix Mailbox Exchange Recovery has always got your back. With the help of its easy to understand user interface and 3 simple steps recovery proced…
Suggested Courses

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question