• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 15703
  • Last Modified:

Problem with ADODB.Stream and large files

I'm working on a system that will allow users to check files in & out by uploading & downloading using a web browser, & I'm running into a problem with larger files.  Smaller files download fine, but when I try to use the system with files in the 8-9 mb range I get an error message stating that IE "was not able to open this Internet site".

The download script I'm using is as follows:

      Response.Buffer = True
      Dim Path, file, strFilePath, strFileSize, strFileName, objStream, strFileType, ContentType
      Path = Request("Path")
      File = Request("File")

      Const adTypeBinary = 1

      strFilePath = Server.MapPath(Path) & "\" & file
      strFileName = file

      strFileType = lcase(Right(strFileName, 4))

      ' Feel Free to Add Your Own Content-Types Here
      Select Case strFileType
      Case ".asf"
      ContentType = "video/x-ms-asf"
      Case ".avi"
      ContentType = "video/avi"
      Case ".doc"
      ContentType = "application/msword"
      Case ".zip"
      ContentType = "application/zip"
      Case ".xls"
      ContentType = "application/vnd.ms-excel"
      Case ".gif"
      ContentType = "image/gif"
      Case ".jpg", "jpeg"
      ContentType = "image/jpeg"
      Case ".wav"
      ContentType = "audio/wav"
      Case ".mp3"
      ContentType = "audio/mpeg3"
      Case ".mpg", "mpeg", "mpga"
      ContentType = "video/mpeg"
      Case ".rtf"
      ContentType = "application/rtf"
      Case ".htm", "html"
      ContentType = "text/html"
      Case ".asp"
      ContentType = "text/asp"
      Case Else
      'Handle All Other Files
      ContentType = "application/octet-stream"
      End Select

      Response.AddHeader "Content-Disposition", "attachment; filename=" & strFileName
      Response.AddHeader "Content-Length", strFileSize
      ' In a Perfect World, Your Client would also have UTF-8 as the default
      ' In Their Browser
      Response.Charset = "UTF-8"
      Response.ContentType = ContentType
      
      Set objStream = Server.CreateObject("ADODB.Stream")
      objStream.Open
      objStream.Type = adTypeBinary
      objStream.LoadFromFile strFilePath

      Response.BinaryWrite objStream.Read

      objStream.Close
      Set objStream = Nothing

      Response.Flush

Any ideas?  Thanks
0
eusebius
Asked:
eusebius
  • 7
  • 6
  • 3
  • +1
2 Solutions
 
deighcCommented:
Maybe it's a simple script timeout? It could a bit of time to stream out a 8mb file.

Try setting the script timeout to something like 5 mins.
0
 
eusebiusAuthor Commented:
Tried setting the timeout to 600, then 1200, then 30000... no success yet.  Also have "Response.Buffer = False".
0
 
deighcCommented:
Hmmmmmm, I'd be interested to know if the script is executing properly. That is, it would be interesting to know if the problem lies with the streaming script or the browser.

Have you tried other browsers?

Maybe you could try writing to a file or sending an email at various points in the script and see if the script executes completely.

As far as I understand it, the Read method of the Stream object reads the binary data into the ADO buffer, and the Close method flushes all content from the buffer. So it'd be interesting to know if your script gets past the Read method. It could be that your server chokes on reading the entire file into memory ????
0
Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

 
thefritterfatboyCommented:
This sounds like a known microsoft issue in Server 2003, that limits the download size to 20MB no matter what you set the buffering limit to. The good news is there is a hot fix that resolves the issue:

More info: http://support.microsoft.com/default.aspx?scid=kb;en-us;826756
0
 
eusebiusAuthor Commented:
Just tried this in Firefox.  It downloads smaller files (provided there's no space in the filename) just fine.  When attempting to download larger files, the downloaded file contains this error message:

Response object error 'ASP 0251 : 80004005'
Response Buffer Limit Exceeded
FileCheckout.asp, line 70
Execution of the ASP page caused the Response Buffer to exceed its configured limit.

Line 70 is:
Response.BinaryWrite objStream.Read

Just realized the same thing was happening with IE as well.  Even for (some) larger files, I do get prompted with the File Download box, which gives me the option to open or save.  If I choose open (hadn't tried this before), it takes me to FileCheckout.asp, and gives me the same 'ASP 0251 : 80004005' error message.

It does appear the script is executing properly for smaller files.  When I execute the script on a text file, it writes any errors to the file, and no errors are currently being written to smaller downloaded text files.
0
 
deighcCommented:
> Response Buffer Limit Exceeded

Sounds like the exact problem that thefritterfatboy mentioned.
0
 
eusebiusAuthor Commented:
Strange thing is, I get the same error message for files that are only 8 or 9 MB.  Any chance this limit would apply to them?
0
 
deighcCommented:
I'm not sure what the default maximum size is for IIS (probably varies with different versions) but maybe it's less than 20 mB.

You can change many settings in IIS (maximum request size etc) but it seems like the response buffer hits a limit at 20 mb, though it may default to less than that value. Perhaps you can bump up the max output size in the IIS metabase. So long as your files are less than 20mB you might be OK.

The bug is actually quite weird: IIS can only make a maximum of 10000 chunk writes per request and the default chunk size is 2kB. Strange!!
0
 
thefritterfatboyCommented:
I think the 20MB figure is purely theoretical as I have never managed to get to 20MB! I think that although the chunk size is 2KB some smaller chunks are being sent. Or maybe when the TCP/IP re-sends a lost or unacknowledged packet this adds another chunk.
0
 
deighcCommented:
...and I had a look at a metabase file for IIS6. There's no obvious value there that specifies an max output size. So maybe thefritterfatboy is right. You may already be over the limit with a 8 or 9MB file.
0
 
eusebiusAuthor Commented:
Unfortunately there may be some who use this system, who don't have access to apply hot fixes or make settings changes on the server the system is hosted on.  I'm trying to figure out if I can create a workaround by splitting the file into chunks, using something like:

      iSz = objStream.Size
      Response.AddHeader "Content-Length", iSz
      For i = 1 To iSz \ chunk
            If Not Response.IsClientConnected Then Exit For
            Response.BinaryWrite objStream.Read(chunk)
      Next

      If iSz Mod chunk > 0 Then
            If Response.IsClientConnected Then
                  Response.BinaryWrite objStream.Read(iSz Mod chunk)
            End If
      End If

      objStream.Close
      Set objStream = Nothing

      Response.End

But my knowledge of the inner workings of IIS is very sketchy.  Does this look like it would land me in the same situation as I'm already in?
0
 
deighcCommented:
> Does this look like it would land me in the same situation as I'm already in?

As I understand it yes. Because the bug lies with the final writing of data from the Response buffer to the client.

What you're proposing will only change the way in which the buffer is filled, not the way in which the buffer writes to the client. This is something that you have no control over.

But, hey, what do I know. Give a try...
0
 
thefritterfatboyCommented:
Splitting the file into chunks won't work. The bug in IIS is due to a file which is a little further into the system than the ASP layer.

If you don't have access to apply hot fixes, maybe your hosting comapny will if you explain that you're having a problem. (Also - send the link regarding the bug)
0
 
eusebiusAuthor Commented:
Believe it or not, it does work.  At least for 8 or 9MB files - haven't tried it with anything really large.

I used "chunk = 2048", as per the code I found at http://aspfaq.com/show.asp?id=2161.

Just to make things fun, if either of you (deighc or thefritterfatboy) can tell me how to get Firefox to properly handle file names with spaces in them (instead of cutting off the file name at the first space), I'll award you the points.  Otherwise I'll just split them between you.

Thanks
0
 
deighcCommented:
Maybe try URL encoding the file name:

Response.AddHeader "Content-Disposition", "attachment; filename=" & Server.URLEncode(strFileName)

Or try replacing spaces with %20
0
 
eusebiusAuthor Commented:
Tried that, which makes sense, but Firefox doesn't re-translate the encoding back into spaces, dots, etc.  So unfortunately it doesn't get my very far beyond where I was before (with a truncated file name).

Really not a big deal just thought I'd give it a shot.

Thanks very much for your input.
0
 
fritz_the_blankCommented:
One thing that might help is to clear the buffer periodically. Consider the following:

      for i = 0 to objStream.size
            i = i + 128000
            Response.BinaryWrite(objStream.Read(128000))
            Response.Flush
      next

What I am doing here is outputting the file but periodically clearing the buffer. This has helped me to overcome issues with large files.

FtB
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Ruby Fundamentals

This course will introduce you to Ruby, as well as teach you about classes, methods, variables, data structures, loops, enumerable methods, and finishing touches.

  • 7
  • 6
  • 3
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now