Solved

Need solution to download large 100+ MB files using PHP.

Posted on 2007-04-10
13
484 Views
Last Modified: 2012-05-05
Need solution to download large 100+ MB files using PHP.

Currently I have in mind chopping the file into parts and then rejoining them durnig the stream.. but don't know if it will work.  I found some code to chop the file up, but am having trouble puttingit back together (good thing I am not a doctor)

Everything seems fine until the download.  The download window is launched but the file is 0bytes.

Thanks in Advance.

Code:

<?php
$filename = $file;
$path = '/var/www/virtual/ftp/' . $filename;
error_reporting(E_ALL);
set_time_limit( 0 );
$bytes = 512000;           //size of the chunks in bytes (500k in this case)

$pieces = file_chop($path, $bytes);
echo "$path was chopped into " . count($pieces) . " $bytes-byte chunks stored in $pieces";
//the last piece may be smaller than $bytes bytes
exit;
foreach ($pieces as $bit){
   //insert $bit into your database
header('Content-Type: application/force-download');
header ("Content-Length: " . filesize($file));
header ("Content-Disposition: attachment; filename=".basename($file));

  echo fread($bit, 4096);
       ob_flush();

}

function file_chop($file_path, $chunk_size){
    $handle = fopen($file_path, 'rb');         //read the file in binary mode
    $size = filesize($file_path);
    $contents = fread($handle, $size);
    fclose($handle);
   
    //find number of full $chunk_size byte portions
    $num_chunks = floor($size/$chunk_size);

    $chunks = array();

    $start = 0;
    for ($kk=0; $kk < $num_chunks; $kk++){
      $chunks[] = substr($contents, $start, $chunk_size); //get $chunk_size bytes at a time
      $start += $chunk_size;
    }
   
    if ($start < $size){
       $chunks[] = substr($contents, $start);  //get any leftover
    }
    return $chunks;
}  




function file_chop_big($file_path, $chunk_size){
  $size = filesize($file_path);

  //find number of full $chunk_size byte portions
  $num_chunks = floor($size/$chunk_size);

  $file_handle = fopen($file_path, 'rb');         //read the file in binary mode

  $chunks = array();

  for ($kk=0; $kk < $num_chunks; $kk++){
    $chunks[$kk] = basename($file_path).'.chunk'.($kk+1);
    $chunk_handle = fopen($chunks[$kk], 'w');   //open the chunk file for writing

    //write the data to the chunk file 1k at a time
    while((ftell($chunk_handle) + 1024) <= $chunk_size){
      fwrite($chunk_handle, fread($file_handle, 1024));
    }

    if(($leftover = $chunk_size-ftell($chunk_handle)) > 0 ){
      fwrite($chunk_handle, fread($file_handle, $leftover));
    }
    fclose($chunk_handle);
  }

  if (($leftover = $size - ftell($file_handle)) > 0){
    $chunks[$num_chunks] = basename($file_path).'.chunk'.($num_chunks + 1);
    $chunk_handle = fopen($chunks[$num_chunks], 'w');
    while(!feof($file_handle)){
      fwrite($chunk_handle, fread($file_handle, 1024));
    }
    fclose($chunk_handle);
  }

  fclose($file_handle);
  return $chunks;
}
?>
0
Comment
Question by:bionicblakey
13 Comments
 

Author Comment

by:bionicblakey
Comment Utility
i had teh exit; in there for testing...    it is not the reason the download failed.. pls comment it out..
thanks.
0
 
LVL 7

Expert Comment

by:Zack Soderquist
Comment Utility
Are you trying to download or upload a file???? i.e. are you trying to download a file to your computer from your site or are you trying to upload a file from your computer to your website?
0
 
LVL 7

Expert Comment

by:Zack Soderquist
Comment Utility
If you are trying to upload a file to your website, it's probably that your hosting server is limiting the upload limit of files. If it is hosted on a "unix" flavored platform running Apache, you may be able to override the limits by creating a .htaccess file in the root of your site.

Put the following lines in your .htaccess file setting the filesize to what you want .. I set it to 200M (200 MB)

php_value upload_max_filesize 200M
php_value post_max_size 200M
php_value memory_limit 200M
php_value max_execution_time 3600
0
 

Author Comment

by:bionicblakey
Comment Utility
"Need solution to download large 100+ MB files using PHP."

I am trying to DOWNLOAD.

Thanks.
0
 
LVL 7

Expert Comment

by:Zack Soderquist
Comment Utility
I read the download.. I was confirming

I'm confused as to why you are chopping up the file .. why can't you just download the file as a whole?
0
 
LVL 7

Expert Comment

by:Zack Soderquist
Comment Utility
As I'm understanding what you're doing ..

PHP is a server side scripting language .. you will be chopping it up and putting it back together before the file is sent to the client .. which is kind of like driving to the post office and back to pick up your mail out of your mailbox in front of your house.

Without knowing what you're trying to accomplish, my initial reaction is that you may be approaching it the wrong way
0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 

Author Comment

by:bionicblakey
Comment Utility
Hi

I have alot of users downloading large files.. and i am trying to cut down on server load
and also cut down on failed downloads.

when the files get 100+ MB , it seems that a high % of downloads fail.

do you have another idea?
0
 
LVL 7

Expert Comment

by:Zack Soderquist
Comment Utility
Yes, change protocols

Provide them an FTP link to download the files instead of using HTTP.
If you want, keep the smaller files passed via HTTP, but I would highly recommend that instead of downloading larger files via HTTP, provide the using with an FTP link to download the file. FTP is a more efficient protocol and faster protocol for downloading large files
0
 
LVL 7

Accepted Solution

by:
Zack Soderquist earned 500 total points
Comment Utility
You are probably getting alot of failed downloads due to HTTP timeouts .. FTP should eliminate that ..

If you get alot of failed downloads, then your users will probably try a several times to download the file, which will dramatically increase your traffic. FTP again should resolve this because they would get the file the first time.
0
 
LVL 48

Expert Comment

by:hernst42
Comment Utility
If you have no security checks if a user is allowed to download a file put the files into a directory which is accessable via static (nonphp) HTTP-Request which can handle the webserver by using kernel functions and not by executing a php-script. This also allows a HTTP-resume (if your webserver supports it) which might also cut down the load a lot.
0
 
LVL 24

Expert Comment

by:slyong
Comment Utility
You could use the fread function in PHP to make the download resumable as in the comment section of php fread (http://www.php.net/fread).
0
 
LVL 14

Expert Comment

by:ygoutham
Comment Utility
i am not commenting on the "chopping" and "putting together".  but the 0 byte problem occurs if apache is not the owner of the file.  in which case it would always download the file as a 0 byte stuff.

change the owner to the httpd user.  use a small file like a 1 mb or something.  try chopping it to pieces and see if it comes wholesome.
0
 
LVL 7

Expert Comment

by:Zack Soderquist
Comment Utility
has this issue been resolved?
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

Introduction Many web sites contain image galleries; a common design for these galleries includes a page with a collection of thumbnail images.  You can click on each of the thumbnail images to see the larger version of the image.  This is easily i…
This article discusses how to create an extensible mechanism for linked drop downs.
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to dynamically set the form action using jQuery.

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now