Solved

I get a request Timeout by upload big Data bigger 1GB

Posted on 2016-08-02
1
71 Views
Last Modified: 2016-08-09
Hello erveryone,

I get a Request Timeout when i upload bigger Data as 1 GB.

This error occurs when different uploaders.
Uploaders in javascript and PHP.

These Uploader: http://blueimp.github.io/jQuery-File-Upload/
and this:
http://www.it-gecko.de/php-5-4-datei-upload-mit-fortschrittsanzeige-progressbar.html

This approach yields a RequestTimeout by big data
Where is the mistake?
0
Comment
Question by:obcc-srs
1 Comment
 
LVL 108

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 41738807
PHP was never designed for uploading large files.  It's 20-year old software, and even today high-quality JPG image files from good cameras are rarely more than a few megabytes.  You have what we colloquially refer to as "ten pounds in a five-pound sack."

There are several configuration options that can cause problems with file uploads.  Most of them are related to the size of the files, and most of them fail silently, without diagnostics, unless your script is specifically written to identify errors and create diagnostics.  Please read this man page.  It describes most of the size-related issues:
http://php.net/manual/en/features.file-upload.common-pitfalls.php

You might also try to use this script.  Obviously, you must change line 35!  It is the simplest file upload example I have.  It will (hopefully) produce some useful diagnostics if it fails.  

However the best advice I can give you is "don't do that" because HTTP requests and PHP responses are not designed for big data uploads.  You may find a good solution in using FTP or a similar concept, where large files are expected and handled correctly.

<?php // demo/upload_example.php
/**
 * Demonstrate how to upload one or more files, using HTML5 and PHP
 *
 * REQUIRED: Man Page References
 * http://www.w3schools.com/tags/att_input_multiple.asp
 *
 * http://php.net/manual/en/reserved.variables.files.php
 * http://php.net/manual/en/features.file-upload.php
 * http://php.net/manual/en/features.file-upload.post-method.php
 * http://php.net/manual/en/features.file-upload.common-pitfalls.php
 * http://php.net/manual/en/features.file-upload.errors.php
 * http://php.net/manual/en/features.file-upload.multiple.php
 *
 * http://php.net/manual/en/function.move-uploaded-file.php
 *
 * IMPORTANT: If dealing with large files
 * http://php.net/manual/en/ini.core.php#ini.upload-max-filesize
 * http://php.net/manual/en/ini.core.php#ini.post-max-size
 * http://php.net/manual/en/info.configuration.php#ini.max-input-time
 */
error_reporting(E_ALL);

// MAY NOT BE NEEDED - CHECK PHP_INI PARAMETERS FOR YOUR TIMEZONE
date_default_timezone_set('America/Chicago');

// ESTABLISH THE NAME OF THE DESTINATION FOLDER ('storage' DIRECTORY)
$storage = 'storage';
if (!is_dir($storage))
{
    mkdir($storage);
}

// ESTABLISH THE BIGGEST FILE SIZE WE WILL ACCEPT - ABOUT 8 MB
$max_file_size = 8 * 1024 * 1024;

// ESTABLISH THE KINDS OF FILE EXTENSIONS WE WILL ACCEPT
$file_exts = array
( 'jpg'
, 'gif'
, 'png'
, 'txt'
, 'pdf'
, 'doc'
, 'docx'
)
;

// ARRAY OF ERRORS THAT MAY BE REPORTED IN $_FILES[]["error"] (THERE IS NO #5)
$errors = array
( UPLOAD_ERR_OK         => "Success!"
, UPLOAD_ERR_INI_SIZE   => "The uploaded file exceeds the upload_max_filesize directive in php.ini"
, UPLOAD_ERR_FORM_SIZE  => "The uploaded file exceeds the MAX_FILE_SIZE directive that was specified in the HTML form"
, UPLOAD_ERR_PARTIAL    => "The uploaded file was only partially uploaded"
, UPLOAD_ERR_NO_FILE    => "No file was uploaded"
, 5                     => "UNDEFINED ERROR #5"
, UPLOAD_ERR_NO_TMP_DIR => "Missing a temporary folder"
, UPLOAD_ERR_CANT_WRITE => "Cannot write file to disk"
, UPLOAD_ERR_EXTENSION  => "A PHP extension stopped the file upload"
)
;


// IF WE HAVE GOT SOMETHING IN $_POST - RUN THE ACTION SCRIPT
if (!empty($_POST))
{
    echo "<h2>Results: File Upload</h2>" . PHP_EOL;

    // ACTIVATE THIS TO SEE WHAT IS COMING THROUGH IN THE REQUEST
    // echo "<pre>"; var_dump($_FILES); var_dump($_POST); echo "</pre>";

    // REORGANIZE THE CONTENTS OF $_FILES SO WE CAN USE AN ITERATOR MORE SENSIBLY
    $nf = count($_FILES['userfile']['name']);
    while ($nf)
    {
        $nf--;
        $my_uploaded_files[$nf]['name']     = $_FILES['userfile']['name'][$nf];
        $my_uploaded_files[$nf]['type']     = $_FILES['userfile']['type'][$nf];
        $my_uploaded_files[$nf]['tmp_name'] = $_FILES['userfile']['tmp_name'][$nf];
        $my_uploaded_files[$nf]['error']    = $_FILES['userfile']['error'][$nf];
        $my_uploaded_files[$nf]['size']     = $_FILES['userfile']['size'][$nf];
    }

    // ITERATE OVER THE COLLECTION OF UPLOADED FILES
    foreach ($my_uploaded_files as $my_uploaded_file)
    {
        // SKIP OVER EMPTY SPOTS - NOTHING UPLOADED
        $error_code = $my_uploaded_file["error"];
        if ($error_code == UPLOAD_ERR_NO_FILE) continue;

        // IF THERE ARE ERRORS
        if ($error_code != UPLOAD_ERR_OK)
        {
            $error_message = $errors[$error_code];
            trigger_error("Upload error code: $error_code: $error_message", E_USER_WARNING);
            continue;
        }

        // SYNTHESIZE THE NEW FILE NAME
        $f_type = explode('.', basename($my_uploaded_file['name']));
        $f_type = end($f_type);
        $f_type = trim(strtolower($f_type));

        $f_name = explode('.', basename($my_uploaded_file['name']));
        $f_name = current($f_name);
        $f_name = trim(strtolower($f_name));

        // SERVER PATH TO THE NEW FILE
        $my_file_path
        = getcwd()
        . DIRECTORY_SEPARATOR
        . $storage
        . DIRECTORY_SEPARATOR
        . $f_name
        . '.'
        . $f_type
        ;

        // URL PATH TO THE NEW FILE
        $my_file_url
        = $storage
        . DIRECTORY_SEPARATOR
        . $f_name
        . '.'
        . $f_type
        ;

        // OPTIONAL TEST FOR ALLOWABLE EXTENSIONS
        if (!in_array($f_type, $file_exts))
        {
            trigger_error("$f_type Not allowed", E_USER_WARNING);
            continue;
        }

        // GET THE FILE SIZE
        $file_size = number_format($my_uploaded_file["size"]);

        // IF THE FILE IS NEW (DOES NOT EXIST)
        if (!file_exists($my_file_path))
        {
            // IF THE MOVE FUNCTION WORKED CORRECTLY
            if (move_uploaded_file($my_uploaded_file['tmp_name'], $my_file_path))
            {
                $upload_success = 1;
            }
            // IF THE MOVE FUNCTION FAILED
            else
            {
                $upload_success = -1;
            }
        }

        // IF THE FILE ALREADY EXISTS
        else
        {
            echo "<br/><b><i>$my_file_url</i></b> already exists." . PHP_EOL;

            // SHOULD WE OVERWRITE THE FILE? IF NOT
            if (empty($_POST["overwrite"]))
            {
                $upload_success = 0;
            }
            // IF WE SHOULD OVERWRITE THE FILE, TRY TO MAKE A BACKUP
            else
            {
                $now    = date('Y-m-d\THis');
                $my_bak = $my_file_path . '.' . $now . '.bak';
                if (!copy($my_file_path, $my_bak))
                {
                    trigger_error("Backup Failed for $my_file_url", E_USER_WARNING);
                }
                if (move_uploaded_file($my_uploaded_file['tmp_name'], $my_file_path))
                {
                    $upload_success = 2;
                }
                else
                {
                    $upload_success = -1;
                }
            }
        }

        // REPORT OUR SUCCESS OR FAILURE
        if ($upload_success == 2) { echo "<br/>It has been overwritten." . PHP_EOL; }
        if ($upload_success == 1) { echo "<br/><b>$my_file_url</b> has been saved." . PHP_EOL; }
        if ($upload_success == 0) { echo "<br/><b>It was NOT overwritten.</b>" . PHP_EOL; }
        if ($upload_success < 0)  { echo "<br/><b>ERROR: $my_file_url NOT SAVED - SEE WARNING FROM <i>move_uploaded_file()</i></b>" . PHP_EOL; }
        if ($upload_success > 0)
        {
            echo "$file_size bytes uploaded." . PHP_EOL;
            if (!chmod ($my_file_path, 0755))
            {
                echo '<br/>chmod(0755) FAILED: fileperms() = ';
                echo substr(sprintf('%o', fileperms($my_file_path)), -4);
            }
            echo '<br/>See the file: <a target="_blank" href="' . $my_file_url . '">' . $my_file_url . '</a><br/>' . PHP_EOL;
        }
    } // END FOREACH ITERATOR - EACH ITERATION PROCESSES ONE FILE
} // END ACTION SCRIPT


// CREATE THE HTML FORM USING HEREDOC NOTATION
$form = <<<EOF
<h2>Upload file(s)</h2>
<!--
    SOME IMPORTANT THINGS TO NOTE ABOUT THIS FORM...
    REQUIRES THE HTML5 DOCTYPE
    ENCTYPE= ATTRIBUTE IN THE HTML <FORM> TAG
    MAX_FILE_SIZE HIDDEN CONTROL MUST PRECEDE THE FILE INPUT CONTROLS
    INPUT NAME= IN TYPE=FILE DETERMINES THE NAME YOU FIND IN _FILES ARRAY
    ABSENCE OF ACTION= ATTRIBUTE IN <FORM> TAG CAUSES POST TO SAME URL
-->
<form name="UploadForm" enctype="multipart/form-data" method="post">
<input type="hidden" name="MAX_FILE_SIZE" value="$max_file_size" />
<p>
Find the file(s) you want to upload and click the "Upload" button below.
</p>

<input type="file" multiple name="userfile[]" size="80" />

<br/>Check this box <input autocomplete="off" type="checkbox" name="overwrite" /> to <b>overwrite</b> existing files.
<input type="submit" value="Upload" />
</form>
EOF;

echo $form;

Open in new window

0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

A list of useful business intelligence software.
These days socially coordinated efforts have turned into a critical requirement for enterprises.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now