Solved

I get a request Timeout by upload big Data bigger 1GB

Posted on 2016-08-02
1
192 Views
Last Modified: 2016-08-09
Hello erveryone,

I get a Request Timeout when i upload bigger Data as 1 GB.

This error occurs when different uploaders.
Uploaders in javascript and PHP.

These Uploader: http://blueimp.github.io/jQuery-File-Upload/
and this:
http://www.it-gecko.de/php-5-4-datei-upload-mit-fortschrittsanzeige-progressbar.html

This approach yields a RequestTimeout by big data
Where is the mistake?
0
Comment
Question by:obcc-srs
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
1 Comment
 
LVL 110

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 41738807
PHP was never designed for uploading large files.  It's 20-year old software, and even today high-quality JPG image files from good cameras are rarely more than a few megabytes.  You have what we colloquially refer to as "ten pounds in a five-pound sack."

There are several configuration options that can cause problems with file uploads.  Most of them are related to the size of the files, and most of them fail silently, without diagnostics, unless your script is specifically written to identify errors and create diagnostics.  Please read this man page.  It describes most of the size-related issues:
http://php.net/manual/en/features.file-upload.common-pitfalls.php

You might also try to use this script.  Obviously, you must change line 35!  It is the simplest file upload example I have.  It will (hopefully) produce some useful diagnostics if it fails.  

However the best advice I can give you is "don't do that" because HTTP requests and PHP responses are not designed for big data uploads.  You may find a good solution in using FTP or a similar concept, where large files are expected and handled correctly.

<?php // demo/upload_example.php
/**
 * Demonstrate how to upload one or more files, using HTML5 and PHP
 *
 * REQUIRED: Man Page References
 * http://www.w3schools.com/tags/att_input_multiple.asp
 *
 * http://php.net/manual/en/reserved.variables.files.php
 * http://php.net/manual/en/features.file-upload.php
 * http://php.net/manual/en/features.file-upload.post-method.php
 * http://php.net/manual/en/features.file-upload.common-pitfalls.php
 * http://php.net/manual/en/features.file-upload.errors.php
 * http://php.net/manual/en/features.file-upload.multiple.php
 *
 * http://php.net/manual/en/function.move-uploaded-file.php
 *
 * IMPORTANT: If dealing with large files
 * http://php.net/manual/en/ini.core.php#ini.upload-max-filesize
 * http://php.net/manual/en/ini.core.php#ini.post-max-size
 * http://php.net/manual/en/info.configuration.php#ini.max-input-time
 */
error_reporting(E_ALL);

// MAY NOT BE NEEDED - CHECK PHP_INI PARAMETERS FOR YOUR TIMEZONE
date_default_timezone_set('America/Chicago');

// ESTABLISH THE NAME OF THE DESTINATION FOLDER ('storage' DIRECTORY)
$storage = 'storage';
if (!is_dir($storage))
{
    mkdir($storage);
}

// ESTABLISH THE BIGGEST FILE SIZE WE WILL ACCEPT - ABOUT 8 MB
$max_file_size = 8 * 1024 * 1024;

// ESTABLISH THE KINDS OF FILE EXTENSIONS WE WILL ACCEPT
$file_exts = array
( 'jpg'
, 'gif'
, 'png'
, 'txt'
, 'pdf'
, 'doc'
, 'docx'
)
;

// ARRAY OF ERRORS THAT MAY BE REPORTED IN $_FILES[]["error"] (THERE IS NO #5)
$errors = array
( UPLOAD_ERR_OK         => "Success!"
, UPLOAD_ERR_INI_SIZE   => "The uploaded file exceeds the upload_max_filesize directive in php.ini"
, UPLOAD_ERR_FORM_SIZE  => "The uploaded file exceeds the MAX_FILE_SIZE directive that was specified in the HTML form"
, UPLOAD_ERR_PARTIAL    => "The uploaded file was only partially uploaded"
, UPLOAD_ERR_NO_FILE    => "No file was uploaded"
, 5                     => "UNDEFINED ERROR #5"
, UPLOAD_ERR_NO_TMP_DIR => "Missing a temporary folder"
, UPLOAD_ERR_CANT_WRITE => "Cannot write file to disk"
, UPLOAD_ERR_EXTENSION  => "A PHP extension stopped the file upload"
)
;


// IF WE HAVE GOT SOMETHING IN $_POST - RUN THE ACTION SCRIPT
if (!empty($_POST))
{
    echo "<h2>Results: File Upload</h2>" . PHP_EOL;

    // ACTIVATE THIS TO SEE WHAT IS COMING THROUGH IN THE REQUEST
    // echo "<pre>"; var_dump($_FILES); var_dump($_POST); echo "</pre>";

    // REORGANIZE THE CONTENTS OF $_FILES SO WE CAN USE AN ITERATOR MORE SENSIBLY
    $nf = count($_FILES['userfile']['name']);
    while ($nf)
    {
        $nf--;
        $my_uploaded_files[$nf]['name']     = $_FILES['userfile']['name'][$nf];
        $my_uploaded_files[$nf]['type']     = $_FILES['userfile']['type'][$nf];
        $my_uploaded_files[$nf]['tmp_name'] = $_FILES['userfile']['tmp_name'][$nf];
        $my_uploaded_files[$nf]['error']    = $_FILES['userfile']['error'][$nf];
        $my_uploaded_files[$nf]['size']     = $_FILES['userfile']['size'][$nf];
    }

    // ITERATE OVER THE COLLECTION OF UPLOADED FILES
    foreach ($my_uploaded_files as $my_uploaded_file)
    {
        // SKIP OVER EMPTY SPOTS - NOTHING UPLOADED
        $error_code = $my_uploaded_file["error"];
        if ($error_code == UPLOAD_ERR_NO_FILE) continue;

        // IF THERE ARE ERRORS
        if ($error_code != UPLOAD_ERR_OK)
        {
            $error_message = $errors[$error_code];
            trigger_error("Upload error code: $error_code: $error_message", E_USER_WARNING);
            continue;
        }

        // SYNTHESIZE THE NEW FILE NAME
        $f_type = explode('.', basename($my_uploaded_file['name']));
        $f_type = end($f_type);
        $f_type = trim(strtolower($f_type));

        $f_name = explode('.', basename($my_uploaded_file['name']));
        $f_name = current($f_name);
        $f_name = trim(strtolower($f_name));

        // SERVER PATH TO THE NEW FILE
        $my_file_path
        = getcwd()
        . DIRECTORY_SEPARATOR
        . $storage
        . DIRECTORY_SEPARATOR
        . $f_name
        . '.'
        . $f_type
        ;

        // URL PATH TO THE NEW FILE
        $my_file_url
        = $storage
        . DIRECTORY_SEPARATOR
        . $f_name
        . '.'
        . $f_type
        ;

        // OPTIONAL TEST FOR ALLOWABLE EXTENSIONS
        if (!in_array($f_type, $file_exts))
        {
            trigger_error("$f_type Not allowed", E_USER_WARNING);
            continue;
        }

        // GET THE FILE SIZE
        $file_size = number_format($my_uploaded_file["size"]);

        // IF THE FILE IS NEW (DOES NOT EXIST)
        if (!file_exists($my_file_path))
        {
            // IF THE MOVE FUNCTION WORKED CORRECTLY
            if (move_uploaded_file($my_uploaded_file['tmp_name'], $my_file_path))
            {
                $upload_success = 1;
            }
            // IF THE MOVE FUNCTION FAILED
            else
            {
                $upload_success = -1;
            }
        }

        // IF THE FILE ALREADY EXISTS
        else
        {
            echo "<br/><b><i>$my_file_url</i></b> already exists." . PHP_EOL;

            // SHOULD WE OVERWRITE THE FILE? IF NOT
            if (empty($_POST["overwrite"]))
            {
                $upload_success = 0;
            }
            // IF WE SHOULD OVERWRITE THE FILE, TRY TO MAKE A BACKUP
            else
            {
                $now    = date('Y-m-d\THis');
                $my_bak = $my_file_path . '.' . $now . '.bak';
                if (!copy($my_file_path, $my_bak))
                {
                    trigger_error("Backup Failed for $my_file_url", E_USER_WARNING);
                }
                if (move_uploaded_file($my_uploaded_file['tmp_name'], $my_file_path))
                {
                    $upload_success = 2;
                }
                else
                {
                    $upload_success = -1;
                }
            }
        }

        // REPORT OUR SUCCESS OR FAILURE
        if ($upload_success == 2) { echo "<br/>It has been overwritten." . PHP_EOL; }
        if ($upload_success == 1) { echo "<br/><b>$my_file_url</b> has been saved." . PHP_EOL; }
        if ($upload_success == 0) { echo "<br/><b>It was NOT overwritten.</b>" . PHP_EOL; }
        if ($upload_success < 0)  { echo "<br/><b>ERROR: $my_file_url NOT SAVED - SEE WARNING FROM <i>move_uploaded_file()</i></b>" . PHP_EOL; }
        if ($upload_success > 0)
        {
            echo "$file_size bytes uploaded." . PHP_EOL;
            if (!chmod ($my_file_path, 0755))
            {
                echo '<br/>chmod(0755) FAILED: fileperms() = ';
                echo substr(sprintf('%o', fileperms($my_file_path)), -4);
            }
            echo '<br/>See the file: <a target="_blank" href="' . $my_file_url . '">' . $my_file_url . '</a><br/>' . PHP_EOL;
        }
    } // END FOREACH ITERATOR - EACH ITERATION PROCESSES ONE FILE
} // END ACTION SCRIPT


// CREATE THE HTML FORM USING HEREDOC NOTATION
$form = <<<EOF
<h2>Upload file(s)</h2>
<!--
    SOME IMPORTANT THINGS TO NOTE ABOUT THIS FORM...
    REQUIRES THE HTML5 DOCTYPE
    ENCTYPE= ATTRIBUTE IN THE HTML <FORM> TAG
    MAX_FILE_SIZE HIDDEN CONTROL MUST PRECEDE THE FILE INPUT CONTROLS
    INPUT NAME= IN TYPE=FILE DETERMINES THE NAME YOU FIND IN _FILES ARRAY
    ABSENCE OF ACTION= ATTRIBUTE IN <FORM> TAG CAUSES POST TO SAME URL
-->
<form name="UploadForm" enctype="multipart/form-data" method="post">
<input type="hidden" name="MAX_FILE_SIZE" value="$max_file_size" />
<p>
Find the file(s) you want to upload and click the "Upload" button below.
</p>

<input type="file" multiple name="userfile[]" size="80" />

<br/>Check this box <input autocomplete="off" type="checkbox" name="overwrite" /> to <b>overwrite</b> existing files.
<input type="submit" value="Upload" />
</form>
EOF;

echo $form;

Open in new window

0

Featured Post

Enroll in June's Course of the Month

June's Course of the Month is now available! Every 10 seconds, a consumer gets hit with ransomware. Refresh your knowledge of ransomware best practices by enrolling in this month's complimentary course for Premium Members, Team Accounts, and Qualified Experts.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

These days socially coordinated efforts have turned into a critical requirement for enterprises.
This article discusses four methods for overlaying images in a container on a web page
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to count occurrences of each item in an array.

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question