phpftp, too much load? same with CURL?

you need to separate all steps, so you can recognize in which point is the problem.

are you telling us the upload goes well and you get the file in /tmp, and then when you start the ftp to the internal host, your file get lost?

Correct.

The file from the upload form is beeing uploaded to the tmp folder correct (can echo the filesize). The error accure when I FTP it - this both count internal (on the same server) and to our external fileservers.

When I use phpftp I just get a blank page without any error messages.
When I use CURL it upload the file but bigger files has a filesize of 0 byte.

Please note, phpftp upload from time to time without problems (also files about 50MB) but receiving the blank page 60-70% when the file exceed 1-2MB.
CURL also upload files about 10MB which is okay, but the successrate is more like 5-10%.

maybe it is a question of timeouts and filesizes.

check you have a .htaccess file on the directory where the upload page is, with something like

<ifModule mod_php4.c>
php_value max_execution_time 1200
php_value memory_limit 200M
php_value post_max_size 200M
php_value upload_max_filesize 200M
</ifModule>

this is to allow a maximun of 200Mb

think I already have all those set very high - havent heard of memory_limit before though. Can it be a memory issue?
Will add the .htaccess file into the dir where the upload script is located

Please note that it can upload files up to 60MB (which is max) from time to time without problems.
 - Getting avg 10 uploads per day of files above 50MB.

Added the info into the htaccess file but still receive uploads with 0 byte filesize (CURL)

mmhh...
this leads me in a different direction then
what is in the error_log of apache when you get a blank page?

Where do I find that error_log?
(know how to use putty ect, just need the location of the log)

you have apache running.
I assume you didn't changed anything

so the apache log should be located at /var/log/apache

inside you should find two logs: access_log and error_log (maybe name slightly different) you must check these in order to see why a blank page.

Didnt changed anything, maby my servers admins did (dont know).

Cant find the apache folder in the log folder.
The content of the "log" folder:

drwxr-xr-x   13 root     root         4096 Jul  4 19:24 ./
drwxr-xr-x   27 root     root         4096 Jun 11 09:16 ../
lrwxrwxrwx    1 root     root           22 May 19 21:12 audit -> /var/log/audit.d/bin.0
drwx------    2 root     root         4096 May 19 21:12 audit.d/
drwxr-xr-x    3 root     root         4096 Apr 29 14:02 bandwidth/
-rw-------    1 root     root            0 Jul  3 04:02 boot.log
-rw-------    1 root     root         1244 Jul  3 04:02 boot.log.1
-rw-------    1 root     root          777 Jun 26 04:02 boot.log.2
-rw-------    1 root     root          342 Jun 19 04:02 boot.log.3
-rw-------    1 root     root         1490 Jun 12 04:02 boot.log.4
drwxr-xr-x    2 bin      bin          4096 Sep 16  2003 canna/
-rw-r--r--    1 root     root            0 May 19 06:52 checkphpv_error.log
-rw-r--r--    1 root     root          882 May 19 06:52 checkphpv_output.log
-rw-r--r--    1 root     root      2130577 Jul  4 19:24 chkservd.log
-rw-r--r--    1 root     root        67879 Apr 29 14:24 cpanel-install-thread0.log
-rw-r--r--    1 root     root       679552 Apr 29 14:23 cpanel-install-thread1.log
-rw-r--r--    1 root     root         1334 May 19 06:49 cpupdate.env
-rw-r--r--    1 root     root        20062 May 19 06:50 cpupdate.log
-rw-------    1 root     root       313606 Jul  4 19:25 cron
-rw-------    1 root     root      1347940 Jul  3 04:02 cron.1
-rw-------    1 root     root      1346427 Jun 26 04:02 cron.2
-rw-------    1 root     root      1346327 Jun 19 04:02 cron.3
-rw-------    1 root     root      1346716 Jun 12 04:02 cron.4
drwxr-xr-x    2 lp       sys          4096 Jul  3 04:02 cups/
drwx------    3 root     root         4096 Jul  4 19:25 dcpumon/
-rw-r--r--    1 root     root        14869 Apr 29 14:34 dmesg
-rw-r-----    1 mailnull mail       669787 Jul  4 19:24 exim_mainlog
-rw-r-----    1 mailnull mail       377907 Jul  3 04:02 exim_mainlog.1.gz
-rw-r-----    1 mailnull mail       345071 Jun 26 04:02 exim_mainlog.2.gz
-rw-r-----    1 mailnull mail       258007 Jun 19 04:02 exim_mainlog.3.gz
-rw-r-----    1 mailnull mail       288485 Jun 12 04:02 exim_mainlog.4.gz
-rw-r-----    1 mailnull mail            0 Jul  3 04:02 exim_paniclog
-rw-r-----    1 mailnull mail           36 Jun 26 04:02 exim_paniclog.1.gz
-rw-r-----    1 mailnull mail           36 Jun 19 04:02 exim_paniclog.2.gz
-rw-r-----    1 mailnull mail           36 Jun 12 04:02 exim_paniclog.3.gz
-rw-r-----    1 mailnull mail           36 Jun  5 04:02 exim_paniclog.4.gz
-rw-r-----    1 mailnull mail         4232 Jul  3 15:57 exim_rejectlog
-rw-r-----    1 mailnull mail         2597 Jul  2 22:44 exim_rejectlog.1.gz
-rw-r-----    1 mailnull mail         1663 Jun 25 09:40 exim_rejectlog.2.gz
-rw-r-----    1 mailnull mail         2555 Jun 19 02:36 exim_rejectlog.3.gz
-rw-r-----    1 mailnull mail         2419 Jun 11 17:46 exim_rejectlog.4.gz
drwx------    2 root     root         4096 May  1 04:02 httpd/
-rw-------    1 root     root        86041 Apr 29 14:34 ksyms.0
-rw-------    1 root     root        85572 Apr 29 12:58 ksyms.1
-r--------    1 root     root     19136220 Jul  4 19:20 lastlog
drwxr-xr-x    2 root     root         4096 Apr 29 14:02 mail/
-rw-------    1 root     root      1311403 Jul  4 19:24 maillog
-rw-------    1 root     root      4297380 Jul  3 04:00 maillog.1
-rw-------    1 root     root      2856517 Jun 26 03:59 maillog.2
-rw-------    1 root     root      3722621 Jun 19 04:02 maillog.3
-rw-------    1 root     root      4572802 Jun 12 04:00 maillog.4
drwxrwsr-x    2 root     mailman      4096 Jul  3 04:02 mailman/
-rw-------    1 root     root       633359 Jul  4 19:24 messages
-rw-------    1 root     root      1441029 Jul  3 04:02 messages.1
-rw-------    1 root     root      2492745 Jun 26 04:02 messages.2
-rw-------    1 root     root      2240957 Jun 19 04:02 messages.3
-rw-------    1 root     root      2297121 Jun 12 04:02 messages.4
-rw-rw----    1 mysql    root       241135 Jun 29 13:42 mysqld.log
-rw-r--r--    1 root     root        18124 Jul  4 04:02 rpmpkgs
-rw-r--r--    1 root     root        18124 Jul  2 04:02 rpmpkgs.1
-rw-r--r--    1 root     root        18124 Jun 25 04:02 rpmpkgs.2
-rw-r--r--    1 root     root        18124 Jun 18 04:02 rpmpkgs.3
-rw-r--r--    1 root     root        18117 Jun 11 04:02 rpmpkgs.4
drwx------    2 root     root         4096 Mar  3 18:35 samba/
-rw-------    1 root     root       421875 Jul  4 19:24 secure
-rw-------    1 root     root       706120 Jul  3 04:00 secure.1
-rw-------    1 root     root      1578737 Jun 26 03:56 secure.2
-rw-------    1 root     root      1266953 Jun 19 04:02 secure.3
-rw-------    1 root     root      1183893 Jun 12 04:00 secure.4
-rw-------    1 root     root            0 Jul  3 04:02 spooler
-rw-------    1 root     root            0 Jun 26 04:02 spooler.1
-rw-------    1 root     root            0 Jun 19 04:02 spooler.2
-rw-------    1 root     root            0 Jun 12 04:02 spooler.3
-rw-------    1 root     root            0 Jun  5 04:02 spooler.4
drwxr-x---    2 squid    squid        4096 Feb 16 14:52 squid/
-rw-r--r--    1 root     root        21949 Apr 29 14:32 stunnel-4.04-build.log
-rw-r--r--    1 root     root            0 Jul  3 04:02 up2date
-rw-r--r--    1 root     root         1061 Jun 30 03:41 up2date.1
-rw-r--r--    1 root     root         2635 Jun 23 01:30 up2date.2
-rw-r--r--    1 root     root         2342 Jun 16 02:21 up2date.3
-rw-r--r--    1 root     root         3948 Jun 11 10:49 up2date.4
drwxr-xr-x    2 root     root         4096 Aug 11  2003 vbox/
-rw-rw-r--    1 root     utmp       113280 Jul  4 19:20 wtmp
-rw-rw-r--    1 root     utmp       711936 Jul  1 03:45 wtmp.1
lrwxrwxrwx    1 root     root           41 May 17 08:59 xferlog -> ../../usr/local/apache/domlogs/ftpxferlog
-rw-r--r--    1 root     root            2 Jul  4 02:51 xferlog.offset
-rw-r--r--    1 root     root            2 Jul  4 19:24 xferlog.offsetftpsep

Is this the one:
/usr/local/apache/logs/error_log ?

hmmm, /etc/httpd/logs also have a error_log :P
(me is confused now :P)

those logs are huge, anything specific to search for?

a) they must not be huge. if they are right now, is because you left them grow unoticed.
b) you can stop apache, rename them, and start apache so you get new fresh logs. then at the moment you get a blank page, you should go directly there and see last log lines to see the error apache is sending when it cannot create the pages.

c) is your hard disk full or your swap? you can check the disk with the utility "df" and the swap with "free"

didnt had a favicon so the logs was full of 404 errors.
cleaned the logs and waiting for new upload reports.

found this to monitor error_log realtime:
tail -f /usr/local/apache/logs/error_log

could it be a permissions issue with the files being dropped into your temp folder having a different UID/GID than the process that is to move the files to their final resting place?

the files are created but have a filesize of 0 bytes - dont know if that is normal in the error you describe above.

Received two new uploads since the error_log was reset, both with 0 byte filesize.
This is what the error_log contain (except alot of 'file does not exist' errors):

[Tue Jul  5 02:36:07 2005] [error] PHP Warning:  MAX_FILE_SIZE of 25000 bytes exceeded - file [avatar=C:\\Documents and Settings\\milenk0\\My Documents\\My Pictures\\ICP\\milenko avatar.jpg] not savedDocuments\\My Pictures\\ICP\\milenko avatar.jpg] not saved in Unknown on line 0
[Tue Jul  5 01:14:59 2005] [error] PHP Warning:  MAX_FILE_SIZE of 62914560 bytes exceeded - file [upld_file=C:\\Documents and Settings\\Beulah Wils
[Tue Jul  5 02:28:24 2005] [error] PHP Warning:  session_destroy(): Session object destruction failed in /home/savefile/public_html/fileh
[Tue Jul  5 02:22:51 2005] [error] PHP Warning:  ob_start(): output handler 'ob_gzhandler' cannot be used after 'URL-Rewriter' in /home/provostp/pu
[Tue Jul  5 02:19:23 2005] [error] (2)No such file or directory: Incorrect permissions on webroot "/home/savefile/public_html/_vti_pvt" and webroot'$
[Tue Jul  5 01:13:20 2005] [error] [client 203.160.1.70] request failed: erroneous characters after protocol string: If-Modified-Since: Wed, 22 Jun
[Mon Jul  4 23:55:36 2005] [error] [client 203.160.1.70] request failed: erroneous characters after protocol string: If-Modifi
[Mon Jul  4 23:35:23 2005] [error] [client 70.68.79.4] Invalid method in request \\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\
[Mon Jul  4 23:34:05 2005] [error] PHP Warning:  POST Content-Length of 1130177048 bytes exceeds the limit of 209715200 bytes in Unknown on line 0
[Mon Jul  4 23:02:54 2005] [error] [client 140.117.170.156] Invalid method in request \\xe3Z
[Mon Jul  4 23:02:55 2005] [error] [client 140.117.170.156] Invalid URI in request GNUTELLA CONNECT/0.6
[Mon Jul  4 22:39:56 2005] [error] [client 203.160.1.70] request failed: erroneous characters after protocol string: If-Modified-Since: Wed, 22

check you php.ini file, what is  MAX_FILE_SIZE set to?

upload_max_filesize er 100MB
cant find MAX_FILE_SIZE in php.ini

sometimes they can upload files about 50-60MB without problems, so I dont think that is the problem - not sure what this problem is about though.

I wonder where MAX_FILE_SIZE is defined then... take a look at this thread dealing with php uploads: http://forums.devshed.com/t26027/s.html
granted it is for windows but the method sounds similar.
they also mention two php.ini variables:
 -> POST_MAX_SIZE and
-> UPLOAD_MAX_FILESIZE

are those not getting in your way?

I wonder if there isn't a quota of some sort letting the user who wants to upload a single file in a session work but the one who wants to upload multiple files within a single session will fail.

HTH

-t

kgp43:

please do not doubt what you are reading in the logs.

file size is being restricted, so you need to increase it's value.

that's why I comment you about increasing all these values in .htaccess, but looking you are in windows, maybe this setting is not being honored. It should, however.

would you mind to try adding all settings I recommended in a previous post into your php.ini an restart apache and try again?

Please note that every upload from the form to the tmp folder is okay, it's just the FTP move that goes wrong.
The max_file_size errors in the logs is becuase they try to upload a file bigger than the allowed 60MB.

Redimido: I can try to change php.ini to the values you suggest, but I though that was only to the form-uploader and not FTP?

how are you instantiating the phpftp function?

how did you change it to accomodate curl?

www.savefile.com/filehost/test.php
(please note you wont get any error when a file is 0 byte)

------------------------------------------------------------------------------------------------------------------
this is the code at the test uploader:


<?
// Get time, 36000 seconds (10h) (60MB at 10Kb/sec)
set_time_limit(36000);

// Display all errors
ini_set('display_errors', true);

// Error handling, 0=off 1=on
error_reporting(1);

// Check if any file is submitted
if (!$_FILES['upld_file']['name']) {

      echo "<font color=red>No file selected</font>";
      exit();
}

// Get the file extention
$file_type = $_FILES['upld_file']['name'];
$file_type = pathinfo($file_type);
$file_type = strtolower($file_type['extension']);

// FTP server info
$username = 'user';
$password = 'pass';
$ip = '123.123.123.123';
$path = 'testuploads/';

// FTP url
$ftpURL = 'ftp://'.$ip.'/'.$path;

// File info
$filepath = $_FILES['upld_file']['tmp_name'];
$filename = $_FILES['upld_file']['name'];
$fp = fopen($filepath, 'rb');

// Check for errors
switch ($_FILES['upld_file']['error']) // http://www.php.net/manual/en/features.file-upload.errors.php
{

      case 1:
            echo "<font color=red>The uploaded file exceeds the upload_max_filesize directive in php.ini</font>";
            break;
      
      case 2:
            echo "<font color=red>The uploaded file exceeds the MAX_FILE_SIZE directive that was specified in the HTML form</font>";
            break;
      
      case 3:
            echo "<font color=red>The uploaded file was only partially uploaded</font>";
            break;
      
      case 4:
            echo "<font color=red>No file was uploaded</font>";
            break;
      
      case 6:
            echo "<font color=red>Missing a temporary folder</font>";
            break;
      
      case 0:
            
            // Post filesize
            echo $_FILES['upld_file']['size']." bytes ready to be uploaded (not uploaded yet)<br />
            <br />
            --------------------------------------------<br />
            <b>STATUS:</b><br />
            --------------------------------------------<br />
            <br />";
      
            // ASCII filetypes
            $ascii_filetypes = array('htm', 'html', 'shtml', 'cnf', 'css', 'forward', 'map', 'pwd', 'txt', 'grp', 'ctl', 'nfo');
      
            if (in_array($file_type, $ascii_filetypes)) {
                        
                  $ch = curl_init();
                  curl_setopt($ch, CURLOPT_URL, $ftpURL.$filename);
                  curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
                  curl_setopt($ch, CURLOPT_INFILE, $fp);
                  curl_setopt($ch, CURLOPT_INFILESIZE, filesize($filepath));
                  curl_setopt($ch, CURLOPT_UPLOAD, 1);
                  curl_exec($ch);
                  fclose($fp);
                  
                  if(curl_errno($ch)) {
                        echo 'Error uploading your file using ASCII mode<br />'.curl_error($ch);
                  }else{
                        curl_close($ch);
                        echo "File uploaded using ASCII mode<br />";
                        echo "Filename: $filename<br />";
                        echo "Filesize: ".$_FILES['upld_file']['size']." bytes<br />";
                  }
            
            }else{
                  
                  $ch = curl_init();
                  curl_setopt($ch, CURLOPT_URL, $ftpURL.$filename);
                  curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
                  curl_setopt($ch, CURLOPT_INFILE, $fp);
                  curl_setopt($ch, CURLOPT_INFILESIZE, filesize($filepath));
                  curl_setopt($ch, CURLOPT_UPLOAD, 1);
                  curl_exec($ch);
                  fclose($fp);
                  
                  if(curl_errno($ch)) {
                        echo 'Error uploading your file using BINARY mode<br />'.curl_error($ch);
                  }else{
                        curl_close($ch);
                        echo "File uploaded using BINARY mode<br />";
                        echo "Filename: $filename<br />";
                        echo "Filesize: ".$_FILES['upld_file']['size']." bytes<br />";
                  }
                  
            }
      break;
      
}
echo "<br /><br /><br />
<font color=red><b>Remember to paste and copy the entire text on this page when you report back! (including this one)</b></font>";
?>

found the error, seems the tmp folder was too small.
Raised it and everything is working again.

what do i do about the points?