[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 675
  • Last Modified:

how to convert images in PHP

Hello, I have a web application that involves lots of images upload and we are converting those images on the flight using PHP.
this is creating lots of logs in the apache logs:
i.e: Conversion of ../../backup/files/images/2011/09/9/83726.jpg into ../../backup/files/images/thumbs/200/2011/09/9/83726.jpeg OK
this is a proof that conversion requests are going through apache.

From another side apache memory is constantly increasing and within 24 hours apache is crashing.
Could the image conversion be related to that?
I am on a windows 2003 server.
thanks



[Tue Sep 06 19:25:34 2011] [warn] (OS 64)Unrecognized Win32 error code 64: winnt_accept: Asynchronous AcceptEx failed.
Cannot initialize zend_mm storage [win32]
[Tue Sep 06 19:25:39 2011] [notice] Parent: child process exited with status 255 -- Restarting.
[Tue Sep 06 19:26:04 2011] [notice] Apache/2.2.17 (Win32) PHP/5.2.8 configured -- resuming normal operations
[Tue Sep 06 19:26:04 2011] [notice] Server built: Oct 18 2010 01:58:12
[Tue Sep 06 19:26:04 2011] [notice] Parent: Created child process 3372
[Tue Sep 06 19:26:04 2011] [notice] Child 3372: Child process is running
[Tue Sep 06 19:26:04 2011] [notice] Child 3372: Acquired the start mutex.
[Tue Sep 06 19:26:04 2011] [notice] Child 3372: Starting 500 worker threads.
[Tue Sep 06 19:26:04 2011] [notice] Child 3372: Starting thread to listen on port 80.

0
jmhabis2
Asked:
jmhabis2
  • 6
  • 4
  • 3
  • +2
1 Solution
 
Mark BradyCommented:
Apache is creating log entries because you are using PHP script to convert the images and that goes through the Apache Server. There is no way of avoiding the log files. However, if you have large amounts of files being uploaded and this is causing memory to run low or even run out, you could consider other options such as:

1: Have the conversion script on a different server altogether so that server gets the load. You would need to either create a link to the image so it reflects on your main website or somehow get it back to your uploads folder.

2: Use an online image converter to do the converting first. For example, you could contact someone like http://www.coolutils.com/online/image-converter/ and ask if you could have a copy of their converting script for use - I mean you could ask if you could use their site to convert files (I doubt they would allow this but there may be other converting sites where you could upload to and have the result saved to your site.

3: Ask/insist your users only upload .jpg/.gif/.png files (or your choice) only. This is my favorite option.
4: Deal with the other image types and let people use whatever they have. No big problem here.
5: Finally, you could look at your config and see if you can up the amount of memory/buffer that Apache is using. It may even be in the php.ini file but it sounds like Apache does not have enough memory to keep converting files. Because when Php converts a file it actually creates a new file on your server (not on the users computer) then it copies the data from the uploaded file to the new file. The server will be dealing with two images at the same time which is not a bad thing as long as you have the memory for it.

Have you limited the file size a user can upload?

You may also need to check your upload/conversion script to make sure that the files are being closed/destroyed and memory released after each conversion. If there is a memory leek or the server is allocating memory for the file creation and not releasing it then you will be in trouble.

I recommend looking at the script first. I'm no expert in image conversion but I thought I would state a few of the obvious to help you out.
Good Luck :)
0
 
DerokorianCommented:
On of the biggest mistakes people make with image processing in php is forgetting to call imagedestroy($image_resource) after finishing with the image. even when you output to the browser this is necessary. I've never figured out why, but it seems as if images aren't automatically removed from the cache when the script ends (unlike other resources). The only reason I suggest this might be a problem is because you say the memory takes time to fill up after many many calls to the script. I would check to make sure of this. You could also make sure GC is on often enough. For more info on that: http://php.net/manual/en/features.gc.php
0
 
jmhabis2Author Commented:
@Dekorian
hi, isn't the used memory is deallocated when the function ends?
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
Ray PaseurCommented:
Agree with the suggestions from Elvin66 and Derokorian.  If you've got the image conversion code right it sounds like a memory leak.  If you want to post the code you're using for the file conversion we may be able to show you how to find out.  One of these functions might help.
http://us.php.net/manual/en/function.memory-get-usage.php
http://us.php.net/manual/en/function.memory-get-peak-usage.php

Another possibility might be to store the uploaded files and use an asynchronous script to do the conversions.  If you do that, you should be able to watch the Apache logs as the async script it running.

What version of PHP are you using?
0
 
DerokorianCommented:
Hi jmhabis,
>> hi, isn't the used memory is deallocated when the function ends?
PHP is supposed to deallocate everything used by the process when the process ends. However In my experience I have found many people have memory problems as a result of image processing. In an ideal world things work as they are supposed to at all times, however we do not live in a perfect world. I don't know why this happens with PHP and the GD library - maybe it cleans up everything but GD properly? I'm not sure but I've found a large amount of these problems are fixed with image_destroy.

Again I have no solid reasoning for it, just my previous experience with similar problems. I agree with Ray - if you wanted to post code we may be able to help isolate and fix the problem if this isn't it.
0
 
Ray PaseurCommented:
And I agree with Derokorian.  PHP is open source and therefore it depends to some extent on the contributions of a large and diverse developer community.  Maybe the GD functions did not get as much attention as they needed, in terms of releasing memory at function end?  NB: "function end" is not the same as "process end."
0
 
jmhabis2Author Commented:
we are using a third party application similar to imagemagick. Do you think this can eat up apache memory? I am using the exec command to call the application.

as a general note, anyway to trace the memory leaks besides zend server?
0
 
Ray PaseurCommented:
anyway to trace the memory leaks besides zend server?

See the post at ID:36975790 -- maybe you can try using those functions.
0
 
hankknightCommented:
There are known memory leaks in GD, the most commonly used PHP library for managing images.

MagickWand is a good alternative:
http://www.magickwand.org/

PHP ImageMagick is also good:
http://php.net/manual/en/book.imagick.php

Also, you may consider using ImageMagick with shell_exec().
0
 
jmhabis2Author Commented:
We don't use GD to convert images. We do use imagemagick and nconvert. We call them using the exec() function. Of course the request is passed through apache. I'm not sure if this is what is causing memory issues in apache. We had known issues with the exec function before. The problem was that the request is sent to server and executed successfully however no response is returned back from exec() causing apache to keep its child process open. Measure this on thousands of images being converted the result  was catastrophic and the web server kept on freezing and then we had to stop/start apache to kill all the processes. The solution was to put session_write_close() before the exec function and session_start() after. The solution is very strange i know but it did the job. So many other people faced the same problem and solved using the same way i did. It turned out exec had issues with php when called through apache. So what i am assuming is that even though the thread issue was solved but maybe apache is not cleaning memory properly.
0
 
hankknightCommented:
"We had known issues with the exec function before. The problem was that the request is sent to server and executed successfully however no response is returned back from exec() causing apache to keep its child process open."

That explains the problem.  If possible, use MagickWand instead of exec().  If that is not possible, try making the exec command run in the background:

exec('nohup xyz &');
// replace xyz with your nconvert code

Open in new window


0
 
jmhabis2Author Commented:
what will be the benefit of running the request in the background? wouldn't this keep the apache thread open and what would happen with the memory reserved by apache?
0
 
hankknightCommented:
If you run the process in the background, it will immediately close the Apache connection.   The memory reserved by Apache would then become available again and only the memory required by ImageMagick would be in use.
0
 
jmhabis2Author Commented:
If you run in background how can i catch the response of the exec function in order to know when the command finished so i can display a notification to the user or perform some action?

thanks
0
 
hankknightCommented:
If you need to catch the response you cannot run the process in the background.

Given your problem and your requirements, I am afraid the only way to improve performance would be to either use more powerful hardware or use a PHP library instead of exec.

These are both good.  Either one of them would improve performance.  
http://www.magickwand.org/
http://php.net/manual/en/book.imagick.php

I personally prefer MagickWand.  Both of these packages use ImageMagick's algorithms, so the quality should be the same.
0
 
jmhabis2Author Commented:
I'm still not sure that using for example magickwand could solve my problem but it may be a solution
0

Featured Post

Nothing ever in the clear!

This technical paper will help you implement VMware’s VM encryption as well as implement Veeam encryption which together will achieve the nothing ever in the clear goal. If a bad guy steals VMs, backups or traffic they get nothing.

  • 6
  • 4
  • 3
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now