Link to home
Start Free TrialLog in
Avatar of RBraat
RBraatFlag for Netherlands

asked on

IIS 10 on Microsoft Server 2016 with PHP7 FastCGI invokes cron-job by executing URL serverside, results in Error 500 (probably timeout?)

Hi,
I'm running Joomla 3.8.5 on Windows server 2016 with IIS v10.0, PHP 7.2.2 (32-bit) all on an Azure VM Standard A2 v2 (2 vcpus, 4 GB memory).
On Joomla I am running JomSocial 4.5.1, which is community software. Users can amoung other things, upload video to post on site. Uploaded video's are converted with ffmpeg which I installed on my server, version version N-90143-gb6652f5100.
ffmpeg is called by a scheduled task running periodically.

When uploading no so big video's, everything works fine, video's got converted an publshed on site as they should.
When uploading bigger video's (+/- 10-50Mb) video's don't get published.

I've gone through extensive troubleshooting with JomSocial support. We can see video upload is fine.
To troubleshoot we stopped scheduled task on web server (whick invokes ffmpeg with URL) and ran URL to invoke ffmeg manually on same web server.

For small video's, you can see output of URL telling video conversion is succesfull and video is published on site.
For larger video's, there is no output and after about +/- a minute, error 500 - Internal server error is shown.
In background ffmpeg is still runing for minutes after that and when you look in the file location where converted files end up, you can see the converted video is just fine. But, the video is not published because the URL which invokes ffmpeg and published video afterwards, has died (error 500).

So ffmpeg is not the problem, because conversion was successfull. But some way, the batch-process which the URL sets off, was not able to finish.

To troubleshoot, I've upgraded the Azure VM to A8_V2 (8 CPU's, 16Gb of memory). During ffmpeg conversion on my 2 core VM, the ffmpeg process raises CPU load to 100% over a longer period. On this temporary 8 core VM, ffmpeg is also high, but not 100% and also the conversion process is much quicker. Now the URL which invokes ffmpeg shows conversion is ok and video is published. This is where I get error 500 on the 2 core VM.

So my conclusion is, that when server is in less stress, the whole conversion and publishing process works, and when server is in stress, conversion also works, but the job which should wait till conversion is complete, has died before it can finish publication of the video.

I asumed that inside IIS I could try to raise the "Ping Period" of the worker process, on which the website runs. So I raised ping from default 30 sec. to 120 sec. Made no difference, same error 500. I also disabled the ping to stop health monitoring, but also no difference.

Hope anyone has an idea how to tackle this problem. Running permanently a costly 8 core VM for only a very short moment of conversion and publication of video is absurd, since during normal load my 2 core VM is more than enough and 4 times cheaper.

Regards, Rini
Avatar of noci
noci

Error 500 means the intermediate proxy is unable to connect to the backend system.
==> can the proxy server do the queries to the backend system (can you browse from there...)
if not try to fix the issues n reaching the backend first.
If the backend IS reachable then you need to check the proxy config & error logs of the webserver.
why don't you just split the processing of the video onto another azure instance?
save the video to a azure storage location run an azure batch file to do the conversion and retrieve the video to be published.

What happens if several users upload a video at the same time?  Auto-Scaling will help here (use cpu load)
Avatar of RBraat

ASKER

Hi noci,
Please read my question carefully, the url runs on web servers itselve, there is no proxy.
Regards Rini
Avatar of RBraat

ASKER

Hi David,
I cannot split the process. I use commercial software JomSocisl, that is as it is. I’m not a developer. I understand splitting would have done advantages, but that is not an option.
Regards Rini
Avatar of RBraat

ASKER

I’m quite sure it has something to do with a timeout, limitation or threshold within IIS, which happens on slower machine with 2 cores, and not on faster machine with 8 cores, where FFmpeg finishes within some threshold or acceptable time.
I also tried settings queue length from 1000 to 5000 but that also did not help me.
Regards, Rini
have you tried scaling your web server depending upon cpu load?
Avatar of RBraat

ASKER

Sorry no experience with that
SOLUTION
Avatar of Julian Hansen
Julian Hansen
Flag of South Africa image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Understood - sounds like you are on top of it.
Avatar of RBraat

ASKER

Increasing script execution time with set_time_limit, together with fastcgi Params solved issue