Apache on Hyper-V is very slow

Hi all,

I am having a serious problem hosting websites based on Redhat or Centos on Hyper-V.  While on physical server that is not the case even when the CPU and memory is much less than what is been allocated for on Hyper-V.

30 - 50 user's try to access a page to upload document or login, the server gets hanged completely.  sometime we have to force restart because we cannot even login to reboot the server.  but that is not the case with physical server even when the memory and CPU is half of what is been used on Hyper-V.

One of the issues I am facing on Hyper-V is the memory releases, which keeps increasing and never get release unless I use a run /bin/sync; /bin/echo 3 > /proc/sys/vm/drop_caches on crontab to release the memory or force restart the server.

I am getting to a point to move all my web onto physical machine due to that problem.  can you please help me resolve that problem.  Is the fix from the Windows, Linux or Apache?
rawandnetAsked:
Who is Participating?
 
rawandnetConnect With a Mentor Author Commented:
The running modules are followings:
I use MySQL, PHP, Drupal.
Would you be able to tell which one is not required?

 core_module (static)
 so_module (static)
 http_module (static)
 access_compat_module (shared)
 actions_module (shared)
 alias_module (shared)
 allowmethods_module (shared)
 auth_basic_module (shared)
 auth_digest_module (shared)
 authn_anon_module (shared)
 authn_core_module (shared)
 authn_dbd_module (shared)
 authn_dbm_module (shared)
 authn_file_module (shared)
 authn_socache_module (shared)
 authz_core_module (shared)
 authz_dbd_module (shared)
 authz_dbm_module (shared)
 authz_groupfile_module (shared)
 authz_host_module (shared)
 authz_owner_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 cache_module (shared)
 cache_disk_module (shared)
 data_module (shared)
 dbd_module (shared)
 deflate_module (shared)
 dir_module (shared)
 dumpio_module (shared)
 echo_module (shared)
 env_module (shared)
 expires_module (shared)
 ext_filter_module (shared)
 filter_module (shared)
 headers_module (shared)
 include_module (shared)
 info_module (shared)
 log_config_module (shared)
 logio_module (shared)
 mime_magic_module (shared)
 mime_module (shared)
 negotiation_module (shared)
 remoteip_module (shared)
 reqtimeout_module (shared)
 rewrite_module (shared)
 setenvif_module (shared)
 slotmem_plain_module (shared)
 slotmem_shm_module (shared)
 socache_dbm_module (shared)
 socache_memcache_module (shared)
 socache_shmcb_module (shared)
 status_module (shared)
 substitute_module (shared)
 suexec_module (shared)
 unique_id_module (shared)
 unixd_module (shared)
 userdir_module (shared)
 version_module (shared)
 vhost_alias_module (shared)
 dav_module (shared)
 dav_fs_module (shared)
 dav_lock_module (shared)
 lua_module (shared)
 mpm_prefork_module (shared)
 proxy_module (shared)
 lbmethod_bybusyness_module (shared)
 lbmethod_byrequests_module (shared)
 lbmethod_bytraffic_module (shared)
 lbmethod_heartbeat_module (shared)
 proxy_ajp_module (shared)
 proxy_balancer_module (shared)
 proxy_connect_module (shared)
 proxy_express_module (shared)
 proxy_fcgi_module (shared)
 proxy_fdpass_module (shared)
 proxy_ftp_module (shared)
 proxy_http_module (shared)
 proxy_scgi_module (shared)
 proxy_wstunnel_module (shared)
 ssl_module (shared)
 systemd_module (shared)
 cgi_module (shared)
 fcgid_module (shared)
 php5_module (shared)
0
 
arnoldConnect With a Mentor Commented:
Virtual machines are contending with resource allocations that differ from the fewer contentions they face on physical systems.

You have to identify what the physical host is doing, how much resources you allocated to the VM, where the bottle kneck is VM resource, or dealing with processing the network.

Memory accumulation as part of ...resource demand.
You freeing up the memory is counter productive.

The spec based on what you want the system to do seems under resourced based on your question.

Let's try this you have a crew of people, one of the tasks your crew performs is unloading in addition to other tasks. You based on the criteria said you only need four members.
One deals with task1
One deals with task2
One deals with task 3
One deals with unloading
In all cases when the need exists you can allocate the crew members as needed.
A supply of equipment comes in, you move one from task1 to unload.
The delivery of items is constant
Over time you all four ended up in the unloading task.
Periodically, you see the other tasks not being done, so you pull those members back.
And the cycle starts again until


First thing is to identify how your setup operates, how resources are allocated and then where the resource contention is that leads to this pergormance hit.
The VM uses local or host shared storage.
Is the issue that these uploads get stuck in write queues to disk.
Does the hyper-v host paging (sees a demand on memory more that the available physical memory)

Apache is the web server, there are other components such as php, MySQL, etc that are need and these also require and consume resources.

Please post spec of the hyper-v host system CPU, memory, disk setup, network.
How many VMs and their spec CPU, memory, disk and what services they provide.
0
 
nociConnect With a Mentor Software EngineerCommented:
To what extent can you replace apache with f.e. nginx or lighttpd.
I used to have a physical system showing the same issues you experience, (memory was the limiting factor there).
replacing apache with (first lighttpd , later nginx) + php-fpm   meant i could host 2-3 times the amount of users on the same hardware.

Also using php-fpm allows for distributing resources over multiple systems more specifically setup for the load needed.

Depending on needed features lighttpd can be sufficient, or maybe nginx can be useful. Both are very capable of handling a lot of connections and serving the static part of the content.
both need (fastcgi [ for php this is php-fpm ] or cgi) to offload script processing.
With php-fpm one can configure php - engines for specific functions.. f.e use one engine for bulk website and a  specific engine with different setup (f.e. different security & size settings) for file uploads .
0
Cloud Class® Course: CompTIA Cloud+

The CompTIA Cloud+ Basic training course will teach you about cloud concepts and models, data storage, networking, and network infrastructure.

 
rawandnetAuthor Commented:
Server Processor is: Intel(R) Xeon(R) CPU E5450 @ 3.GHz (2 Processor)
The server is hosting 5 VM, they are all for HTTP access, and they rarely used.  Their CPU usage is 0% most of the time. And maximum Memory does not exceed 2048
The RAM is 32GByte
It is using Dynamic Memory:
Minimum RAM: 512MB
Maximum RAM: 1048576MB
The network setup is EtherChannel with two 1Gbps interfaces bundled together.
The disk format is NTFS and has enough memory.

Somehow when memory get allocated for the Linux Redhat VM, it will not get released. Unlike Windows VM, un-used memory will be released to other servers.

Concerning replacing Apache with other web server, I have not used other web server except Apache and I assume it should be powerful enough and most reliable engine.  That is why it is used all around the world
0
 
arnoldCommented:
Set the memory  limit at how much you want to allow the Linux VM to consume.

What about the storage?
Your issue is not with the application within the VM it is the scheduling/resource allocation of the host to the VM when the network traffic increases and the VM needs resources to access the files/data to serve.
0
 
nociSoftware EngineerCommented:
Actually for the heavy haulage websites lighttpd and/or nginx are used. As apache takes a lot of resources.  (nginx / lighttpd can easily accomodate 2* or 3* the request/clients compared to Apache. (YMMV).
apache loads all modules and keeps those in all sessions. Other webservers have no such modules, and rely on php-fpm or other fastcgi based script processors...
Linux has no concept of dynamic changing memory so indeed it will not release memory. HyperV can use that on windows systems as they are built together to be able to do that.
Then again adjusting memory will cost something, most probably some data will need to be swapped to disk, this will slow down any system to slightly faster then disk speed and get them crawling in stead of running.
0
 
rawandnetAuthor Commented:
Thanks for your input.

So, you advise changing Apache to Lighttpd browser? Would you say that would solve the proble?
Does Lighttpd resolve the dynamic memory reclease issue?
does Lighttpd support PHP and MySQL?
Can I install it from Redhat using Yum?
0
 
rawandnetAuthor Commented:
Is there any way to tweak Apache server to use less resource? As those other Web servers are not included in Redhat Repository? therefore, i don't feel so comfortable installing it on my server?
0
 
arnoldCommented:
Yes, look at httpd.conf look at clients that its paws, how many requestsithandles.

Note depending on yoursitethe issue might related toresourcesneededby php, MySQL, etc which are outside
To illustrate a point, you have a person who receives requests for products (this is where Apache is)
Upon receiving the request, all office and warehouse resources (php And MySQL) are tasked to get the issue addressed.
The person receiving the call, is done within a couple of minutes.

The others are who spend the majority of time to report ask they are done at which point the person who got the call, calls back to indicate the items are on the way.
0
 
nociSoftware EngineerCommented:
Lighttpd doesn't claim that much memory to start with, so not claimed also doesn't need to be released.
php load would probably be the same CPU wise, resource wise the memory for storing the php scripting might be the same, most probably the sharing of the code in php-fpm (php interpreter code) is handled better.
mysql would be the same as that is an outside server (wrt. apache/lighttpd) anyway.

My suggestion would be to build an example server (VM) using each of the setup and then compare them on a test bench.
(some things like .htaccess will need to be rewritten)

OTOH if you want to stick with Apache anyway, then disable all modules that are not strictly needed.  f.e. if mod_perl is not used, disable it,
same for mod_env, mod_echo, mod_dumpio, mod_expires, mod_proxy etc.. etc.  instead of the generic if modules exists load it.
0
 
nociConnect With a Mentor Software EngineerCommented:
Probably most of the auth/authz modules as drupal has it's own scheme.
all proxy modules.

Is Apache also terminating the https? if not (because of load balancing, haproxy or whatever frontend, ssl_module
cgi/fcgi come to mind.
lua_modules. dav modules (unless neded for access by phones)
are you running this a loadbalancer, probably not so lb* modules are useless.
dir & userdir probably can go aswell....
filter modules can probably go,
if you are not running memcache then memcache can go aswell.

after that start with disabling one or two and check if the app still works., if not.. reenable them and try another for removal.
0
 
rawandnetAuthor Commented:
Thank you all for your input
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.