Link to home
Start Free TrialLog in
Avatar of Carletto
CarlettoFlag for Italy

asked on

mysql server stuck

Hello ,
I have the following problem on my web server running apache+php and mysqld , almost once a week I have MySQL blocked with the following message

service mysqld status
mysqld dead but subsys locked


I need to restart the server. this is what I have found in my messages.log file ....
can you help me for the troubleshooting.
Thanks Carlettus

Dec  6 10:40:39 webserver kernel: httpd invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
Dec  6 10:40:39 webserver kernel: httpd cpuset=/ mems_allowed=0
Dec  6 10:40:39 webserver kernel: Pid: 3587, comm: httpd Tainted: G        W  ---------------    2.6.32-504.1.3.el6.x86_64 #1
Dec  6 10:40:39 webserver kernel: Call Trace:
Dec  6 10:40:39 webserver kernel: [<ffffffff810d40c1>] ? cpuset_print_task_mems_allowed+0x91/0xb0
Dec  6 10:40:39 webserver kernel: [<ffffffff81127300>] ? dump_header+0x90/0x1b0
Dec  6 10:40:39 webserver kernel: [<ffffffff8122ea4c>] ? security_real_capable_noaudit+0x3c/0x70
Dec  6 10:40:39 webserver kernel: [<ffffffff81127782>] ? oom_kill_process+0x82/0x2a0
Dec  6 10:40:39 webserver kernel: [<ffffffff811276c1>] ? select_bad_process+0xe1/0x120
Dec  6 10:40:39 webserver kernel: [<ffffffff81127bc0>] ? out_of_memory+0x220/0x3c0
Dec  6 10:40:39 webserver kernel: [<ffffffff811344df>] ? __alloc_pages_nodemask+0x89f/0x8d0
Dec  6 10:40:39 webserver kernel: [<ffffffff8116c79a>] ? alloc_pages_vma+0x9a/0x150
Dec  6 10:40:39 webserver kernel: [<ffffffff81160342>] ? read_swap_cache_async+0xf2/0x160
Dec  6 10:40:39 webserver kernel: [<ffffffff81160e69>] ? valid_swaphandles+0x69/0x150
Dec  6 10:40:39 webserver kernel: [<ffffffff81160437>] ? swapin_readahead+0x87/0xc0
Dec  6 10:40:39 webserver kernel: [<ffffffff8114f69d>] ? handle_pte_fault+0x6dd/0xb00
Dec  6 10:40:39 webserver kernel: [<ffffffff8114fcea>] ? handle_mm_fault+0x22a/0x300
Dec  6 10:40:39 webserver kernel: [<ffffffff8104d0d8>] ? __do_page_fault+0x138/0x480
Dec  6 10:40:39 webserver kernel: [<ffffffff810097cc>] ? __switch_to+0x1ac/0x320
Dec  6 10:40:39 webserver kernel: [<ffffffff815299de>] ? thread_return+0x4e/0x7d0
Dec  6 10:40:39 webserver kernel: [<ffffffff8118f981>] ? __fput+0x1a1/0x210
Dec  6 10:40:39 webserver kernel: [<ffffffff8152ffde>] ? do_page_fault+0x3e/0xa0
Dec  6 10:40:39 webserver kernel: [<ffffffff8152d395>] ? page_fault+0x25/0x30
Dec  6 10:40:39 webserver kernel: Mem-Info:
Dec  6 10:40:39 webserver kernel: Node 0 DMA per-cpu:
Dec  6 10:40:39 webserver kernel: CPU    0: hi:    0, btch:   1 usd:   0
Dec  6 10:40:39 webserver kernel: Node 0 DMA32 per-cpu:
Dec  6 10:40:39 webserver kernel: CPU    0: hi:  186, btch:  31 usd: 178
Dec  6 10:40:39 webserver kernel: Node 0 Normal per-cpu:
Dec  6 10:40:39 webserver kernel: CPU    0: hi:  186, btch:  31 usd: 164
Dec  6 10:40:39 webserver kernel: active_anon:1112576 inactive_anon:281006 isolated_anon:1184
Dec  6 10:40:39 webserver kernel: active_file:227 inactive_file:259 isolated_file:242
Dec  6 10:40:39 webserver kernel: unevictable:0 dirty:0 writeback:54 unstable:0
Dec  6 10:40:39 webserver kernel: free:25520 slab_reclaimable:4409 slab_unreclaimable:15680
Dec  6 10:40:39 webserver kernel: mapped:423 shmem:15 pagetables:40599 bounce:0
Dec  6 10:40:39 webserver kernel: Node 0 DMA free:15672kB min:164kB low:204kB high:244kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15280kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Dec  6 10:40:39 webserver kernel: lowmem_reserve[]: 0 3000 6030 6030
Dec  6 10:40:39 webserver kernel: Node 0 DMA32 free:51196kB min:33540kB low:41924kB high:50308kB active_anon:2145944kB inactive_anon:547548kB active_file:0kB inactive_file:64kB unevictable:0kB isolated(anon):1920kB isolated(file):72kB present:3072160kB mlocked:0kB dirty:0kB writeback:76kB mapped:72kB shmem:4kB slab_reclaimable:4740kB slab_unreclaimable:16520kB kernel_stack:960kB pagetables:60376kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:18 all_unreclaimable? no
Dec  6 10:40:39 webserver kernel: lowmem_reserve[]: 0 0 3030 3030
Dec  6 10:40:39 webserver kernel: Node 0 Normal free:35212kB min:33872kB low:42340kB high:50808kB active_anon:2304360kB inactive_anon:576476kB active_file:908kB inactive_file:972kB unevictable:0kB isolated(anon):2816kB isolated(file):896kB present:3102720kB mlocked:0kB dirty:0kB writeback:140kB mapped:1620kB shmem:56kB slab_reclaimable:12896kB slab_unreclaimable:46200kB kernel_stack:2496kB pagetables:102020kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:3648 all_unreclaimable? no
Dec  6 10:40:39 webserver kernel: lowmem_reserve[]: 0 0 0 0
Dec  6 10:40:39 webserver kernel: Node 0 DMA: 4*4kB 3*8kB 3*16kB 1*32kB 1*64kB 1*128kB 0*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15672kB
Dec  6 10:40:39 webserver kernel: Node 0 DMA32: 1597*4kB 355*8kB 261*16kB 221*32kB 252*64kB 88*128kB 3*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 51196kB
Dec  6 10:40:39 webserver kernel: Node 0 Normal: 3969*4kB 695*8kB 323*16kB 97*32kB 32*64kB 5*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 35212kB
Dec  6 10:40:39 webserver kernel: 96064 total pagecache pages
Dec  6 10:40:39 webserver kernel: 95309 pages in swap cache
Dec  6 10:40:39 webserver kernel: Swap cache stats: add 8946555, delete 8851246, find 2532051/3329682
Dec  6 10:40:39 webserver kernel: Free swap  = 0kB
Dec  6 10:40:39 webserver kernel: Total swap = 6127612kB
Dec  6 10:40:39 webserver kernel: 1572848 pages RAM
Dec  6 10:40:39 webserver kernel: 74597 pages reserved
Dec  6 10:40:39 webserver kernel: 128568 pages shared
Dec  6 10:40:39 webserver kernel: 1465900 pages non-shared
Dec  6 10:40:39 webserver kernel: [ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
Dec  6 10:40:39 webserver kernel: [  381]     0   381     2757       20   0     -17         -1000 udevd
Dec  6 10:40:39 webserver kernel: [  666]     0   666     2756       20   0     -17         -1000 udevd
Dec  6 10:40:39 webserver kernel: [ 1271]     0  1271     6899       25   0     -17         -1000 auditd
Dec  6 10:40:39 webserver kernel: [ 1291]     0  1291    62272       24   0       0             0 rsyslogd
Dec  6 10:40:39 webserver kernel: [ 1345]     0  1345    16672       22   0     -17         -1000 sshd
Dec  6 10:40:39 webserver kernel: [ 1382]     0  1382    27075       25   0       0             0 mysqld_safe
Dec  6 10:40:39 webserver kernel: [ 1673]    27  1673  1332960   132699   0       0             0 mysqld
Dec  6 10:40:39 webserver kernel: [ 1794]     0  1794    20471       46   0       0             0 master
Dec  6 10:40:39 webserver kernel: [ 1804]    89  1804    20537       42   0       0             0 qmgr
Dec  6 10:40:39 webserver kernel: [ 1815]     0  1815    29342       45   0       0             0 crond
Dec  6 10:40:39 webserver kernel: [ 1830]     0  1830     1016       18   0       0             0 mingetty
Dec  6 10:40:39 webserver kernel: [ 1832]     0  1832     1016       18   0       0             0 mingetty
Dec  6 10:40:39 webserver kernel: [ 1834]     0  1834     1016       18   0       0             0 mingetty
Dec  6 10:40:39 webserver kernel: [ 1836]     0  1836     1016       18   0       0             0 mingetty
Dec  6 10:40:39 webserver kernel: [ 1838]     0  1838     1016       18   0       0             0 mingetty
Dec  6 10:40:39 webserver kernel: [14910]     0 14910   125300      618   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [14914]    48 14914   171204     9209   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [15049]    48 15049   167619     8213   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [15087]     0 15087     1016       18   0       0             0 mingetty
Dec  6 10:40:39 webserver kernel: [15132]    48 15132   164847    11951   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [15141]    48 15141   167926    13120   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [15371]    48 15371   171201     4489   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [18235]    48 18235   167812     8793   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [29888]    48 29888   165688     4446   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [29911]    48 29911   165194    12839   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [  459]    48   459   163556     7124   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 1415]    48  1415   166450    15231   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2611]    89  2611    20491       38   0       0             0 pickup
Dec  6 10:40:39 webserver kernel: [ 2651]    48  2651   168032    11028   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2655]    48  2655   165090    11180   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2659]    48  2659   164552    10391   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2660]    48  2660   164415    10011   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2662]    48  2662   162783    11679   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2666]    48  2666   167414    10892   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2695]    48  2695   166553    13375   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2697]    48  2697   164415     9689   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2875]    48  2875   161939     4870   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 2932]    48  2932   163021     6452   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3438]    48  3438   161261     5780   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3442]    48  3442   160257     5486   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3443]    48  3443   163078     4536   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3446]    48  3446   156656     5059   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3447]    48  3447   160509     5640   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3448]    48  3448   164011     5966   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3449]    48  3449   162812     5192   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3452]    48  3452   163131     6159   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3453]    48  3453   162518     5179   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3454]    48  3454   160371     5795   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3455]    48  3455   162719     4868   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3457]    48  3457   164464     8564   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3458]    48  3458   160354     5035   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3459]    48  3459   162747     7241   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3460]    48  3460   161069     5431   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3471]    48  3471   161777     5465   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3475]    48  3475   160203     5812   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3477]    48  3477   160193     5882   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3478]    48  3478   162428     6892   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3479]    48  3479   159616     5623   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3480]    48  3480   160320    10612   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3481]    48  3481   160334     3495   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3482]    48  3482   164191     6593   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3483]    48  3483   164511     6671   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3484]    48  3484   161197     4736   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3485]    48  3485   163324     5757   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3486]    48  3486   163984     9186   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3487]    48  3487   160319     5777   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3488]    48  3488   160572     5442   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3489]    48  3489   160213     4886   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3490]    48  3490   158944     7225   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3504]    48  3504   160195     4754   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3507]    48  3507   161250     6341   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3512]    48  3512   160405     5764   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3515]    48  3515   159921     4036   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3516]    48  3516   161750     4856   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3517]    48  3517   164205     7807   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3518]    48  3518   161544     6430   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3519]    48  3519   160612     5053   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3524]    48  3524   164361     8225   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3526]    48  3526   164011     8629   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3527]    48  3527   164222     8753   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3528]    48  3528   164099     8897   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3529]    48  3529   164153     9003   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3530]    48  3530   164056     6195   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3531]    48  3531   164122     7421   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3539]    48  3539   164110     7457   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3541]    48  3541   163412     5950   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3542]    48  3542   164034     6655   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3547]    48  3547   164350     5834   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3550]    48  3550   164286     6560   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3553]    48  3553   164200     5611   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3554]    48  3554   163476     6883   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3555]    48  3555   162851     5575   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3557]    48  3557   163232     6217   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3559]    48  3559   163158     6017   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3561]    48  3561   163708     7785   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3562]    48  3562   164188     5776   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3569]    48  3569   160651     5929   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3570]    48  3570   162607     5417   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3571]    48  3571   163247     5803   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3579]    48  3579   161617     7113   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3584]    48  3584   164324     6324   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3585]    48  3585   164198     6163   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3586]    48  3586   163156     6615   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3587]    48  3587   159437     5491   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3588]    48  3588   163513     5621   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3589]    48  3589   163439     6351   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3590]    48  3590   164331     6355   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3592]    48  3592   163054     6183   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3593]    48  3593   164068     7035   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3594]    48  3594   163920     6686   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3595]    48  3595   164391     6569   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3596]    48  3596   160206     5272   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3597]    48  3597   162652     6903   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3598]    48  3598   164111     6315   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3600]    48  3600   154804     4390   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3621]    48  3621   162491     5827   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3622]    48  3622   157071     5270   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3627]    48  3627   162074     5126   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3629]    48  3629   163585     6560   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3630]    48  3630   163053     6583   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3631]    48  3631   157702     6255   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3633]    48  3633   163232     6019   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3639]    48  3639   163262     6794   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3641]    48  3641   163234     6104   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3642]    48  3642   163164     6035   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3646]    48  3646   157273     5335   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3647]    48  3647   163184     6561   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3648]    48  3648   156056     4971   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3652]    48  3652   162875     6356   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3660]    48  3660   163103     6335   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3661]    48  3661   163409     6564   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3666]    48  3666   162476     5195   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3667]    48  3667   166255    11027   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3668]    48  3668   156752     5596   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3669]    48  3669   162550     6021   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3670]    48  3670   154081     5012   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3671]    48  3671   162412     5961   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3679]    48  3679   162221     6012   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3684]    48  3684   162550     6060   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3685]    48  3685   162390     6727   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3686]    48  3686   162668     6544   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3688]    48  3688   156753     5024   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3689]    48  3689   161360     6404   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3690]    48  3690   163149     7411   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3693]    48  3693   162413     6985   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3695]    48  3695   156876     5072   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3698]    48  3698   157271     6025   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3700]    48  3700   162091     6609   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3701]    48  3701   162091     6676   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3702]    48  3702   162049     6529   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3711]    48  3711   159072     7218   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3712]    48  3712   161977     6403   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3713]    48  3713   161776     6565   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3717]    48  3717   156872     4650   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3718]    48  3718   162486     7245   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3720]    48  3720   162408     6793   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3722]    48  3722   162091     6973   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3723]    48  3723   162230     6383   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3724]    48  3724   159628     6915   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3725]    48  3725   156872     5252   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3730]    48  3730   162084     6796   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3732]    48  3732   162148     7038   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3735]    48  3735   161977     6663   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3736]    48  3736   160187     5647   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3737]    48  3737   156872     5871   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3738]    48  3738   162090     7033   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3739]    48  3739   162431     6902   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3741]    48  3741   159701     5018   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3756]    48  3756   156872     5432   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3760]    48  3760   158944     6759   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3761]    48  3761   156872     5174   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3762]    48  3762   159343     5355   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3763]    48  3763   156872     5641   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3764]    48  3764   158943     7047   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3765]    48  3765   158988     5604   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3766]    48  3766   159210     5151   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3772]    48  3772   161977     6504   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3774]    48  3774   161913     6689   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3775]    48  3775   158942     6796   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3781]    48  3781   159700     4590   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3783]    48  3783   158942     6593   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3786]    48  3786   158942     6609   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3787]    48  3787   160229     5763   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3791]    48  3791   161048     6626   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3801]    48  3801   158942     7363   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3808]    48  3808   160331     6714   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3809]    48  3809   158942     7201   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3810]    48  3810   158714     6336   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3814]    48  3814   161097     6909   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3816]    48  3816   159210     6195   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3817]    48  3817   160334     6952   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3819]    48  3819   156872     4562   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3820]    48  3820   157810     7684   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3821]    48  3821   159810    10782   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3822]    48  3822   157113     8153   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3823]    48  3823   154739     5620   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3824]    48  3824   161743     9838   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3833]    48  3833   161053     8483   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3835]    48  3835   158609     7981   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3838]    48  3838   159913     9734   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3845]    48  3845   155314     6458   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3868]    48  3868   157924     8779   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3869]    48  3869   129946     5494   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3870]    48  3870   157924     8857   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3878]    48  3878   125368      675   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3881]    48  3881   129927     5515   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3882]    48  3882   129598     5252   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3886]    48  3886   129737     5401   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3889]    48  3889   128251     3873   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3893]    48  3893   129471     5224   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3895]    48  3895   127229     2821   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3897]    48  3897   126695     2337   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3898]    48  3898   126695     2344   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3900]    48  3900   126695     2347   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3904]    48  3904   125335      620   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: [ 3909]    48  3909   125300      552   0       0             0 httpd
Dec  6 10:40:39 webserver kernel: Out of memory: Kill process 1673 (mysqld) score 155 or sacrifice child
Dec  6 10:40:39 webserver kernel: Killed process 1673, UID 27, (mysqld) total-vm:5331840kB, anon-rss:530484kB, file-rss:312kB
Dec  6 11:53:23 webserver init: tty (/dev/tty2) main process (1830) killed by TERM signal
Dec  6 11:53:23 webserver init: tty (/dev/tty3) main process (1832) killed by TERM signal
Dec  6 11:53:23 webserver init: tty (/dev/tty4) main process (1834) killed by TERM signal
Dec  6 11:53:23 webserver init: tty (/dev/tty5) main process (1836) killed by TERM signal
Dec  6 11:53:23 webserver init: tty (/dev/tty6) main process (1838) killed by TERM signal
Dec  6 11:53:54 webserver auditd[1271]: The audit daemon is exiting.
Dec  6 11:53:54 webserver kernel: type=1305 audit(1417863234.153:5343): audit_pid=0 old=1271 auid=4294967295 ses=4294967295 res=1
Dec  6 11:53:54 webserver kernel: type=1305 audit(1417863234.301:5344): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1
Dec  6 11:53:54 webserver kernel: Kernel logging (proc) stopped.
Dec  6 11:53:54 webserver rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1291" x-info="http://www.rsyslog.com"] exiting on signal 15.
Dec  6 11:54:26 webserver kernel: imklog 5.8.10, log source = /proc/kmsg started.
Dec  6 11:54:26 webserver rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1289" x-info="http://www.rsyslog.com"] start
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys cpuset
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys cpu
Dec  6 11:54:26 webserver kernel: Linux version 2.6.32-504.1.3.el6.x86_64 (mockbuild@c6b9.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC) ) #1 SMP Tue Nov 11 17:57:25 UTC 2014
Dec  6 11:54:26 webserver kernel: Command line: ro root=/dev/mapper/vg_newhq11vmsecure-lv_root rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_newhq11vmsecure/lv_swap rd_NO_MD rd_LVM_LV=vg_newhq11vmsecure/lv_root crashkernel=auto SYSFONT=latarcyrheb-sun16 rd_NO_DM rhgb quiet
Dec  6 11:54:26 webserver kernel: KERNEL supported cpus:
Dec  6 11:54:26 webserver kernel:  Intel GenuineIntel
Dec  6 11:54:26 webserver kernel:  AMD AuthenticAMD
Dec  6 11:54:26 webserver kernel:  Centaur CentaurHauls
Dec  6 11:54:26 webserver kernel: BIOS-provided physical RAM map:
Dec  6 11:54:26 webserver kernel: BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 00000000000ca000 - 00000000000cc000 (reserved)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 00000000000dc000 - 0000000000100000 (reserved)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 0000000000100000 - 00000000bfef0000 (usable)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 00000000bfef0000 - 00000000bfeff000 (ACPI data)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 00000000bff00000 - 00000000c0000000 (usable)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 00000000e8000000 - 00000000f0000000 (reserved)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 00000000fffe0000 - 0000000100000000 (reserved)
Dec  6 11:54:26 webserver kernel: BIOS-e820: 0000000100000000 - 00000001c0000000 (usable)
Dec  6 11:54:26 webserver kernel: DMI present.
Dec  6 11:54:26 webserver kernel: SMBIOS version 2.4 @ 0xF6B30
Dec  6 11:54:26 webserver kernel: Phoenix BIOS detected: BIOS may corrupt low RAM, working around it.
Dec  6 11:54:26 webserver kernel: Hypervisor detected: VMware
Dec  6 11:54:26 webserver kernel: last_pfn = 0x1c0000 max_arch_pfn = 0x400000000
Dec  6 11:54:26 webserver kernel: x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106
Dec  6 11:54:26 webserver kernel: total RAM covered: 6144M
Dec  6 11:54:26 webserver kernel: Found optimal setting for mtrr clean up
Dec  6 11:54:26 webserver kernel: gran_size: 64K 	chunk_size: 64K 	num_reg: 4  	lose cover RAM: 0G
Dec  6 11:54:26 webserver kernel: x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106
Dec  6 11:54:26 webserver kernel: last_pfn = 0xc0000 max_arch_pfn = 0x400000000
Dec  6 11:54:26 webserver kernel: init_memory_mapping: 0000000000000000-00000000c0000000
Dec  6 11:54:26 webserver kernel: init_memory_mapping: 0000000100000000-00000001c0000000
Dec  6 11:54:26 webserver kernel: RAMDISK: 37087000 - 37fefe4e
Dec  6 11:54:26 webserver kernel: ACPI: RSDP 00000000000f6b80 00024 (v02 PTLTD )
Dec  6 11:54:26 webserver kernel: ACPI: XSDT 00000000bfef121f 00054 (v01 INTEL  440BX    06040000 VMW  01324272)
Dec  6 11:54:26 webserver kernel: ACPI: FACP 00000000bfefee73 000F4 (v04 INTEL  440BX    06040000 PTL  000F4240)
Dec  6 11:54:26 webserver kernel: ACPI: DSDT 00000000bfef146b 0DA08 (v01 PTLTD  Custom   06040000 MSFT 03000001)
Dec  6 11:54:26 webserver kernel: ACPI: FACS 00000000bfefffc0 00040
Dec  6 11:54:26 webserver kernel: ACPI: BOOT 00000000bfef1443 00028 (v01 PTLTD  $SBFTBL$ 06040000  LTP 00000001)
Dec  6 11:54:26 webserver kernel: ACPI: APIC 00000000bfef13f3 00050 (v01 PTLTD  ? APIC   06040000  LTP 00000000)
Dec  6 11:54:26 webserver kernel: ACPI: MCFG 00000000bfef13b7 0003C (v01 PTLTD  $PCITBL$ 06040000  LTP 00000001)
Dec  6 11:54:26 webserver kernel: ACPI: SRAT 00000000bfef12d7 000E0 (v02 VMWARE MEMPLUG  06040000 VMW  00000001)
Dec  6 11:54:26 webserver kernel: ACPI: WAET 00000000bfef12af 00028 (v01 VMWARE VMW WAET 06040000 VMW  00000001)
Dec  6 11:54:26 webserver kernel: Setting APIC routing to flat.
Dec  6 11:54:26 webserver kernel: SRAT: PXM 0 -> APIC 0 -> Node 0
Dec  6 11:54:26 webserver kernel: SRAT: Node 0 PXM 0 0-a0000
Dec  6 11:54:26 webserver kernel: SRAT: Node 0 PXM 0 100000-10000000
Dec  6 11:54:26 webserver kernel: SRAT: Node 0 PXM 0 10000000-c0000000
Dec  6 11:54:26 webserver kernel: SRAT: Node 0 PXM 0 100000000-1c0000000
Dec  6 11:54:26 webserver kernel: Bootmem setup node 0 0000000000000000-00000001c0000000
Dec  6 11:54:26 webserver kernel:  NODE_DATA [0000000000018840 - 000000000004c83f]
Dec  6 11:54:26 webserver kernel:  bootmap [000000000004d000 -  0000000000084fff] pages 38
Dec  6 11:54:26 webserver kernel: (9 early reservations) ==> bootmem [0000000000 - 01c0000000]
Dec  6 11:54:26 webserver kernel:  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
Dec  6 11:54:26 webserver kernel:  #1 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006000 - 0000008000]
Dec  6 11:54:26 webserver kernel:  #2 [0001000000 - 000202abe4]    TEXT DATA BSS ==> [0001000000 - 000202abe4]
Dec  6 11:54:26 webserver kernel:  #3 [0037087000 - 0037fefe4e]          RAMDISK ==> [0037087000 - 0037fefe4e]
Dec  6 11:54:26 webserver kernel:  #4 [000009f800 - 0000100000]    BIOS reserved ==> [000009f800 - 0000100000]
Dec  6 11:54:26 webserver kernel:  #5 [000202b000 - 000202bf9d]              BRK ==> [000202b000 - 000202bf9d]
Dec  6 11:54:26 webserver kernel:  #6 [0000010000 - 0000012000]          PGTABLE ==> [0000010000 - 0000012000]
Dec  6 11:54:26 webserver kernel:  #7 [0000012000 - 0000015000]          PGTABLE ==> [0000012000 - 0000015000]
Dec  6 11:54:26 webserver kernel:  #8 [0000015000 - 0000018840]       MEMNODEMAP ==> [0000015000 - 0000018840]
Dec  6 11:54:26 webserver kernel: found SMP MP-table at [ffff8800000f6bf0] f6bf0
Dec  6 11:54:26 webserver kernel: Reserving 129MB of memory at 48MB for crashkernel (System RAM: 7168MB)
Dec  6 11:54:26 webserver kernel: Zone PFN ranges:
Dec  6 11:54:26 webserver kernel:  DMA      0x00000010 -> 0x00001000
Dec  6 11:54:26 webserver kernel:  DMA32    0x00001000 -> 0x00100000
Dec  6 11:54:26 webserver kernel:  Normal   0x00100000 -> 0x001c0000
Dec  6 11:54:26 webserver kernel: Movable zone start PFN for each node
Dec  6 11:54:26 webserver kernel: early_node_map[4] active PFN ranges
Dec  6 11:54:26 webserver kernel:    0: 0x00000010 -> 0x0000009f
Dec  6 11:54:26 webserver kernel:    0: 0x00000100 -> 0x000bfef0
Dec  6 11:54:26 webserver kernel:    0: 0x000bff00 -> 0x000c0000
Dec  6 11:54:26 webserver kernel:    0: 0x00100000 -> 0x001c0000
Dec  6 11:54:26 webserver kernel: ACPI: PM-Timer IO Port: 0x1008
Dec  6 11:54:26 webserver kernel: Setting APIC routing to flat.
Dec  6 11:54:26 webserver kernel: ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Dec  6 11:54:26 webserver kernel: ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
Dec  6 11:54:26 webserver kernel: ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
Dec  6 11:54:26 webserver kernel: IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
Dec  6 11:54:26 webserver kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
Dec  6 11:54:26 webserver kernel: Using ACPI (MADT) for SMP configuration information
Dec  6 11:54:26 webserver kernel: SMP: Allowing 1 CPUs, 0 hotplug CPUs
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000000a0000 - 00000000000ca000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000000ca000 - 00000000000cc000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000000cc000 - 00000000000dc000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000000dc000 - 0000000000100000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000bfef0000 - 00000000bfeff000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000bfeff000 - 00000000bff00000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000c0000000 - 00000000e8000000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000e8000000 - 00000000f0000000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000f0000000 - 00000000fec00000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000fec00000 - 00000000fec10000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000fec10000 - 00000000fee00000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000fee01000 - 00000000fffe0000
Dec  6 11:54:26 webserver kernel: PM: Registered nosave memory: 00000000fffe0000 - 0000000100000000
Dec  6 11:54:26 webserver kernel: Allocating PCI resources starting at c0000000 (gap: c0000000:28000000)
Dec  6 11:54:26 webserver kernel: Booting paravirtualized kernel on bare hardware
Dec  6 11:54:26 webserver kernel: NR_CPUS:4096 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
Dec  6 11:54:26 webserver kernel: PERCPU: Embedded 31 pages/cpu @ffff880028200000 s95064 r8192 d23720 u2097152
Dec  6 11:54:26 webserver kernel: pcpu-alloc: s95064 r8192 d23720 u2097152 alloc=1*2097152
Dec  6 11:54:26 webserver kernel: pcpu-alloc: [0] 0 
Dec  6 11:54:26 webserver kernel: Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1547540
Dec  6 11:54:26 webserver kernel: Policy zone: Normal
Dec  6 11:54:26 webserver kernel: Kernel command line: ro root=/dev/mapper/vg_newhq11vmsecure-lv_root rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_newhq11vmsecure/lv_swap rd_NO_MD rd_LVM_LV=vg_newhq11vmsecure/lv_root crashkernel=129M@0M SYSFONT=latarcyrheb-sun16 rd_NO_DM rhgb quiet
Dec  6 11:54:26 webserver kernel: PID hash table entries: 4096 (order: 3, 32768 bytes)
Dec  6 11:54:26 webserver kernel: Checking aperture...
Dec  6 11:54:26 webserver kernel: No AGP bridge found
Dec  6 11:54:26 webserver kernel: AMD-Vi disabled by default: pass amd_iommu=on to enable
Dec  6 11:54:26 webserver kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Dec  6 11:54:26 webserver kernel: Placing 64MB software IO TLB between ffff880020000000 - ffff880024000000
Dec  6 11:54:26 webserver kernel: software IO TLB at phys 0x20000000 - 0x24000000
Dec  6 11:54:26 webserver kernel: Memory: 5973792k/7340032k available (5336k kernel code, 1049092k absent, 317148k reserved, 7016k data, 1292k init)
Dec  6 11:54:26 webserver kernel: Hierarchical RCU implementation.
Dec  6 11:54:26 webserver kernel: NR_IRQS:33024 nr_irqs:256
Dec  6 11:54:26 webserver kernel: Extended CMOS year: 2000
Dec  6 11:54:26 webserver kernel: Console: colour VGA+ 80x25
Dec  6 11:54:26 webserver kernel: console [tty0] enabled
Dec  6 11:54:26 webserver kernel: allocated 25165824 bytes of page_cgroup
Dec  6 11:54:26 webserver kernel: please try 'cgroup_disable=memory' option if you don't want memory cgroups
Dec  6 11:54:26 webserver kernel: TSC freq read from hypervisor : 2593.500 MHz
Dec  6 11:54:26 webserver kernel: Detected 2593.500 MHz processor.
Dec  6 11:54:26 webserver kernel: Calibrating delay loop (skipped) preset value.. 5187.00 BogoMIPS (lpj=2593500)
Dec  6 11:54:26 webserver kernel: pid_max: default: 32768 minimum: 301
Dec  6 11:54:26 webserver kernel: Security Framework initialized
Dec  6 11:54:26 webserver kernel: SELinux:  Initializing.
Dec  6 11:54:26 webserver kernel: Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Dec  6 11:54:26 webserver kernel: Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
Dec  6 11:54:26 webserver kernel: Mount-cache hash table entries: 256
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys ns
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys cpuacct
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys memory
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys devices
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys freezer
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys net_cls
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys blkio
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys perf_event
Dec  6 11:54:26 webserver kernel: Initializing cgroup subsys net_prio
Dec  6 11:54:26 webserver kernel: mce: CPU supports 0 MCE banks
Dec  6 11:54:26 webserver kernel: alternatives: switching to unfair spinlock
Dec  6 11:54:26 webserver kernel: SMP alternatives: switching to UP code
Dec  6 11:54:26 webserver kernel: Freeing SMP alternatives: 36k freed
Dec  6 11:54:26 webserver kernel: ACPI: Core revision 20090903
Dec  6 11:54:26 webserver kernel: ftrace: converting mcount calls to 0f 1f 44 00 00
Dec  6 11:54:26 webserver kernel: ftrace: allocating 21917 entries in 86 pages
Dec  6 11:54:26 webserver kernel: APIC routing finalized to flat.
Dec  6 11:54:26 webserver kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
Dec  6 11:54:26 webserver kernel: CPU0: AMD Opteron(tm) Processor 6320                  stepping 02
Dec  6 11:54:26 webserver kernel: Performance Events: 
Dec  6 11:54:26 webserver kernel: ------------[ cut here ]------------
Dec  6 11:54:26 webserver kernel: WARNING: at arch/x86/kernel/cpu/perf_event_amd.c:701 amd_pmu_init+0x181/0x1f2() (Not tainted)
Dec  6 11:54:26 webserver kernel: Hardware name: VMware Virtual Platform
Dec  6 11:54:26 webserver kernel: Odd, counter constraints enabled but no core perfctrs detected!
Dec  6 11:54:26 webserver kernel: Modules linked in:
Dec  6 11:54:26 webserver kernel: Pid: 1, comm: swapper Not tainted 2.6.32-504.1.3.el6.x86_64 #1
Dec  6 11:54:26 webserver kernel: Call Trace:
Dec  6 11:54:26 webserver kernel: [<ffffffff81074df7>] ? warn_slowpath_common+0x87/0xc0
Dec  6 11:54:26 webserver kernel: [<ffffffff81c34f3a>] ? init_hw_perf_events+0x0/0x4bb
Dec  6 11:54:26 webserver kernel: [<ffffffff81074ee6>] ? warn_slowpath_fmt+0x46/0x50
Dec  6 11:54:26 webserver kernel: [<ffffffff81c35576>] ? amd_pmu_init+0x181/0x1f2
Dec  6 11:54:26 webserver kernel: [<ffffffff81c34f79>] ? init_hw_perf_events+0x3f/0x4bb
Dec  6 11:54:26 webserver kernel: [<ffffffff81c34f3a>] ? init_hw_perf_events+0x0/0x4bb
Dec  6 11:54:26 webserver kernel: [<ffffffff8100204c>] ? do_one_initcall+0x3c/0x1d0
Dec  6 11:54:26 webserver kernel: [<ffffffff81c2a762>] ? kernel_init+0x119/0x2f7
Dec  6 11:54:26 webserver kernel: [<ffffffff8100c20a>] ? child_rip+0xa/0x20
Dec  6 11:54:26 webserver kernel: [<ffffffff81c2a649>] ? kernel_init+0x0/0x2f7
Dec  6 11:54:26 webserver kernel: [<ffffffff8100c200>] ? child_rip+0x0/0x20
Dec  6 11:54:26 webserver kernel: ---[ end trace a7919e7f17c0a725 ]---
Dec  6 11:54:26 webserver kernel: AMD PMU driver.
Dec  6 11:54:26 webserver kernel: ... version:                0
Dec  6 11:54:26 webserver kernel: ... bit width:              48
Dec  6 11:54:26 webserver kernel: ... generic registers:      4
Dec  6 11:54:26 webserver kernel: ... value mask:             0000ffffffffffff
Dec  6 11:54:26 webserver kernel: ... max period:             00007fffffffffff
Dec  6 11:54:26 webserver kernel: ... fixed-purpose events:   0
Dec  6 11:54:26 webserver kernel: ... event mask:             000000000000000f
Dec  6 11:54:26 webserver kernel: NMI watchdog enabled, takes one hw-pmu counter.
Dec  6 11:54:26 webserver kernel: Brought up 1 CPUs
Dec  6 11:54:26 webserver kernel: Total of 1 processors activated (5187.00 BogoMIPS).
Dec  6 11:54:26 webserver kernel: x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106
Dec  6 11:54:26 webserver kernel: devtmpfs: initialized
Dec  6 11:54:26 webserver kernel: PM: Registering ACPI NVS region at bfeff000 (4096 bytes)
Dec  6 11:54:26 webserver kernel: regulator: core version 0.5
Dec  6 11:54:26 webserver kernel: NET: Registered protocol family 16
Dec  6 11:54:26 webserver kernel: ACPI: bus type pci registered
Dec  6 11:54:26 webserver kernel: PCI: MCFG configuration 0: base e8000000 segment 0 buses 0 - 127
Dec  6 11:54:26 webserver kernel: PCI: MCFG area at e8000000 reserved in E820
Dec  6 11:54:26 webserver kernel: PCI: Using MMCONFIG at e8000000 - efffffff
Dec  6 11:54:26 webserver kernel: PCI: Using configuration type 1 for base access
Dec  6 11:54:26 webserver kernel: bio: create slab <bio-0> at 0
Dec  6 11:54:26 webserver kernel: ACPI: BIOS _OSI(Linux) query ignored
Dec  6 11:54:26 webserver kernel: ACPI: Interpreter enabled
Dec  6 11:54:26 webserver kernel: ACPI: (supports S0 S1 S4 S5)
Dec  6 11:54:26 webserver kernel: ACPI: Using IOAPIC for interrupt routing
Dec  6 11:54:26 webserver kernel: ACPI: No dock devices found.
Dec  6 11:54:26 webserver kernel: PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
Dec  6 11:54:26 webserver kernel: ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7f])
Dec  6 11:54:26 webserver kernel: pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
Dec  6 11:54:26 webserver kernel: pci_root PNP0A03:00: host bridge window [mem 0x000cc000-0x000cffff]
Dec  6 11:54:26 webserver kernel: pci_root PNP0A03:00: host bridge window [mem 0x000d0000-0x000d3fff]
Dec  6 11:54:26 webserver kernel: pci_root PNP0A03:00: host bridge window [mem 0x000d4000-0x000d7fff]
Dec  6 11:54:26 webserver kernel: pci_root PNP0A03:00: host bridge window [mem 0x000d8000-0x000dbfff]
Dec  6 11:54:26 webserver kernel: pci_root PNP0A03:00: host bridge window [mem 0xc0000000-0xfebfffff]
Dec  6 11:54:26 webserver kernel: pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7]
Dec  6 11:54:26 webserver kernel: pci_root PNP0A03:00: host bridge window [io  0x0d00-0xfeff]
Dec  6 11:54:26 webserver kernel: PCI host bridge to bus 0000:00
Dec  6 11:54:26 webserver kernel: pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
Dec  6 11:54:26 webserver kernel: pci_bus 0000:00: root bus resource [mem 0x000cc000-0x000cffff]
Dec  6 11:54:26 webserver kernel: pci_bus 0000:00: root bus resource [mem 0x000d0000-0x000d3fff]
Dec  6 11:54:26 webserver kernel: pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000d7fff]
Dec  6 11:54:26 webserver kernel: pci_bus 0000:00: root bus resource [mem 0x000d8000-0x000dbfff]
Dec  6 11:54:26 webserver kernel: pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff]
Dec  6 11:54:26 webserver kernel: pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
Dec  6 11:54:26 webserver kernel: pci_bus 0000:00: root bus resource [io  0x0d00-0xfeff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:07.3: quirk: [io  0x1000-0x103f] claimed by PIIX4 ACPI
Dec  6 11:54:26 webserver kernel: pci 0000:00:07.3: quirk: [io  0x1040-0x104f] claimed by PIIX4 SMB
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.0: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.0: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.1: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.1: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.2: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.2: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.3: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.3: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.4: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.4: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.5: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.5: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.6: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.6: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.7: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.7: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.0: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.0: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.1: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.1: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.2: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.2: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.3: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.3: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.4: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.4: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.5: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.5: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.6: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.6: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.7: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.7: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.0: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.0: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.1: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.1: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.2: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.2: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.3: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.3: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.4: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.4: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.5: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.5: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.6: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.6: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.7: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.7: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.0: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.0: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.1: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.1: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.2: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.2: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.3: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.3: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.4: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.4: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.5: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.5: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.6: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.6: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.7: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.7: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:01.0: PCI bridge to [bus 01-01]
Dec  6 11:54:26 webserver kernel: pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
Dec  6 11:54:26 webserver kernel: pci 0000:02:00.0: PME# disabled
Dec  6 11:54:26 webserver kernel: pci 0000:00:11.0: PCI bridge to [bus 02-02] (subtractive decode)
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.0: PCI bridge to [bus 03-03]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.1: PCI bridge to [bus 04-04]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.2: PCI bridge to [bus 05-05]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.3: PCI bridge to [bus 06-06]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.4: PCI bridge to [bus 07-07]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.5: PCI bridge to [bus 08-08]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.6: PCI bridge to [bus 09-09]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.7: PCI bridge to [bus 0a-0a]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.0: PCI bridge to [bus 0b-0b]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.1: PCI bridge to [bus 0c-0c]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.2: PCI bridge to [bus 0d-0d]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.3: PCI bridge to [bus 0e-0e]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.4: PCI bridge to [bus 0f-0f]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.5: PCI bridge to [bus 10-10]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.6: PCI bridge to [bus 11-11]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.7: PCI bridge to [bus 12-12]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.0: PCI bridge to [bus 13-13]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.1: PCI bridge to [bus 14-14]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.2: PCI bridge to [bus 15-15]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.3: PCI bridge to [bus 16-16]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.4: PCI bridge to [bus 17-17]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.5: PCI bridge to [bus 18-18]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.6: PCI bridge to [bus 19-19]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.7: PCI bridge to [bus 1a-1a]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.0: PCI bridge to [bus 1b-1b]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.1: PCI bridge to [bus 1c-1c]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.2: PCI bridge to [bus 1d-1d]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.3: PCI bridge to [bus 1e-1e]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.4: PCI bridge to [bus 1f-1f]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.5: PCI bridge to [bus 20-20]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.6: PCI bridge to [bus 21-21]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.7: PCI bridge to [bus 22-22]
Dec  6 11:54:26 webserver kernel: pci0000:00: Requesting ACPI _OSC control (0x1d)
Dec  6 11:54:26 webserver kernel: pci0000:00: ACPI _OSC control (0x15) granted
Dec  6 11:54:26 webserver kernel: ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 *9 10 11 14 15)
Dec  6 11:54:26 webserver kernel: ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 10 *11 14 15)
Dec  6 11:54:26 webserver kernel: ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 *10 11 14 15)
Dec  6 11:54:26 webserver kernel: ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 10 11 14 15) *0, disabled.
Dec  6 11:54:26 webserver kernel: vgaarb: device added: PCI:0000:00:0f.0,decodes=io+mem,owns=io+mem,locks=none
Dec  6 11:54:26 webserver kernel: vgaarb: loaded
Dec  6 11:54:26 webserver kernel: vgaarb: bridge control possible 0000:00:0f.0
Dec  6 11:54:26 webserver kernel: SCSI subsystem initialized
Dec  6 11:54:26 webserver kernel: usbcore: registered new interface driver usbfs
Dec  6 11:54:26 webserver kernel: usbcore: registered new interface driver hub
Dec  6 11:54:26 webserver kernel: usbcore: registered new device driver usb
Dec  6 11:54:26 webserver kernel: PCI: Using ACPI for IRQ routing
Dec  6 11:54:26 webserver kernel: NetLabel: Initializing
Dec  6 11:54:26 webserver kernel: NetLabel:  domain hash size = 128
Dec  6 11:54:26 webserver kernel: NetLabel:  protocols = UNLABELED CIPSOv4
Dec  6 11:54:26 webserver kernel: NetLabel:  unlabeled traffic allowed by default
Dec  6 11:54:26 webserver kernel: Switching to clocksource jiffies
Dec  6 11:54:26 webserver kernel: pnp: PnP ACPI init
Dec  6 11:54:26 webserver kernel: ACPI: bus type pnp registered
Dec  6 11:54:26 webserver kernel: pnp: PnP ACPI: found 13 devices
Dec  6 11:54:26 webserver kernel: ACPI: ACPI bus type pnp unregistered
Dec  6 11:54:26 webserver kernel: system 00:01: [io  0x1000-0x103f] has been reserved
Dec  6 11:54:26 webserver kernel: system 00:01: [io  0x1040-0x104f] has been reserved
Dec  6 11:54:26 webserver kernel: system 00:01: [io  0x0cf0-0x0cf1] has been reserved
Dec  6 11:54:26 webserver kernel: system 00:0c: [io  0xfce0-0xfcff] has been reserved
Dec  6 11:54:26 webserver kernel: system 00:0c: [mem 0xe8000000-0xefffffff] has been reserved
Dec  6 11:54:26 webserver kernel: system 00:0c: [mem 0xfe800000-0xfe9fffff] has been reserved
Dec  6 11:54:26 webserver kernel: Switching to clocksource acpi_pm
Dec  6 11:54:26 webserver kernel: pci 0000:00:0f.0: BAR 6: assigned [mem 0xc0000000-0xc0007fff pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:10.0: BAR 6: assigned [mem 0xc0008000-0xc000bfff pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.3: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.4: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.5: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.6: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.7: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.3: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.4: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.5: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.6: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.7: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.3: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.4: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.5: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.6: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.7: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.2: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.3: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.4: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.5: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.6: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.7: BAR 13: can't assign io (size 0x1000)
Dec  6 11:54:26 webserver kernel: pci 0000:00:01.0: PCI bridge to [bus 01-01]
Dec  6 11:54:26 webserver kernel: pci 0000:02:00.0: BAR 6: assigned [mem 0xe7b00000-0xe7b0ffff pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:11.0: PCI bridge to [bus 02-02]
Dec  6 11:54:26 webserver kernel: pci 0000:00:11.0:   bridge window [io  0x2000-0x3fff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:11.0:   bridge window [mem 0xf7500000-0xf7ffffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:11.0:   bridge window [mem 0xe7b00000-0xe7ffffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.0: PCI bridge to [bus 03-03]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.0:   bridge window [io  0x4000-0x4fff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.0:   bridge window [mem 0xf7400000-0xf74fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.0:   bridge window [mem 0xe7a00000-0xe7afffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.1: PCI bridge to [bus 04-04]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.1:   bridge window [io  0x8000-0x8fff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.1:   bridge window [mem 0xf7000000-0xf70fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.1:   bridge window [mem 0xe7600000-0xe76fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.2: PCI bridge to [bus 05-05]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.2:   bridge window [io  0xc000-0xcfff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.2:   bridge window [mem 0xf6c00000-0xf6cfffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.2:   bridge window [mem 0xe7200000-0xe72fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.3: PCI bridge to [bus 06-06]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.3:   bridge window [mem 0xf6800000-0xf68fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.3:   bridge window [mem 0xe6e00000-0xe6efffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.4: PCI bridge to [bus 07-07]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.4:   bridge window [mem 0xf6400000-0xf64fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.4:   bridge window [mem 0xe6a00000-0xe6afffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.5: PCI bridge to [bus 08-08]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.5:   bridge window [mem 0xf6000000-0xf60fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.5:   bridge window [mem 0xe6600000-0xe66fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.6: PCI bridge to [bus 09-09]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.6:   bridge window [mem 0xf5c00000-0xf5cfffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.6:   bridge window [mem 0xe6200000-0xe62fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.7: PCI bridge to [bus 0a-0a]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.7:   bridge window [mem 0xf5800000-0xf58fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:15.7:   bridge window [mem 0xe5e00000-0xe5efffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.0: PCI bridge to [bus 0b-0b]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.0:   bridge window [io  0x5000-0x5fff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.0:   bridge window [mem 0xf7300000-0xf73fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.0:   bridge window [mem 0xe7900000-0xe79fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.1: PCI bridge to [bus 0c-0c]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.1:   bridge window [io  0x9000-0x9fff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.1:   bridge window [mem 0xf6f00000-0xf6ffffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.1:   bridge window [mem 0xe7500000-0xe75fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.2: PCI bridge to [bus 0d-0d]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.2:   bridge window [io  0xd000-0xdfff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.2:   bridge window [mem 0xf6b00000-0xf6bfffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.2:   bridge window [mem 0xe7100000-0xe71fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.3: PCI bridge to [bus 0e-0e]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.3:   bridge window [mem 0xf6700000-0xf67fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.3:   bridge window [mem 0xe6d00000-0xe6dfffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.4: PCI bridge to [bus 0f-0f]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.4:   bridge window [mem 0xf6300000-0xf63fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.4:   bridge window [mem 0xe6900000-0xe69fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.5: PCI bridge to [bus 10-10]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.5:   bridge window [mem 0xf5f00000-0xf5ffffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.5:   bridge window [mem 0xe6500000-0xe65fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.6: PCI bridge to [bus 11-11]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.6:   bridge window [mem 0xf5b00000-0xf5bfffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.6:   bridge window [mem 0xe6100000-0xe61fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.7: PCI bridge to [bus 12-12]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.7:   bridge window [mem 0xf5700000-0xf57fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:16.7:   bridge window [mem 0xe5d00000-0xe5dfffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.0: PCI bridge to [bus 13-13]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.0:   bridge window [io  0x6000-0x6fff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.0:   bridge window [mem 0xf7200000-0xf72fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.0:   bridge window [mem 0xe7800000-0xe78fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.1: PCI bridge to [bus 14-14]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.1:   bridge window [io  0xa000-0xafff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.1:   bridge window [mem 0xf6e00000-0xf6efffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.1:   bridge window [mem 0xe7400000-0xe74fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.2: PCI bridge to [bus 15-15]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.2:   bridge window [io  0xe000-0xefff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.2:   bridge window [mem 0xf6a00000-0xf6afffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.2:   bridge window [mem 0xe7000000-0xe70fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.3: PCI bridge to [bus 16-16]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.3:   bridge window [mem 0xf6600000-0xf66fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.3:   bridge window [mem 0xe6c00000-0xe6cfffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.4: PCI bridge to [bus 17-17]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.4:   bridge window [mem 0xf6200000-0xf62fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.4:   bridge window [mem 0xe6800000-0xe68fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.5: PCI bridge to [bus 18-18]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.5:   bridge window [mem 0xf5e00000-0xf5efffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.5:   bridge window [mem 0xe6400000-0xe64fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.6: PCI bridge to [bus 19-19]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.6:   bridge window [mem 0xf5a00000-0xf5afffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.6:   bridge window [mem 0xe6000000-0xe60fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.7: PCI bridge to [bus 1a-1a]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.7:   bridge window [mem 0xf5600000-0xf56fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:17.7:   bridge window [mem 0xe5c00000-0xe5cfffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.0: PCI bridge to [bus 1b-1b]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.0:   bridge window [io  0x7000-0x7fff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.0:   bridge window [mem 0xf7100000-0xf71fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.0:   bridge window [mem 0xe7700000-0xe77fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.1: PCI bridge to [bus 1c-1c]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.1:   bridge window [io  0xb000-0xbfff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.1:   bridge window [mem 0xf6d00000-0xf6dfffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.1:   bridge window [mem 0xe7300000-0xe73fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.2: PCI bridge to [bus 1d-1d]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.2:   bridge window [mem 0xf6900000-0xf69fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.2:   bridge window [mem 0xe6f00000-0xe6ffffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.3: PCI bridge to [bus 1e-1e]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.3:   bridge window [mem 0xf6500000-0xf65fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.3:   bridge window [mem 0xe6b00000-0xe6bfffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.4: PCI bridge to [bus 1f-1f]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.4:   bridge window [mem 0xf6100000-0xf61fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.4:   bridge window [mem 0xe6700000-0xe67fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.5: PCI bridge to [bus 20-20]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.5:   bridge window [mem 0xf5d00000-0xf5dfffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.5:   bridge window [mem 0xe6300000-0xe63fffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.6: PCI bridge to [bus 21-21]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.6:   bridge window [mem 0xf5900000-0xf59fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.6:   bridge window [mem 0xe5f00000-0xe5ffffff 64bit pref]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.7: PCI bridge to [bus 22-22]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.7:   bridge window [mem 0xf5500000-0xf55fffff]
Dec  6 11:54:26 webserver kernel: pci 0000:00:18.7:   bridge window [mem 0xe5b00000-0xe5bfffff 64bit pref]
Dec  6 11:54:26 webserver kernel: NET: Registered protocol family 2
Dec  6 11:54:26 webserver kernel: IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
Dec  6 11:54:26 webserver kernel: TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
Dec  6 11:54:26 webserver kernel: TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
Dec  6 11:54:26 webserver kernel: TCP: Hash tables configured (established 524288 bind 65536)
Dec  6 11:54:26 webserver kernel: TCP reno registered
Dec  6 11:54:26 webserver kernel: NET: Registered protocol family 1
Dec  6 11:54:26 webserver kernel: pci 0000:00:00.0: Limiting direct PCI/PCI transfers
Dec  6 11:54:26 webserver kernel: Trying to unpack rootfs image as initramfs...
Dec  6 11:54:26 webserver kernel: Freeing initrd memory: 15779k freed
Dec  6 11:54:26 webserver kernel: Simple Boot Flag at 0x36 set to 0x80
Dec  6 11:54:26 webserver kernel: futex hash table entries: 256 (order: 2, 16384 bytes)
Dec  6 11:54:26 webserver kernel: audit: initializing netlink socket (disabled)
Dec  6 11:54:26 webserver kernel: type=2000 audit(1417863245.633:1): initialized
Dec  6 11:54:26 webserver kernel: HugeTLB registered 2 MB page size, pre-allocated 0 pages
Dec  6 11:54:26 webserver kernel: VFS: Disk quotas dquot_6.5.2
Dec  6 11:54:26 webserver kernel: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Dec  6 11:54:26 webserver kernel: msgmni has been set to 11698
Dec  6 11:54:26 webserver kernel: alg: No test for stdrng (krng)
Dec  6 11:54:26 webserver kernel: ksign: Installing public key data
Dec  6 11:54:26 webserver kernel: Loading keyring
Dec  6 11:54:26 webserver kernel: - Added public key EE81BF431C03EFE7
Dec  6 11:54:26 webserver kernel: - User ID: CentOS (Kernel Module GPG key)
Dec  6 11:54:26 webserver kernel: Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
Dec  6 11:54:26 webserver kernel: io scheduler noop registered
Dec  6 11:54:26 webserver kernel: io scheduler anticipatory registered
Dec  6 11:54:26 webserver kernel: io scheduler deadline registered
Dec  6 11:54:26 webserver kernel: io scheduler cfq registered (default)
Dec  6 11:54:26 webserver kernel: pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:15.0:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:15.1:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:15.2:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:15.3:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:15.4:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:15.5:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:15.6:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:15.7:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:16.0:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:16.1:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:16.2:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:16.3:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:16.4:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:16.5:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:16.6:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:16.7:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:17.0:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:17.1:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:17.2:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:17.3:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:17.4:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:17.5:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:17.6:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:17.7:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:18.0:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:18.1:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:18.2:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:18.3:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:18.4:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:18.5:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:18.6:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp 0000:00:18.7:pcie04: HPC vendor_id 15ad device_id 7a0 ss_vid 0 ss_did 0
Dec  6 11:54:26 webserver kernel: pciehp: PCI Express Hot Plug Controller Driver version: 0.4
Dec  6 11:54:26 webserver kernel: acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [32] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [33] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [34] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [35] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [36] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [37] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [38] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [39] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [40] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [41] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [42] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [43] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [44] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [45] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [46] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [47] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [48] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [49] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [50] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [51] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [52] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [53] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [54] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [55] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [56] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [57] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [58] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [59] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [60] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [61] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [62] registered
Dec  6 11:54:26 webserver kernel: acpiphp: Slot [63] registered
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 160 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 192 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 224 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 256 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 161 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 162 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 163 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 164 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 165 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 166 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 167 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 193 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 194 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 195 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 196 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 197 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 198 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 199 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 225 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 226 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 227 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 228 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 229 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 230 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 231 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 257 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 258 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 259 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 260 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 261 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 262 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: acpiphp_glue: Slot 263 already registered by another hotplug driver
Dec  6 11:54:26 webserver kernel: ACPI: AC Adapter [ACAD] (on-line)
Dec  6 11:54:26 webserver kernel: input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
Dec  6 11:54:26 webserver kernel: ACPI: Power Button [PWRF]
Dec  6 11:54:26 webserver kernel: [Firmware Bug]: No valid trip found
Dec  6 11:54:26 webserver kernel: GHES: HEST is not enabled!
Dec  6 11:54:26 webserver kernel: Non-volatile memory driver v1.3
Dec  6 11:54:26 webserver kernel: Linux agpgart interface v0.103
Dec  6 11:54:26 webserver kernel: agpgart-intel 0000:00:00.0: Intel 440BX Chipset
Dec  6 11:54:26 webserver kernel: agpgart-intel 0000:00:00.0: AGP aperture is 256M @ 0x0
Dec  6 11:54:26 webserver kernel: crash memory driver: version 1.1
Dec  6 11:54:26 webserver kernel: Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
Dec  6 11:54:26 webserver kernel: serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
Dec  6 11:54:26 webserver kernel: serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
Dec  6 11:54:26 webserver kernel: 00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
Dec  6 11:54:26 webserver kernel: 00:0a: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
Dec  6 11:54:26 webserver kernel: brd: module loaded
Dec  6 11:54:26 webserver kernel: loop: module loaded
Dec  6 11:54:26 webserver kernel: input: Macintosh mouse button emulation as /devices/virtual/input/input1
Dec  6 11:54:26 webserver kernel: Fixed MDIO Bus: probed
Dec  6 11:54:26 webserver kernel: ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
Dec  6 11:54:26 webserver kernel: ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
Dec  6 11:54:26 webserver kernel: uhci_hcd: USB Universal Host Controller Interface driver
Dec  6 11:54:26 webserver kernel: PNP: PS/2 Controller [PNP0303:KBC,PNP0f13:MOUS] at 0x60,0x64 irq 1,12
Dec  6 11:54:26 webserver kernel: serio: i8042 KBD port at 0x60,0x64 irq 1
Dec  6 11:54:26 webserver kernel: serio: i8042 AUX port at 0x60,0x64 irq 12
Dec  6 11:54:26 webserver kernel: mice: PS/2 mouse device common for all mice
Dec  6 11:54:26 webserver kernel: rtc_cmos 00:04: rtc core: registered rtc_cmos as rtc0
Dec  6 11:54:26 webserver kernel: rtc0: alarms up to one month, y3k, 114 bytes nvram
Dec  6 11:54:26 webserver kernel: cpuidle: using governor ladder
Dec  6 11:54:26 webserver kernel: cpuidle: using governor menu
Dec  6 11:54:26 webserver kernel: EFI Variables Facility v0.08 2004-May-17
Dec  6 11:54:26 webserver kernel: input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
Dec  6 11:54:26 webserver kernel: usbcore: registered new interface driver hiddev
Dec  6 11:54:26 webserver kernel: usbcore: registered new interface driver usbhid
Dec  6 11:54:26 webserver kernel: usbhid: v2.6:USB HID core driver
Dec  6 11:54:26 webserver kernel: GRE over IPv4 demultiplexor driver
Dec  6 11:54:26 webserver kernel: TCP cubic registered
Dec  6 11:54:26 webserver kernel: Initializing XFRM netlink socket
Dec  6 11:54:26 webserver kernel: NET: Registered protocol family 17
Dec  6 11:54:26 webserver kernel: registered taskstats version 1
Dec  6 11:54:26 webserver kernel: input: ImPS/2 Generic Wheel Mouse as /devices/platform/i8042/serio1/input/input3
Dec  6 11:54:26 webserver kernel: rtc_cmos 00:04: setting system clock to 2014-12-06 10:54:07 UTC (1417863247)
Dec  6 11:54:26 webserver kernel: Initalizing network drop monitor service
Dec  6 11:54:26 webserver kernel: Freeing unused kernel memory: 1292k freed
Dec  6 11:54:26 webserver kernel: Write protecting the kernel read-only data: 10240k
Dec  6 11:54:26 webserver kernel: Freeing unused kernel memory: 788k freed
Dec  6 11:54:26 webserver kernel: Freeing unused kernel memory: 1568k freed
Dec  6 11:54:26 webserver kernel: dracut: dracut-004-356.el6
Dec  6 11:54:26 webserver kernel: dracut: rd_NO_LUKS: removing cryptoluks activation
Dec  6 11:54:26 webserver kernel: device-mapper: uevent: version 1.0.3
Dec  6 11:54:26 webserver kernel: device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel@redhat.com
Dec  6 11:54:26 webserver kernel: udev: starting version 147
Dec  6 11:54:26 webserver kernel: dracut: Starting plymouth daemon
Dec  6 11:54:26 webserver kernel: Refined TSC clocksource calibration: 2593.500 MHz.
Dec  6 11:54:26 webserver kernel: Switching to clocksource tsc
Dec  6 11:54:26 webserver kernel: dracut: rd_NO_MD: removing MD RAID activation
Dec  6 11:54:26 webserver kernel: scsi0 : ata_piix
Dec  6 11:54:26 webserver kernel: scsi1 : ata_piix
Dec  6 11:54:26 webserver kernel: ata1: PATA max UDMA/33 cmd 0x1f0 ctl 0x3f6 bmdma 0x1060 irq 14
Dec  6 11:54:26 webserver kernel: ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1068 irq 15
Dec  6 11:54:26 webserver kernel: ata2.00: ATAPI: VMware Virtual IDE CDROM Drive, 00000001, max UDMA/33
Dec  6 11:54:26 webserver kernel: ata2.00: configured for UDMA/33
Dec  6 11:54:26 webserver kernel: scsi 1:0:0:0: CD-ROM            NECVMWar VMware IDE CDR10 1.00 PQ: 0 ANSI: 5
Dec  6 11:54:26 webserver kernel: Fusion MPT base driver 3.04.20
Dec  6 11:54:26 webserver kernel: Copyright (c) 1999-2008 LSI Corporation
Dec  6 11:54:26 webserver kernel: Fusion MPT SPI Host driver 3.04.20
Dec  6 11:54:26 webserver kernel: mptspi 0000:00:10.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
Dec  6 11:54:26 webserver kernel: mptbase: ioc0: Initiating bringup
Dec  6 11:54:26 webserver kernel: ioc0: LSI53C1030 B0: Capabilities={Initiator}
Dec  6 11:54:26 webserver kernel: scsi2 : ioc0: LSI53C1030 B0, FwRev=01032920h, Ports=1, MaxQ=128, IRQ=17
Dec  6 11:54:26 webserver kernel: scsi 2:0:0:0: Direct-Access     VMware   Virtual disk     1.0  PQ: 0 ANSI: 2
Dec  6 11:54:26 webserver kernel: scsi target2:0:0: Beginning Domain Validation
Dec  6 11:54:26 webserver kernel: scsi target2:0:0: Domain Validation skipping write tests
Dec  6 11:54:26 webserver kernel: scsi target2:0:0: Ending Domain Validation
Dec  6 11:54:26 webserver kernel: scsi target2:0:0: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)
Dec  6 11:54:26 webserver kernel: sr0: scsi3-mmc drive: 1x/1x writer dvd-ram cd/rw xa/form2 cdda tray
Dec  6 11:54:26 webserver kernel: Uniform CD-ROM driver Revision: 3.20
Dec  6 11:54:26 webserver kernel: STARTING CRC_T10DIF
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: [sda] 104857600 512-byte logical blocks: (53.6 GB/50.0 GiB)
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: [sda] Write Protect is off
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: [sda] Cache data unavailable
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: [sda] Assuming drive cache: write through
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: [sda] Cache data unavailable
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: [sda] Assuming drive cache: write through
Dec  6 11:54:26 webserver kernel: sda: sda1 sda2
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: [sda] Cache data unavailable
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: [sda] Assuming drive cache: write through
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: [sda] Attached SCSI disk
Dec  6 11:54:26 webserver kernel: dracut: Scanning devices sda2  for LVM logical volumes vg_newhq11vmsecure/lv_swap vg_newhq11vmsecure/lv_root 
Dec  6 11:54:26 webserver kernel: dracut: inactive '/dev/vg_newhq11vmsecure/lv_root' [43.66 GiB] inherit
Dec  6 11:54:26 webserver kernel: dracut: inactive '/dev/vg_newhq11vmsecure/lv_swap' [5.84 GiB] inherit
Dec  6 11:54:26 webserver kernel: EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: 
Dec  6 11:54:26 webserver kernel: dracut: Mounted root filesystem /dev/mapper/vg_newhq11vmsecure-lv_root
Dec  6 11:54:26 webserver kernel: SELinux:  Disabled at runtime.
Dec  6 11:54:26 webserver kernel: type=1404 audit(1417863249.263:2): selinux=0 auid=4294967295 ses=4294967295
Dec  6 11:54:26 webserver kernel: dracut: 
Dec  6 11:54:26 webserver kernel: dracut: Switching root
Dec  6 11:54:26 webserver kernel: udev: starting version 147
Dec  6 11:54:26 webserver kernel: shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
Dec  6 11:54:26 webserver kernel: sr 1:0:0:0: Attached scsi generic sg0 type 5
Dec  6 11:54:26 webserver kernel: sd 2:0:0:0: Attached scsi generic sg1 type 0
Dec  6 11:54:26 webserver kernel: piix4_smbus 0000:00:07.3: Host SMBus controller not enabled!
Dec  6 11:54:26 webserver kernel: e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
Dec  6 11:54:26 webserver kernel: e1000: Copyright (c) 1999-2006 Intel Corporation.
Dec  6 11:54:26 webserver kernel: e1000 0000:02:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
Dec  6 11:54:26 webserver kernel: e1000 0000:02:00.0: eth0: (PCI:66MHz:32-bit) 00:50:56:a4:b3:88
Dec  6 11:54:26 webserver kernel: e1000 0000:02:00.0: eth0: Intel(R) PRO/1000 Network Connection
Dec  6 11:54:26 webserver kernel: parport_pc 00:08: reported by Plug and Play ACPI
Dec  6 11:54:26 webserver kernel: parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
Dec  6 11:54:26 webserver kernel: ppdev: user-space parallel port driver
Dec  6 11:54:26 webserver kernel: EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: 
Dec  6 11:54:26 webserver kernel: Adding 6127612k swap on /dev/mapper/vg_newhq11vmsecure-lv_swap.  Priority:-1 extents:1 across:6127612k 
Dec  6 11:54:26 webserver kernel: NET: Registered protocol family 10
Dec  6 11:54:26 webserver kernel: lo: Disabled Privacy Extensions
Dec  6 11:54:26 webserver kernel: ip6_tables: (C) 2000-2006 Netfilter Core Team
Dec  6 11:54:26 webserver kernel: nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
Dec  6 11:54:26 webserver kernel: e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Dec  6 11:54:26 webserver kernel: type=1305 audit(1417863265.904:3): audit_pid=1269 old=0 auid=4294967295 ses=4294967295 res=1

Open in new window

SOLUTION
Avatar of Guy Hengel [angelIII / a3]
Guy Hengel [angelIII / a3]
Flag of Luxembourg image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Carletto

ASKER

Thank you ....
this is the MySQL log file ... I'm not able to find the issue .....
does it says something where I should fix ?
Thank you




141203  8:47:49 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.41'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi
141206 10:40:40 mysqld_safe Number of processes running now: 0
141206 10:40:40 mysqld_safe mysqld restarted
141206 10:40:45 [Note] Plugin 'FEDERATED' is disabled.
141206 10:40:47 InnoDB: The InnoDB memory heap is disabled
141206 10:40:47 InnoDB: Mutexes and rw_locks use GCC atomic builtins
141206 10:40:47 InnoDB: Compressed tables use zlib 1.2.3
141206 10:40:47 InnoDB: Using Linux native AIO
141206 10:41:13 InnoDB: Initializing buffer pool, size = 4.0G
InnoDB: mmap(4395630592 bytes) failed; errno 12
141206 10:41:13 InnoDB: Completed initialization of buffer pool
141206 10:41:13 InnoDB: Fatal error: cannot allocate memory for the buffer pool
141206 10:41:13 [ERROR] Plugin 'InnoDB' init function returned error.
141206 10:41:13 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
141206 10:41:15 [ERROR] Unknown/unsupported storage engine: InnoDB
141206 10:41:15 [ERROR] Aborting

141206 10:41:16 [Note] /usr/libexec/mysqld: Shutdown complete

141206 10:41:17 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
141206 11:50:06 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
141206 11:50:06 [Note] Plugin 'FEDERATED' is disabled.
141206 11:50:06 InnoDB: The InnoDB memory heap is disabled
141206 11:50:06 InnoDB: Mutexes and rw_locks use GCC atomic builtins
141206 11:50:06 InnoDB: Compressed tables use zlib 1.2.3
141206 11:50:06 InnoDB: Using Linux native AIO
141206 11:50:06 InnoDB: Initializing buffer pool, size = 4.0G
141206 11:50:07 InnoDB: Completed initialization of buffer pool
141206 11:50:07 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 2216254135953
141206 11:50:07  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 2216257355147
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 1 row operations to undo
InnoDB: Trx id counter is A424A700
141206 11:50:08  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
InnoDB: Starting in background the rollback of uncommitted transactions
141206 11:50:09  InnoDB: Rolling back trx with id A424A52C, 1 rows to undo
141206 11:50:09  InnoDB: Waiting for the background threads to start

InnoDB: Rolling back of trx id A424A52C completed
141206 11:50:09  InnoDB: Rollback of non-prepared transactions completed
141206 11:50:10 InnoDB: 5.5.41 started; log sequence number 2216257355147
141206 11:50:10 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
141206 11:50:10 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
141206 11:50:10 [Note] Server socket created on IP: '0.0.0.0'.
141206 11:50:10 [Note] Event Scheduler: Loaded 0 events
141206 11:50:10 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.41'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi
141206 11:53:48 [Note] /usr/libexec/mysqld: Normal shutdown

141206 11:53:48 [Note] Event Scheduler: Purging the queue. 0 events
141206 11:53:48  InnoDB: Starting shutdown...
141206 11:53:48  InnoDB: Waiting for 1 pages to be flushed
141206 11:53:52  InnoDB: Shutdown completed; log sequence number 2216275683780
141206 11:53:52 [Note] /usr/libexec/mysqld: Shutdown complete

141206 11:53:52 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
141206 11:54:26 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
141206 11:54:27 [Note] Plugin 'FEDERATED' is disabled.
141206 11:54:27 InnoDB: The InnoDB memory heap is disabled
141206 11:54:27 InnoDB: Mutexes and rw_locks use GCC atomic builtins
141206 11:54:27 InnoDB: Compressed tables use zlib 1.2.3
141206 11:54:27 InnoDB: Using Linux native AIO
141206 11:54:27 InnoDB: Initializing buffer pool, size = 4.0G
141206 11:54:27 InnoDB: Completed initialization of buffer pool
141206 11:54:27 InnoDB: highest supported file format is Barracuda.
141206 11:54:28  InnoDB: Waiting for the background threads to start
141206 11:54:29 InnoDB: 5.5.41 started; log sequence number 2216275683780
141206 11:54:29 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
141206 11:54:29 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
141206 11:54:29 [Note] Server socket created on IP: '0.0.0.0'.
141206 11:54:29 [Note] Event Scheduler: Loaded 0 events
141206 11:54:29 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.41'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi

Open in new window

>InnoDB: mmap(4395630592 bytes) failed; errno 12
>141206 10:41:13 InnoDB: Fatal error: cannot allocate memory for the buffer pool
check the InnoDB configuration; the error indicates it failes (not enough memory on the system )


other than that, I see nothing abnormal, just the usual startup/shutdown information
Thank you

I'm looking my research on this event from the messages.log

Dec  6 10:40:39 webserver kernel: httpd invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0

Open in new window


apparently several people having centos and a web server experienced this problem.
Carlettus
May be worth looking at setting up a page file if you haven't already.
Hello David,

 swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       6127612 240676  -1


is this correct ?
Avatar of Kerem ERSOY
Kerem ERSOY

Hi,

It seems that there are two problems with your setup:
1) as Angelll pointed out earlier you need to check inno_db memory usage and for the test make sure that it is  50-60% of the actual RAM. You need to check /etc/my.cnf  the variable
innodb_buffer_pool_size

Open in new window

.

2) It seems that your HTTPD processes are taking up much memory and there are so many instances. You might need to adjust your http.conf and make sure that staring number of instances are similar to this (for beginning):

StartServers 1
MinSpareServers 1
MaxSpareServers 5
MaxClients 5
.
.
MaxRequestsPerChild 200

Open in new window


Try to restart. Your swap indicates it is almost full and it seems that something (probably your HTTPD daemon) is eating up all available memory.


Cheers,
K.
Hello,
thank you for your message .
this is my configuration , I will correct the MySQL value to 3GB.
 
mysqld
the server has 6gb ram and the parameter is
innodb_buffer_pool_size = 4G

Open in new window


These is the value of httpd.conf ....

httpd.conf
<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000
</IfModule>

<IfModule worker.c>
StartServers         4
MaxClients         300
MinSpareThreads     25
MaxSpareThreads     75
ThreadsPerChild     25
MaxRequestsPerChild  0
</IfModule>

Open in new window

ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Hello,
I've changed the settings based on Kerem message .
Up to now no problems , one day passed .... is it normal that the behavior I have now is the web site slower ?

well , we remain on hold till Monday than I'll increase the values .
Thank you
Hi,

Than you would start to increase HTTP Server settings gradually. It seems that you need to process more requests. My suggestion would be to keep the  MaxRequestsPerChild, MaxClients,StartServers and MaxSparseServers as low as possible.

Keep in mind that the real problem with your setup was HTTPD consuming all your memory. It was only because MySQL quick to kill some child processes  when there's a memory shortage and this is why your logs indicated some child has been terminated.

Cheers,
K.
Hello,
I'm back after some time of inactivity. The server is responding better after the previous changes.
I would like to thank all of you for your support.
Bye
Carlo
You're welcome. I am glad I could help.