rajsolaris
asked on
Clock skew help ?
the skew is growing all the time. Once it hits 100 seconds, my jobs fail. I have to reset the clocks every 10 mins manually. an some expert throw some light on this any help is highly appreciated. This is a Virtual Machine Linux Redhat ES 4.
Thanks In Advance.
Thanks In Advance.
ASKER
There are some jobs running on those servers which needs to match the time with the DB server and the timeserver they should be in sync with each other.
yes. We have 2 solaris global zones which acts as a timeserver for this VM.
I noticed one more thing, I have a cron job set on this box as init 6 every night at 2:00 AM the /var/log/messages shows the correct entry .....
May 7 02:00:45 ap01 init: Switching to runlevel: 6
but the command
11:20:27 root@ap01 # who -b
system boot May 7 03:24
shows some different time why is it so ?
Thanks.
yes. We have 2 solaris global zones which acts as a timeserver for this VM.
I noticed one more thing, I have a cron job set on this box as init 6 every night at 2:00 AM the /var/log/messages shows the correct entry .....
May 7 02:00:45 ap01 init: Switching to runlevel: 6
but the command
11:20:27 root@ap01 # who -b
system boot May 7 03:24
shows some different time why is it so ?
Thanks.
run
date
hwclock
and see if the hardware clock is the same as the os clock
also check this link
http://communities.vmware.com/thread/93541
date
hwclock
and see if the hardware clock is the same as the os clock
also check this link
http://communities.vmware.com/thread/93541
add boot parameter nohz=off
and make sure that you have vmware client reading date from host, and host using NTP to synchronize.
and make sure that you have vmware client reading date from host, and host using NTP to synchronize.
ASKER
These are the contents from my grub.conf where I have added nosmp noapic nolapic these boot parameters. Pl advise accordingly.
#boot=/dev/sda
default=1
timeout=5
splashimage=(hd0,0)/boot/g rub/splash .xpm.gz
hiddenmenu
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.EL)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-78.0.1 3.EL ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
initrd /boot/initrd-2.6.9-78.0.13 .EL.img
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.ELsmp)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-78.0.1 3.ELsmp ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
initrd /boot/initrd-2.6.9-78.0.13 .ELsmp.img
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.EL)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-42.0.8 .EL ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
initrd /boot/initrd-2.6.9-42.0.8. EL.img
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.ELsmp)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-42.0.8 .ELsmp ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
initrd /boot/initrd-2.6.9-42.0.8. ELsmp.img
Thanks a lot for the response.
#boot=/dev/sda
default=1
timeout=5
splashimage=(hd0,0)/boot/g
hiddenmenu
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.EL)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-78.0.1
initrd /boot/initrd-2.6.9-78.0.13
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.ELsmp)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-78.0.1
initrd /boot/initrd-2.6.9-78.0.13
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.EL)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-42.0.8
initrd /boot/initrd-2.6.9-42.0.8.
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.ELsmp)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-42.0.8
initrd /boot/initrd-2.6.9-42.0.8.
Thanks a lot for the response.
Add nohz=off to boot parameters.
What is your virtual machine host? What is exact RHEL release including Update version?
What is your virtual machine host? What is exact RHEL release including Update version?
ASKER
Red Hat Enterprise Linux ES release 4 (Nahant Update 7)
shall I remove the other parameters from the grub.conf ?
shall I remove the other parameters from the grub.conf ?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
... and with nohz enabled this VMWare misfeature makes realtime clock extremely slow ...
(But it helps to save electricity in all other environments)
(But it helps to save electricity in all other environments)
And also - do not use NTPD, uninstall it. VMWare tools does timesyncing part.
ASKER
yes. We are using ESX host OS 3.5.
8 May 09:57:30 ntpdate[9080]: step time server 10.65.192.127 offset 28.027599 sec
8 May 09:58:45 ntpdate[9102]: step time server 10.65.192.127 offset 43.196146 sec
8 May 09:59:17 ntpdate[9121]: step time server 10.65.192.124 offset 15.146884 sec
8 May 10:00:19 ntpdate[9135]: step time server 10.65.192.124 offset 11.712952 sec
8 May 10:01:05 ntpdate[9379]: step time server 10.65.192.124 offset 3.154559 sec
8 May 10:02:02 ntpdate[9433]: step time server 10.65.192.127 offset 0.909640 sec
8 May 10:03:02 ntpdate[9436]: adjust time server 10.65.192.127 offset 0.049909 sec
8 May 10:04:01 ntpdate[9439]: adjust time server 10.65.192.124 offset -0.007343 sec
8 May 10:05:01 ntpdate[9445]: adjust time server 10.65.192.124 offset -0.014649 sec
8 May 10:06:01 ntpdate[9465]: adjust time server 10.65.192.127 offset 0.008264 sec
8 May 10:07:01 ntpdate[9468]: adjust time server 10.65.192.124 offset 0.071408 sec
8 May 10:08:09 ntpdate[9473]: step time server 10.65.192.127 offset 7.034123 sec
This is what the output is after running ntpdate every 1 mint.
And tried adding following boot parameters:
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.EL)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-78.0.1 3.EL ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
initrd /boot/initrd-2.6.9-78.0.13 .EL.img
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.ELsmp)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-78.0.1 3.ELsmp ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
initrd /boot/initrd-2.6.9-78.0.13 .ELsmp.img
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.EL)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-42.0.8 .EL ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
initrd /boot/initrd-2.6.9-42.0.8. EL.img
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.ELsmp)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-42.0.8 .ELsmp ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
initrd /boot/initrd-2.6.9-42.0.8. ELsmp.img
Thanks.
8 May 09:57:30 ntpdate[9080]: step time server 10.65.192.127 offset 28.027599 sec
8 May 09:58:45 ntpdate[9102]: step time server 10.65.192.127 offset 43.196146 sec
8 May 09:59:17 ntpdate[9121]: step time server 10.65.192.124 offset 15.146884 sec
8 May 10:00:19 ntpdate[9135]: step time server 10.65.192.124 offset 11.712952 sec
8 May 10:01:05 ntpdate[9379]: step time server 10.65.192.124 offset 3.154559 sec
8 May 10:02:02 ntpdate[9433]: step time server 10.65.192.127 offset 0.909640 sec
8 May 10:03:02 ntpdate[9436]: adjust time server 10.65.192.127 offset 0.049909 sec
8 May 10:04:01 ntpdate[9439]: adjust time server 10.65.192.124 offset -0.007343 sec
8 May 10:05:01 ntpdate[9445]: adjust time server 10.65.192.124 offset -0.014649 sec
8 May 10:06:01 ntpdate[9465]: adjust time server 10.65.192.127 offset 0.008264 sec
8 May 10:07:01 ntpdate[9468]: adjust time server 10.65.192.124 offset 0.071408 sec
8 May 10:08:09 ntpdate[9473]: step time server 10.65.192.127 offset 7.034123 sec
This is what the output is after running ntpdate every 1 mint.
And tried adding following boot parameters:
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.EL)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-78.0.1
initrd /boot/initrd-2.6.9-78.0.13
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.ELsmp)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-78.0.1
initrd /boot/initrd-2.6.9-78.0.13
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.EL)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-42.0.8
initrd /boot/initrd-2.6.9-42.0.8.
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.ELsmp)
root (hd0,0)
kernel /boot/vmlinuz-2.6.9-42.0.8
initrd /boot/initrd-2.6.9-42.0.8.
Thanks.
My suggestion is to remove all the kernel parameters you have added to fix the clock and add only nohz=off
Your ntpdate result confirms that clock is way too slow, vmware counts time in interrupts, linux not anymore.
Your ESX has security issue http://support.vmware.com/selfsupport/download/ but no timing fixes until latest patch4.
Your ntpdate result confirms that clock is way too slow, vmware counts time in interrupts, linux not anymore.
Your ESX has security issue http://support.vmware.com/selfsupport/download/ but no timing fixes until latest patch4.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
It looks like covering hole in the wall with piece of carpet.
Their timing infrastructure is incomplete. Running fixed speed clock in VM makes it somewhat accurate at least to the degree it does not drift extremely far away. On less loaded systems clock drift will go outside ntpd's limits and it will exit.
Their timing infrastructure is incomplete. Running fixed speed clock in VM makes it somewhat accurate at least to the degree it does not drift extremely far away. On less loaded systems clock drift will go outside ntpd's limits and it will exit.
are you using ntp to set the time ?