?
Solved

Clock skew help ?

Posted on 2009-05-07
15
Medium Priority
?
812 Views
Last Modified: 2013-12-06
the skew is growing all the time. Once it hits 100 seconds, my jobs fail. I have to  reset the clocks every 10 mins manually. an some expert throw some light on this any help is highly appreciated. This is a Virtual Machine Linux Redhat ES 4.

Thanks In Advance.
0
Comment
Question by:rajsolaris
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 4
  • 2
  • +1
15 Comments
 
LVL 5

Expert Comment

by:0ren
ID: 24328749
what do you mean you jobs ?
are you using ntp to set the time ?
0
 

Author Comment

by:rajsolaris
ID: 24328916
There are some jobs running on those servers which needs to match the time with the DB server and the timeserver they should be in sync with each other.

yes. We have 2 solaris global zones which acts as a timeserver for this VM.
I noticed one more thing, I have a cron job set on this box as init 6 every night at 2:00 AM the /var/log/messages shows the correct entry .....
May  7 02:00:45 ap01 init: Switching to runlevel: 6
but the command

11:20:27 root@ap01 # who -b
         system boot  May  7 03:24

shows some different time why is it so ?

Thanks.


0
 
LVL 5

Expert Comment

by:0ren
ID: 24329318
run
date
hwclock
and see if the hardware clock is the same as the os clock
also check this link

http://communities.vmware.com/thread/93541
0
Moving data to the cloud? Find out if you’re ready

Before moving to the cloud, it is important to carefully define your db needs, plan for the migration & understand prod. environment. This wp explains how to define what you need from a cloud provider, plan for the migration & what putting a cloud solution into practice entails.

 
LVL 62

Expert Comment

by:gheist
ID: 24329414
add boot parameter nohz=off
and make sure that you have vmware client reading date from host, and host using NTP to synchronize.
0
 

Author Comment

by:rajsolaris
ID: 24329543
These are the contents from my grub.conf where I have added nosmp noapic nolapic these boot parameters. Pl advise accordingly.
#boot=/dev/sda
default=1
timeout=5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.EL)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.9-78.0.13.EL ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
        initrd /boot/initrd-2.6.9-78.0.13.EL.img
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.ELsmp)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.9-78.0.13.ELsmp ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
        initrd /boot/initrd-2.6.9-78.0.13.ELsmp.img
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.EL)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.9-42.0.8.EL ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
        initrd /boot/initrd-2.6.9-42.0.8.EL.img
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.ELsmp)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.9-42.0.8.ELsmp ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
        initrd /boot/initrd-2.6.9-42.0.8.ELsmp.img

Thanks a lot for the response.
0
 
LVL 62

Expert Comment

by:gheist
ID: 24329564
Add nohz=off to boot parameters.

What is your virtual machine host? What is exact RHEL release including Update version?
0
 

Author Comment

by:rajsolaris
ID: 24329625
Red Hat Enterprise Linux ES release 4 (Nahant Update 7)

shall I remove the other parameters from the grub.conf ?
0
 
LVL 62

Accepted Solution

by:
gheist earned 1000 total points
ID: 24329853
4u7 is latest, no need for software updates.

What is virtualization software you are using?i,e ESX or Server or Infrastructure? Server is worst by means of timekeeping.

Anyway - click patches for your product - http://www.vmware.com/download/
First patch is latest, no improvements in timekeeping, but you may gain something from Paravirtualzation.

Other parameters do not help with timesync. clock=pit changes timer source, but at least vmware is set to serve daytime at fixed increments every clock cycle.

0
 
LVL 62

Expert Comment

by:gheist
ID: 24329877
... and with nohz enabled this VMWare misfeature makes realtime clock extremely slow ...
(But it helps to save electricity in all other environments)
0
 
LVL 62

Expert Comment

by:gheist
ID: 24329915
And also - do not use NTPD, uninstall it. VMWare tools does timesyncing part.
0
 

Author Comment

by:rajsolaris
ID: 24338282
yes. We are using ESX host OS 3.5.

 8 May 09:57:30 ntpdate[9080]: step time server 10.65.192.127 offset 28.027599 sec
 8 May 09:58:45 ntpdate[9102]: step time server 10.65.192.127 offset 43.196146 sec
 8 May 09:59:17 ntpdate[9121]: step time server 10.65.192.124 offset 15.146884 sec
 8 May 10:00:19 ntpdate[9135]: step time server 10.65.192.124 offset 11.712952 sec
 8 May 10:01:05 ntpdate[9379]: step time server 10.65.192.124 offset 3.154559 sec
 8 May 10:02:02 ntpdate[9433]: step time server 10.65.192.127 offset 0.909640 sec
 8 May 10:03:02 ntpdate[9436]: adjust time server 10.65.192.127 offset 0.049909 sec
 8 May 10:04:01 ntpdate[9439]: adjust time server 10.65.192.124 offset -0.007343 sec
 8 May 10:05:01 ntpdate[9445]: adjust time server 10.65.192.124 offset -0.014649 sec
 8 May 10:06:01 ntpdate[9465]: adjust time server 10.65.192.127 offset 0.008264 sec
 8 May 10:07:01 ntpdate[9468]: adjust time server 10.65.192.124 offset 0.071408 sec
 8 May 10:08:09 ntpdate[9473]: step time server 10.65.192.127 offset 7.034123 sec

This is what the output is after running ntpdate every 1 mint.
And tried adding following boot parameters:

title Red Hat Enterprise Linux ES (2.6.9-78.0.13.EL)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.9-78.0.13.EL ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
        initrd /boot/initrd-2.6.9-78.0.13.EL.img
title Red Hat Enterprise Linux ES (2.6.9-78.0.13.ELsmp)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.9-78.0.13.ELsmp ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
        initrd /boot/initrd-2.6.9-78.0.13.ELsmp.img
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.EL)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.9-42.0.8.EL ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
        initrd /boot/initrd-2.6.9-42.0.8.EL.img
title Red Hat Enterprise Linux ES (2.6.9-42.0.8.ELsmp)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.9-42.0.8.ELsmp ro root=LABEL=/ rhgb quiet clock=pit nosmp noapic nolapic
        initrd /boot/initrd-2.6.9-42.0.8.ELsmp.img


Thanks.
0
 
LVL 62

Expert Comment

by:gheist
ID: 24340629
My suggestion is to remove all the kernel parameters you have added to fix the clock and add only nohz=off

Your ntpdate result confirms that clock is way too slow, vmware counts time in interrupts, linux not anymore.

Your ESX has security issue http://support.vmware.com/selfsupport/download/ but no timing fixes until latest patch4.
0
 
LVL 9

Assisted Solution

by:svs
svs earned 1000 total points
ID: 24421130
gheist, VMware specifically recommends to run ntpd and not rely on vmware tools.  

Read their best practice paper on time synchronization for Linux guests:

http://kb.vmware.com/kb/1006427
0
 
LVL 62

Expert Comment

by:gheist
ID: 24423486
It looks like covering hole in the wall with piece of carpet.
Their timing infrastructure is incomplete. Running fixed speed clock in VM makes it somewhat accurate at least to the degree it does not drift extremely far away. On less loaded systems clock drift will go outside ntpd's limits and it will exit.
0

Featured Post

Why Off-Site Backups Are The Only Way To Go

You are probably backing up your data—but how and where? Ransomware is on the rise and there are variants that specifically target backups. Read on to discover why off-site is the way to go.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

1. Introduction As many people are interested in Linux but not as many are interested or knowledgeable (enough) to install Linux on their system, here is a safe way to try out Linux on your existing (Windows) system. The idea is that you insta…
The purpose of this article is to demonstrate how we can use conditional statements using Python.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Suggested Courses
Course of the Month10 days, left to enroll

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question