Solved

NTP: Time Drift on Domain Controllers

Posted on 2016-07-14
22
88 Views
Last Modified: 2016-07-29
We have all physical Domain Controllers (for now).

6 Total Domain Controllers all in a single site and single subnet

OS'es:
3 Windows Server 2003
3 Windows Server 2008 R2

We have noticed time drift as pictured - can get into the "minutes" also.  Any idea why this would be happening?

ntp.png
0
Comment
Question by:K B
  • 10
  • 5
  • 3
  • +4
22 Comments
 
LVL 23

Expert Comment

by:Dr. Klahn
Comment Utility
You didn't specify the period over which you're seeing these drifts.  If the time update period is a week, a drift of minutes would not be unusual.  

My own experience is that drift of up to 10 seconds a day is common, and I've seen a few systems with particularly bad clocks where it was as high as a minute per day.  Over 7 days, a drift ot 15 seconds per day is a minute and a half.

On non-critical systems I set an update period of once per day, and on important systems every six hours.  Any system requiring very precise time (e.g., transaction processing) should probably have its own WWV or GPS clock.
2
 
LVL 82

Expert Comment

by:Dave Baldwin
Comment Utility
Internal PC clocks are notorious for drift.  They are very inexpensive (cheap) circuits dating from 1981...
0
 
LVL 39

Accepted Solution

by:
footech earned 250 total points
Comment Utility
It looks like that output is the result from running w32tm /monitor correct?

What I find odd is that they (except for one) are listed as stratum 1.  The PDCe should be set to sync with an external NTP source, and then every domain controller should sync with it.  So I would expect one stratum 2 (at best, depending on the source), and the others stratum 3 (or +1 of the PDCe).  And the RefIds should list the name and/or IP of the server it's syncing with.

You may want to run something like the following on the PDCe.
net stop w32time
w32tm /unregister
w32tm /register
net start w32time
w32tm /config /manualpeerlist:"0.us.pool.ntp.org,0x8 1.us.pool.ntp.org,0x8 2.us.pool.ntp.org,0x8 3.us.pool.ntp.org,0x8" /syncfromflags:MANUAL /reliable:yes /update

Open in new window


And the following to reset the other DCs to defaults.
net stop w32time
w32tm /unregister
w32tm /register
net start w32time

Open in new window


As Dr. Klahn said, if you need something more precise (typical resolution with a single site can vary by a few seconds, but it's often within +/-2 secs), then you need to move to something other than the Windows time service.
1
 
LVL 5

Author Comment

by:K B
Comment Utility
Thank you for your replies.
Yes, /monitor
That image was from 7 months ago.. this is from today... interesting the Stratum change.  what could cause this change in stratum?

ntp2.png
0
 
LVL 16

Expert Comment

by:Malmensa
Comment Utility
Odd.

The "Stratum 1" is a definite clue.

A Stratum 1 time server is a device that has its own internal clock, such as an inbuilt atomic clock.  The stratum increases with every network connection. So, usually you would have:
Stratum1: NIST Atomic clock at the National Institute of Standards and Technology, in Colorado.
Stratum2: A local NTP server, in your locality. Often an ISP will provide this server.
Stratum3: The DC in your organisation that holds the PDC Emulator role.
Stratum4: Other DCs in your organisation.

Client machines then get their time from a Stratum 3 or4 source, depending on which DC they find. With a  default install, the DCs will automatically sync time from the PDC, and clients will automatically sync from any DC, not configuration is required.

It is probably a good idea to set an NTP source for your DC, HOWEVER, this is not secured, and in theory a hacker could launch a DOS attack by skewing your time. For high security applications, it is possible to use dedicated RTC hardware, and in effect have your own time standard.

The fact that your DCs were reporting back as Stratum1 would mean that they think they are proper time sources, with an inbuilt atomic clock; a Stratum1 device can only sync to such a device, and not via a network connection.
0
 
LVL 5

Author Comment

by:K B
Comment Utility
Thank you for that information.. that really helps.
So I tried this in my lab to see what I got:

From elevated CMD prompt:

1. On PDC (restarted Windows Time service afterward):

w32tm /config /update /manualpeerlist:"0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org" /syncfromflags:manual /reliable:YES

w32tm /resync /rediscover /nowait

Open in new window


2. On only other Domain Controller (restarted Windows Time service afterward):
w32tm /config /syncfromflags:domhier /update

w32tm /resync /rediscover /nowait

Open in new window


Strangely I keep getting Stratum values of 2 for both DCs:

ntpLAB.png
Then I ran the commands that footech provided above and now it looks like this (is the Stratum of 2 for the PDC correct from your explaination?)

ntpLAB2.png
Could one force the stratum of "1".   How did the company end up with Stratum values of "1" to begin with?
0
 
LVL 5

Author Comment

by:K B
Comment Utility
also with a stratum of "1", would DCs ever check in with the PDC with a Stratum of "3"?
0
 
LVL 5

Author Comment

by:K B
Comment Utility
oh and these are just basic OLD servers.. not an inbuilt atomic clock to be found anywhere.
0
 
LVL 16

Expert Comment

by:Malmensa
Comment Utility
Stratum 1 devices can only ever sync to a Stratum 0 device, and not via a non-deterministic network. Has to be a direct, unshared physical connection. Typically a wire which receives a pulse every second.

Usually the Stratum0 device is an actual atomic clock, while the Stratum1 device is the computer connected directly to the device, via a direct connection, not a LAN. The Stratum1 device is therefore usually in some large, government owned facility.

A stand alone PC with no network connection is a kinda Stratum1 device, with the battery backed clock as the Stratum0 time source, but of course it is not at all accurate. This is what your DCs seem to have been.

Stratum 2 devices are usually dedicated servers at an ISP or other organisation, which check frequently with one or more Stratum 1 devices.

Your PDC looks to be a Stratum 3 device, which seems correct. It should be accurate to well under a second, so close enough for almost any purpose.
0
 
LVL 5

Author Comment

by:K B
Comment Utility
But how did all the non PDC domain controllers end up with that value of 1.. it was obviously wrong as each DC with the Stratum of "1" were pointing to the PDC (I just had it blurred out for privacy).  Could it have been forced that way?  Could some corruption in AD or .. could have caused it?  I just don't want it happening again as it caused widespread issues in an Active Directory of over 15,000 objects.

Thanks again for such detail explanations!
0
 
LVL 16

Expert Comment

by:Malmensa
Comment Utility
Are the DCs (non-PDC) physical machines, or VMs?  Most virtualisation software includes client side software that syncs the VMs with the host machine. Perhaps the Virtual DCs are getting that time instead of the time from the PDC?  

Not sure how that would explain the skew though.
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 5

Author Comment

by:K B
Comment Utility
They are all physical and at one point the time difference amounted to several minutes
0
 
LVL 16

Expert Comment

by:Malmensa
Comment Utility
What do: w32tm /query /status  and  w32tm /query /configuration spit out?
0
 
LVL 5

Author Comment

by:K B
Comment Utility
keep in mind that the crazy Stratum was almost 7 months ago, the second image with the correct Stratum was from today.. so this information will probably look correct (i am presuming)

from the PDC emulator:

C:\Windows\system32>w32tm /query /configuration
[Configuration]

EventLogFlags: 2 (Local)
AnnounceFlags: 5 (Local)
TimeJumpAuditOffset: 28800 (Local)
MinPollInterval: 6 (Local)
MaxPollInterval: 10 (Local)
MaxNegPhaseCorrection: 172800 (Local)
MaxPosPhaseCorrection: 172800 (Local)
MaxAllowedPhaseOffset: 300 (Local)

FrequencyCorrectRate: 4 (Local)
PollAdjustFactor: 5 (Local)
LargePhaseOffset: 50000000 (Local)
SpikeWatchPeriod: 900 (Local)
LocalClockDispersion: 10 (Local)
HoldPeriod: 5 (Local)
PhaseCorrectRate: 7 (Local)
UpdateInterval: 100 (Local)


[TimeProviders]

NtpClient (Local)
DllName: C:\Windows\system32\w32time.dll (Local)
Enabled: 1 (Local)
InputProvider: 1 (Local)
AllowNonstandardModeCombinations: 1 (Local)
ResolvePeerBackoffMinutes: 15 (Local)
ResolvePeerBackoffMaxTimes: 7 (Local)
CompatibilityFlags: 2147483648 (Local)
EventLogFlags: 1 (Local)
LargeSampleSkew: 3 (Local)
SpecialPollInterval: 3600 (Local)
Type: NTP (Local)
NtpServer: time.windows.com (Local)

NtpServer (Local)
DllName: C:\Windows\system32\w32time.dll (Local)
Enabled: 1 (Local)
InputProvider: 0 (Local)
AllowNonstandardModeCombinations: 1 (Local)

VMICTimeProvider (Local)
DllName: C:\Windows\System32\vmictimeprovider.dll (Local)
Enabled: 1 (Local)
InputProvider: 1 (Local)


C:\Windows\system32>w32tm /query /status
Leap Indicator: 0(no warning)
Stratum: 3 (secondary reference - syncd by (S)NTP)
Precision: -6 (15.625ms per tick)
Root Delay: 0.0625000s
Root Dispersion: 0.0919703s
ReferenceId: 0x1765BB44 (source IP:  23.101.187.68)
Last Successful Sync Time: 7/14/2016 1:38:46 PM
Source: time.windows.com
Poll Interval: 10 (1024s)

Open in new window

0
 
LVL 78

Expert Comment

by:David Johnson, CD, MVP
Comment Utility
you state that you set the time provider to 0.pool.ntp.org yet I see your time provider as time.windows.com ? which is it?
0
 
LVL 13

Expert Comment

by:frankhelk
Comment Utility
Hmmm ... W32time, the timekeeping service in Windows. I experienced enough trouble with that piece of crap when in NTP mode to avoid using it whenever I can.

For a mature timekeeping service with well documented behaviour, I'd recommend this:

Use a Windows port of the classic *ix NTP service on your DC VMs, and sync 'em with NTP time sources from pool.ntp.org. Ensure to disable the time sync features of VMware (to timekeeping services on one clock will cause time chaos). The NTP service software is free. Easy to install and configure, works like a charm and is stable as a rock. And it is nicer when it comes to one of the rare cases of troubleshooting.

See my article on NTP basics for the "How To".

The "classic" NTP service has a low ressource footprint, therefore the NTP functionality could be hooked onto existing machines or VM's like webservers, ftp servers, mailservers or database hosts - even in a DMZ - without visible performance impact.

If securtity is an issue, you might as well use local radio controlled clock appliances (see the article for that, too) in your LAN who serve times very reliable and precise.
0
 
LVL 5

Author Comment

by:K B
Comment Utility
David,
That was my lab.  I plan of using it in the production example too however (soon).
0
 
LVL 16

Assisted Solution

by:Malmensa
Malmensa earned 250 total points
Comment Utility
Only thing I am a little unsure about there is line 46. vmictimeprovider.dll is the client HyperV component that grabs the time from the host machine. It looks to be enabled, which seems a little odd, on a physical box.

To be honest, I don't know if this is normal, and I don't have a non-virtualised DC I can RDP too right now.
0
 
LVL 5

Author Comment

by:K B
Comment Utility
Malmensa,

that is very interesting!!  Here is a screenshot of another one of my lab domains where I have one Domain Controller and it is physical ..  it shows vmictimeprovider.dll as not enabled!:

ntplabVMICtimeproviderDLL.png
0
 
LVL 39

Expert Comment

by:footech
Comment Utility
One point I think I misspoke on.
typical resolution with a single site can vary by a few seconds, but it's often within +/-2 secs
I meant to say that while you may often be within these limits, the Windows Time service should not be relied upon if you require a precision of +/- 2 secs.  A good reference:
https://blogs.technet.microsoft.com/askds/2007/10/23/high-accuracy-w32time-requirements/

We have a NTP device that can sync with GPS, and when it does it is a Stratum 1 device.  When it syncs with some other NTP server via the internet, then its stratum will vary according to the stratum of the other NTP server.

The last thing I'm wondering about are the lines which read like
    NTP: +0.0000000s offset from local clock
I would expect that to refer to your PDCe (assuming default settings in a domain hierarchy) instead of "local clock".
0
 
LVL 5

Author Comment

by:K B
Comment Utility
Footech, what does your PDCe show when you run?

W32TM.EXE /monitor /computers:DC1,DC2,DC3,DC4

Open in new window


I believe it is looking to see the time difference from itself?
0
 
LVL 39

Expert Comment

by:footech
Comment Utility
Ah, I do see local clock when I specify the /computers switch.  I didn't notice that was a difference before.  I've just been running
w32tm /monitor /domain:
or
w32tm /monitor

Not quite sure what it's referring to.  Maybe difference between hardware and software clock?
I don't know if I'll get a chance to try to find out.
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

Suggested Solutions

SSL is a very common protocol used these days when browsing the web.  The purpose is to provide security to communication, but how does it do it?  There are several pieces at work that have to be setup before SSL will even work and it requires both …
Are you one of those front-line IT Service Desk staff fielding calls, replying to emails, all-the-while working to resolve end-user technological nightmares? I am! That's why I have put together this brief overview of tools and techniques I use in o…
This tutorial will walk an individual through the steps necessary to join and promote the first Windows Server 2012 domain controller into an Active Directory environment running on Windows Server 2008. Determine the location of the FSMO roles by lo…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now