tjnoles
asked on
Windows Server 2003 SP1 Lost Network Connections after Reboot
This is one of the more bizarre situations I've encountered and any help would be appreciated GREATLY. Here's the scenario:
We have several computers on a development network, all running Windows Server 2003 w/ SP1. They've been running SP1 for a while and some of the hotfixes that came after it, but no new patches have been installed in several weeks. The new patches are downloaded every night, if any, but they are not automatically installed. The machines are not used to browse the Internet and no unauthorized software is allowed on them, plus they have appropriate scanners running, so the chance of a virus or spyware is minimal. Everything has been working great for quite some time. Nothing that I know of has recently changed EXCEPT for the installation of AuthDiag 1.0 from Microsoft on two of the machines TODAY.
Shortly after the installation of AuthDiag, one of the machines was rebooted. Note that this was NOT one of the machines that AuthDiag was installed on (however, one of the machines on which it WAS installed was also later "afflicted"). When it attempted to come back up, it hung at "Applying Computer Settings" and would never proceed past this point. During troubleshooting, another machine was rebooted. It too seemed to take a while to finish the "Applying Computer Settings", but at that stage it presented the message that a service had failed. Upon checking, it was determined that the "Network Connections" service did not start. Attempts to start this service manually also failed with a timeout. After opening the Network Connections pane (right-click My Network Places >Â Properties) NO network connections were found. Suspecting a pattern, two more machines were rebooted with the exact same results - slow to pass "Applying Computer settings, Network Connections service failure, and no connections shown in the Network Connections pane.
AuthDiag was removed and some preliminary research on the Net seemed to indicate SP1 as a potential culprit, as did another employee who suggested the same thing, so as an experiment SP1 was removed from two of the machines and they were rebooted, at which time the network connections were restored. However, this is kind of a drastic step and I'd like to find out more about WHY this occurred if I can, so I decided to try this forum on the chance that maybe someone else has seen something like this before.
Note that I'm not pointing at AuthDiag, or even SP1, as a definitive culprit. All I know at this point is that these entities happened to coincide with the start of the problem or are otherwise potentially related to the problem in some fashion, as described. So if anyone can shed some light on this strange and highly-irritating issue, I would definitely appreciate it. I would like to put SP1 back on so I can stay current with security updates, but at this point I don't know if that can be done. Due to the bizarre nature of the problem and the fact that our development system is pretty important, I'm assigning 500 points.
We have several computers on a development network, all running Windows Server 2003 w/ SP1. They've been running SP1 for a while and some of the hotfixes that came after it, but no new patches have been installed in several weeks. The new patches are downloaded every night, if any, but they are not automatically installed. The machines are not used to browse the Internet and no unauthorized software is allowed on them, plus they have appropriate scanners running, so the chance of a virus or spyware is minimal. Everything has been working great for quite some time. Nothing that I know of has recently changed EXCEPT for the installation of AuthDiag 1.0 from Microsoft on two of the machines TODAY.
Shortly after the installation of AuthDiag, one of the machines was rebooted. Note that this was NOT one of the machines that AuthDiag was installed on (however, one of the machines on which it WAS installed was also later "afflicted"). When it attempted to come back up, it hung at "Applying Computer Settings" and would never proceed past this point. During troubleshooting, another machine was rebooted. It too seemed to take a while to finish the "Applying Computer Settings", but at that stage it presented the message that a service had failed. Upon checking, it was determined that the "Network Connections" service did not start. Attempts to start this service manually also failed with a timeout. After opening the Network Connections pane (right-click My Network Places >Â Properties) NO network connections were found. Suspecting a pattern, two more machines were rebooted with the exact same results - slow to pass "Applying Computer settings, Network Connections service failure, and no connections shown in the Network Connections pane.
AuthDiag was removed and some preliminary research on the Net seemed to indicate SP1 as a potential culprit, as did another employee who suggested the same thing, so as an experiment SP1 was removed from two of the machines and they were rebooted, at which time the network connections were restored. However, this is kind of a drastic step and I'd like to find out more about WHY this occurred if I can, so I decided to try this forum on the chance that maybe someone else has seen something like this before.
Note that I'm not pointing at AuthDiag, or even SP1, as a definitive culprit. All I know at this point is that these entities happened to coincide with the start of the problem or are otherwise potentially related to the problem in some fashion, as described. So if anyone can shed some light on this strange and highly-irritating issue, I would definitely appreciate it. I would like to put SP1 back on so I can stay current with security updates, but at this point I don't know if that can be done. Due to the bizarre nature of the problem and the fact that our development system is pretty important, I'm assigning 500 points.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Netman66,
So, unless you have any further information re: this situation, I suppose I need to follow the grading guidelines and assign a "B", which I will do in a few days (to give you time to respond) if I do not see any additional responses. If you know WHY these systems are behaving this way, that definitely ranks as an "A". In either case, you deserve the points and you have my thanks.
So, unless you have any further information re: this situation, I suppose I need to follow the grading guidelines and assign a "B", which I will do in a few days (to give you time to respond) if I do not see any additional responses. If you know WHY these systems are behaving this way, that definitely ranks as an "A". In either case, you deserve the points and you have my thanks.
The servers that are functioning with the Network Service account - can you compare te setting below between the servers you set to Local System?
Open Administrative Tools
Open Component Services
Expand Component Services>Computers
Right-click My Computer and select Properties
On the MSDTC tab, select the Security Configuration button
Check the DTC Logon account - is it the same between servers that are functional and ones that were not?
You may want to go through each tab and setting to compare the differences (if any) between servers using the Network account for RPC and the servers using SYSTEM for RPC.
Let me know if you find anything interesting.
As for your last post, do what you think is fair - it's okay with me.
Open Administrative Tools
Open Component Services
Expand Component Services>Computers
Right-click My Computer and select Properties
On the MSDTC tab, select the Security Configuration button
Check the DTC Logon account - is it the same between servers that are functional and ones that were not?
You may want to go through each tab and setting to compare the differences (if any) between servers using the Network account for RPC and the servers using SYSTEM for RPC.
Let me know if you find anything interesting.
As for your last post, do what you think is fair - it's okay with me.
ASKER
Netman66,
I'm leaving on a business trip today and won't be back until the Monday after next, but I will compare these settings when I return. Thank you.
I'm leaving on a business trip today and won't be back until the Monday after next, but I will compare these settings when I return. Thank you.
ASKER
I saw that article earlier but put it on the back burner since the summary stated that "despite these issues, networking continues to function correctly" and our affected systems' networking were definitely not working. I decided to revisit it since you mentioned it and noticed the following:
Following it verbatim didn't work but it gave me some ideas that did, as far as I can tell. First, I was able to restore network connectivity by switching the RPC service logon from "Network Service" to "Local System". I know there is a dependency of the "Network Connections" service on the "RPC" service, so I decided to try the account switch. I've read about some security issues involving the use of "Local System" vs. "Network Service" but on the affected systems the "Network Connections" service will NOT start unless the "RPC" service is logging on with it, so there it is.
I checked what some other working systems had their "RPC" service using for a comparison. The systems that had service pack 1 removed ARE using "Local System" for the "RPC" service (this happened automatically) - after installation of the service pack it switches to "Network Service" (I tried this). However, I have some "pristine" Win2K3 SP1 systems running on a virtual network that have their "RPC" service using "Network Service" and they aren't having any problems, so go figure. For some reason, the systems on our development network can't run like that.
So, I've been testing the systems pretty heavily, and have even installed SP1 and all following hotfixes and, as long as "Local System" is used for the "RPC" service, all is well at this point. I intend to keep investigating this, but I'm awarding the points to you since your help was the "catalyst" for this discovery and the systems ARE working.