scotchyd
asked on
Strange Network Issue
I am running a Win2k domain, 100Mbps hubs. I have a main building and a remote building 200 yards away connected via 10 Mbps fiber switches.
The PCs in the main building are operating normally. The PCs in the remote are experiencing severe lag, almost like the bandwidth has been whittled down.
There are only 6 PCs on the remote 10Mbps link, and I have shut all but 1 down to rule out congestion.
I have reset the switches and the remote hub.
I ran a ping -t -l 64000 to the gateway in the main building, and all requests timed out.
I ran a ping -t to the gateway and all replied in <1ms.
When trying to access any domain resources, including Active Directory itself, I either get a resource not available error, or a not responding in the window.
When I browse the internet, I get reasonably quick page loads.
I also tried streaming music at 128kbps from Shoutcast, coming through the aforementioned gateway and it streamed fine! No drops or snags.
This remote segement was running OK up until a few weeks ago when it dropped off the network entirely. I ended up powering off all hubs and switches in the main building, and voila, connection re-established. That didn't work this time.
Any ideas?
The PCs in the main building are operating normally. The PCs in the remote are experiencing severe lag, almost like the bandwidth has been whittled down.
There are only 6 PCs on the remote 10Mbps link, and I have shut all but 1 down to rule out congestion.
I have reset the switches and the remote hub.
I ran a ping -t -l 64000 to the gateway in the main building, and all requests timed out.
I ran a ping -t to the gateway and all replied in <1ms.
When trying to access any domain resources, including Active Directory itself, I either get a resource not available error, or a not responding in the window.
When I browse the internet, I get reasonably quick page loads.
I also tried streaming music at 128kbps from Shoutcast, coming through the aforementioned gateway and it streamed fine! No drops or snags.
This remote segement was running OK up until a few weeks ago when it dropped off the network entirely. I ended up powering off all hubs and switches in the main building, and voila, connection re-established. That didn't work this time.
Any ideas?
Are they managed switches? Anyway to see the table they are using to resolve ip's to MAC's? Maybe the switch is not getting out of discovery mode or something is spoofing the MAC's. Specifiying a packet size isn't really necessary to trouble shooot this problem. Sounds like the DNS is working fine if you are surfing the web. Why did the switch knock off in the first place? Replace the switch....
10Mb fiber!?!? Was that right? That's kind of strange to begin with - these days old equipment uses 100FX (100Mb Fiber) - newer stuff would use 1000FX (Fiber GigE).
Could be an autonegotiate problem on your uplinks. If it's 10Mb - do you have the interfaces set to 10Mb or Autonegotiate? can you get into your switches and look at the interface stats - are they up? Do they have errors?
If you can't ping with 64000 packet size, you might have equipment which can't or won't fragment the packets. Have you tried smaller packets - like 1500B or 1400B? There MIGHT be an MTU issue in the network, but it's strange it would show up all of a sudden.
As Nick asked, you need to determine the root cause of your equipment dropping - if it's old, and it sounds like it with 10Mb fiber connections - then it could simply be aging and dying. Or you could have fans which have died and your equipment is shutting down when it gets too hot, and then it cools, and will come back on fine, until it gets hot again.
After you get it working, you might consider putting in a better connection - newer equipment with faster connections, and using this as a backup link - so use Spanning Tree or something similar which would bring it up in case of a failure on your newer faster connection.
Could be an autonegotiate problem on your uplinks. If it's 10Mb - do you have the interfaces set to 10Mb or Autonegotiate? can you get into your switches and look at the interface stats - are they up? Do they have errors?
If you can't ping with 64000 packet size, you might have equipment which can't or won't fragment the packets. Have you tried smaller packets - like 1500B or 1400B? There MIGHT be an MTU issue in the network, but it's strange it would show up all of a sudden.
As Nick asked, you need to determine the root cause of your equipment dropping - if it's old, and it sounds like it with 10Mb fiber connections - then it could simply be aging and dying. Or you could have fans which have died and your equipment is shutting down when it gets too hot, and then it cools, and will come back on fine, until it gets hot again.
After you get it working, you might consider putting in a better connection - newer equipment with faster connections, and using this as a backup link - so use Spanning Tree or something similar which would bring it up in case of a failure on your newer faster connection.
ASKER
The only switches are the fiber switches (Allied Telesyn MC13), and they are about 6 years old. They were already installed when I took over. The rest of the network is on 100 Mbps hubs. The fiber was installed to overcome the distance limitation, not because of bandwidth needs.
I have already ordered GigE switches to replace all the hubs and new 100 Mbps fiber tranceivers to replace the 10 Mbps ones. I believe that this may be a hardware issue.
But, it is perplexing that the remote PCs can ping, use DNS, access emails (slowly) from the central exchange server, and access the internet, while at the same time, network shares and Active Directory are almost inaccessible. Occasionally we can pull up the network neighborhood (slowly), but it is too slow to work with.
Also, a regular sized ping series gets good response, while a 64,000 byte ping series fails 100%. It seems like something is limiting packet size, but there is no layer 3 equipment between the server and the PCs. It is: SERVER - 100mb HUB - 10mb fiber - 10mb fiber - 10mb hub - 6 remote PCs.
To summarize: Network traffic to and from outside the domain operates OK. Network traffic within the domain crawls and dies.
I'm stumped.
I have already ordered GigE switches to replace all the hubs and new 100 Mbps fiber tranceivers to replace the 10 Mbps ones. I believe that this may be a hardware issue.
But, it is perplexing that the remote PCs can ping, use DNS, access emails (slowly) from the central exchange server, and access the internet, while at the same time, network shares and Active Directory are almost inaccessible. Occasionally we can pull up the network neighborhood (slowly), but it is too slow to work with.
Also, a regular sized ping series gets good response, while a 64,000 byte ping series fails 100%. It seems like something is limiting packet size, but there is no layer 3 equipment between the server and the PCs. It is: SERVER - 100mb HUB - 10mb fiber - 10mb fiber - 10mb hub - 6 remote PCs.
To summarize: Network traffic to and from outside the domain operates OK. Network traffic within the domain crawls and dies.
I'm stumped.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
That's a very good recommendation. We've already got management's attention, and their open pocket books. :) Alas, sometimes it takes downtime to get it through their thick skulls.
The switches are on their way.
We have very few collisions as it is, but it seems odd that one segment would have this issue while the others are flying. Admittedly, this is the only 10mb segment, hence the new equipment order.
I just wanted to bounce this off of some of you networking gurus to see if it lights up any new ideas while I am waiting for my switches.
Do you recommend configuring the NICs in the remote PCs to 10mb full duplex?
The switches are on their way.
We have very few collisions as it is, but it seems odd that one segment would have this issue while the others are flying. Admittedly, this is the only 10mb segment, hence the new equipment order.
I just wanted to bounce this off of some of you networking gurus to see if it lights up any new ideas while I am waiting for my switches.
Do you recommend configuring the NICs in the remote PCs to 10mb full duplex?
I'd put them at 10 half. If the hubs are Auto, you've GOT to go HALF or Auto.
No comment has been added to this question in more than 21 days, so it is now classified as abandoned.
I will leave the following recommendation for this question in the Cleanup topic area:
Accept - pseudocyber
Any objections should be posted here in the next 4 days. After that time, the question will be closed.
Bill_Fleury
EE Cleanup Volunteer
I will leave the following recommendation for this question in the Cleanup topic area:
Accept - pseudocyber
Any objections should be posted here in the next 4 days. After that time, the question will be closed.
Bill_Fleury
EE Cleanup Volunteer