Network connections and/or speeds plummet daily, usually late afternoon

I have recurring network speed issues very similar to that described in this post (http://www.experts-exchange.com/Operating_Systems/SBS_Small_Business_Server/Q_22078159.html).  Basically, various machines in my network lose connectivity with the server or experience drastic reductions in speed on a daily basis, usually between 4:00PM and 6:00PM.  This is not an internet bandwidth issue; the main problem is with connections to the server or printing on the network and can be fixed with a reboot of the affected machine.  All the following machines are regulary affected, although never all at once and rarely on the same day:

Notebooks 1, 2, 3, 4:  Wireless through Buffalo access point
Desktops: 1, 2, 3:  Ethernet (10/100 cards)
Notebook 5 Ethernet (gigabit card)

All the machines are running XP Pro, and the server is running MS Server 2003, SP1, with ten licenses.  It is used only as a fileserver & domain controller (my terminology may be off here as far as the domain controller is concerned; all I'm trying to say is that this machine is where I keep all the usernames/passwords, permissions, etc.).  I have a Netgear gigabit router, and that connects to a Netgear gigabit switch and to the Buffalo wireless access point.

This problem affects different machines, but never too many at a time and always at the end of the day.  I'm stumped-   Help???

Thanks,

jdj
jdjintxAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

giltjrCommented:
Are you running any backups?  Automated disk defrager?  Any messages in the event   logs?

You may want to install a packet sniffer (I recomment www.wireshark.org) and see what network traffic is flowing, or not as the case may be.
0
pkutterCommented:
I've had problems with old firmware versions on Netgear equipment. What models are the Router switch and the Buffalo AP? Have you checked for firmware updates? Also did this problem just start or has it been on going? Can you tie the start of the problem to any upgrade or equipment that you may have added to the network?
0
jdjintxAuthor Commented:
Backups run at midnight, there is no scheduled defrag, and I have not looked at event logs (I will try tomorrow).  I will also look into the packet sniffer.  

The netgear & buffalo equipment is recent, and were added as attempts at solving this problem.  I'll double chack models & firmware tomorrow.  The problem has been ongoing, and the regular schedule is what has me so bewildered; everyone in the office seems to know that if they "win" the daily drawing, they can expect problems by 6:00...

jdj
0
C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

drtoto82Commented:
I guess we have a Windows Update Schedule working from 4 - 6 am !!
Try to use a network monitor and post us the resul at this time if it weren't the windows update !!
0
jdjintxAuthor Commented:
DrToto82: Our difficulties typically happen at 4-6 PM, but I see your point.  I'm afraid I'm not familiar with "network monitors," could you elaborate?

Thanks,

jdj
0
drtoto82Commented:
I wish I can be of somehelp to you .

You can use the microsoft network monitor . Or I do recomment that u use something like ethereal (www.ethereal.com)  which is free .

U can then make a capture of the traffic for a while and then stop the capture .

We can then check the traffic ports , source and destination and even the traffic itself if unencrypted.
So , after making the capture, we can make a filter on a suspect computer's IP address to check its transmissited packets to see what it is sending and receving to/from whom on what ports ...

That will help us to have better understanding . ...

Try it and tell me if u still need more help .
0
drtoto82Commented:
Did u try it ? Did it work ?
If your problem is solved plz accept and give me the points ..

If not ,just  let us know , ur welcomed to ask any more questions to let us help u . :)
0
jdjintxAuthor Commented:
Sorry for the delay - I'm an engineer, not an IT guy, so I've been dragged into some other things at work.  I'm going to try to come in over the weekend to sort this out & I'll let everyone know what happens...
0
drtoto82Commented:
nop . ... Take care , .. .they are trapping you to change your career. Anybody gets involved in IT willl love it to the brain.
I was in a medical school , then moved to dentistry . I 'm in my last year now . ... and I 'm MCSE too !!!!! So , welcome to the IT world ...
0
jdjintxAuthor Commented:
All- I'm still around & still interested, but I got pulled into a project thas prevented me from digging into the server issues.  Maybe this weekend or early next week.  Thanks so much for your patience!
0
jdjintxAuthor Commented:
I'm finally back if anyone is still interested in this issue...

I have installed Ethereal and am currently reading through the user's guide.  If there are any quick tips that will jump-start my ability to use this software that would be great, otherwise it may take some time before I'm proficient enough with this to get any results.

Thanks for your patience!
0
jdjintxAuthor Commented:
Correction:  For what it's worth, I'm using Wireshark,. not Ethereal.

jdj
0
jdjintxAuthor Commented:
As my delays seem to have killed this topic, can anyone advise how to lcose this out without a solution?

Thanks,

jdj
0
giltjrCommented:
Sorry I meant to reply earlier.

What you can do is install wireshark on a desktop and run it between 1600-1800 and see what type of traffic it sees.  You may not want to run it for the full 2 hours, as it could collect a LOT of data.

On the server I you can use MS's Netmon tool if you don't feel comfortable installing Wireshark on it.  Wireshark can read netmon files, so you can copy the file someplace else and analyze with Wireshark.
0
jdjintxAuthor Commented:
I installed Wireshark, I have the log files, and nothing in them makes any sense to me.  What next?

Thanks,

jdj
0
giltjrCommented:
Do you see a large volume of traffic during the time period?  How long did you run the capture?  Were you experiencing the problem during the capture?
0
jdjintxAuthor Commented:
I ran the capture for about 45 minutes, and we did have the "lag" during that time.  During the capture, I rebooted my laptop and the lag for that machine disappeared (as usual).  As far as the traffic is concerned,  we're a small company and nothing that was happening at that time was out of the ordinary, so we should not have been seeing anything unusual.  "Large" is a relative term, however, so I'm not sure how our level of traffic would best be categorized.  


I have to admit, most of the logged information is completely meaningless to me.  Something that does jump out is a significant quantity of "Read Andx Response, FID: 0x0038, 4096 bytes" immediately followed by "NBSS Continuation Message".  Wireshark had these highlighted in a black background and all of these events were for the same machine (same IP address).  Looks pretty ominous...

jdj
0
giltjrCommented:
The NBSS  Read Andx Response and the NBSS Continution Message means that you are reading a file off of the file server.  The 1st packet for the 1st part of the file will be the "NBSS Read Andx Reponse".  If the file is bigger than 1400 or so bytes then you will get a NBSS Continution Message.  

Say you have a file that is 10,000 bytes long.  You should get one NBSS Read Andx Response packet and somewhere between 6 and 8 NBSS Continuation Messages for that single file.

You may want to see what file it is and how big it is.

You may have it in black with the message "TCP CHECKSUM INCORRECT".  This is fine, as long as it is from your comptuer back to the server.  Most computers these days come with NIC's that allow the OS to offload the TCP checksum process to the NIC.  This reduces the overhead on the OS and computuer.  So when the packet goes from the OS to the NIC (where Wireshark captures it) the checksum is bad.  The NIC will calculate the checksum and it will be correct when it hits the wire.

By large I mean enough traffic to drive the NIC's utilization to 70% or higher.
0
jdjintxAuthor Commented:
Thanks, I guess all those entries weren't so ominous after all.  The PC and router have gigabit connections, and the machines connected are all at a minimum of 100MB.  The router and NIC card in the server are both relatively new, and were purchased as my first attempt at solving this problem.

With that said, is there anywhere in the WireShark log that shows NIC utilization?  I doesn't jump out at me as I review the logs.

Thanks,

jdj
0
giltjrCommented:
There is nothing that shows %utilization.  If you click on Statistics --> Summary it will give you bytes per second.  Since a 100 Mbps NIC can do a maximum of 12.5 MBps you can calculate the %utilization.

You can also produce graphic.  Click on Statictics --> IO Graphs and play with it a little bit.
0
jdjintxAuthor Commented:
Thanks.  I couldn't find a session that an average MB/Sec higher than 3.5, and most were less than 2.  Any other suggestions as to what I should look for in the reports as a sign of something going haywire?

Thanks,

jdj
0
giltjrCommented:
Was that an average or a peak?

Could there be something not network related going on?  A server maxing out on CPU or disk IO?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
jdjintxAuthor Commented:
Actually, I'm still working on this.  After giltjr's latest response, I looked at non-network issues and discovered that the OS was in a small (<6GB) partition.  I spent several days (with help from EE archives) trying to re-size the the drive partitions, but could not do so without losing the content.  My next step was to add a SATA drive (in addition to the mirrored 30GB SCSI drives), copy the OS onto a 30GB partition on the SATA drive, and then use the 30GB SCSI drives only for data.  I only recently got this all set up, and now I'm waiting for conclusive results.

Overall performance has improved in certain areas- believe it or not, my KVM switch now works properly.  This is not a coincidence, and I can explain in more detail if anyone is interested.

In the last week, one user has had connectivity trouble, and I have personally experienced a brief slowdown that corrected itself without a reboot.  Both problems were at the dreaded 6:00-6:30PM standard-problem-time, but on different days.  I'm hoping that these were flukes and that the overall problem has gone away, but I think it's probably premature to decide I'm finally out of the woods.

Sorry I haven't had time to provide this update earlier, but non-IT responsibilities have to come first here, and I'm still not absolutely certain that we've actually fixed the problem.  If it is resolved, I would expect to award the points to glitrjr for pointing me in the right direction.  Can we have one more week to evaluate performance?

Thanks again for all the time & effort!

jdj
0
giltjrCommented:
Umm, you basically increased the size of the partition and changed the type of drive to a faster (I assume) drive. and the problems have seem to drastically reduced in frequency and duration.

I would double check the backup and defragging again.

A larger partition would mean that the disk should not get as fragmented.  A faster drive would mean that when the defragging process runs, it will take less time.

If you were backing to a medium that was faster than the original drive, then backups would take less time also.

You may want to setup perfmon on this server and have it watch things like CPU busy, disk IO, and network IO.  Then when it happens again, see what spiked when.

0
jdjintxAuthor Commented:
Thanks, I'll do that.  I understand (I think) your points about what one should expect when the OS partition size is increased and the drive speed is improved, but doesn't this require that the OS was functioning properly to begin with?  It is my hope that the different drive did not just improve performance, but that it actually fixed some parts of the OS that did not function properly when the it was constrained in a partition that was less than 6GB.  Part of my learning curve with the drive addition required a call to Gateway tech support, and they were amazed that the machine would even function when the OS partition was less than 10GB (they recommend 25GB or more).

As far as the backup is concerned, it runs at 11:00PM daily, and I check this every morning so I am positive that the backup is not an issue at 6:00PM.  The fact that the problems used to happen as early as 4:00PM also support the idea that this is not the result of a regularly scheduled event.  My hope is that there were some buffers/temp-files/other-OS-stuff that would build up in size over the course of a day until they just caused the whole thing to bog down.  I undersatnd that this also could be optimistic nonsense and that the improved performance could be a coincidence.

You obviously know a lot more about this than I do, so what do you think?  Is there a possibility that the drive change could "fix" some problems instead of merely "improving" what was already there?  If not, let me know and I'll get started on the performance logs again.

Thanks!

jdj
0
jdjintxAuthor Commented:
It's been two more weeks and we have not had any slowdowns or other issues reported by the guys (I've been on a trip, so I don't have any first-hand observations).  It's also completing windows updates which were unsuccessful before the drive change, so I think I've got this wrapped up.  Hopefully I won't be back in a few more weeks with the same issues!!!!

Thanks for all the help.

jdj
0
giltjrCommented:
Great and thanks.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Networking

From novice to tech pro — start learning today.