Copying files to/from a Windows 2003 server with Raid(1) is ridiculously slow!!

I'm on the way to dinner but first thing that crosses my mind is actual performance numbers. Have you timed copying a semi-large file between clients, and the same file to or from the server?

You've confirmed every NIC is running at 100Mb/s?

Oh, and avoid the pencils. You need depth perception to plug in those enet cables.

Dave

100 MB file from server, approx 45 minutes
65 MB file from another client 4.5 minutes

I swear on all that is holy that all nics are 100 Mb/s, full duplex.  

I know, but I have to ask. Does the Cisco agree? No port or protocol throttling? Have you tried changing the server to another Cisco port?

How is copying locally at the server?

Anything else going on at the server?

Dave
(Using the pencil method while waiting for dinner guests...)

>infrastructure over Cisco switch (I've checked config many times)
Can you past output of "show interface fast 0/x" x=port connected to server..

Have you checked the disk manager utility to see if all the disks are marked "healthy"?

Also, are you seeing any disk related errors in the event log (system event log)?

Are the hard drives the exact same model, size, etc?

I had similar issues with two IDE drived mirrored with a Promise card (can't remember the actual model number).  Anyhow, I had all kinds of problems with one of the drives always going down and really slow copying between that machine and anywhere. The only solution I came up with was to plug one drive into the promise card and plug the other one to be used in the set into the IDE controller on the mother board.  Once I did that, the drive failure and performance problems vanished.  

I seem to remember seeing something on the promise site that hinted I should try this.  Give it a shot and see if it works.

Good luck!

Things to check:
* crc errors on that switch port, where server is connected. may be failing port, bad contact, bad cable. can be bad network card on server, too.
* as mentioned by tjq100, check server HDDs - maybe one of mirror disks is gone.
* is antivirus software running on server? although it may have impact when copying large amount of small files (had this problem with Kaspersky antivirus)

I know, but I have to ask. Does the Cisco agree? No port or protocol throttling? Have you tried changing the server to another Cisco port?
>>I can't recall if I tried another port, but I can say that this problem existed when the only network device was a Linksys Business Router (Wireless G) with the server plugged into one of its 100 mb/s full ports.  Exact same symptoms.

How is copying locally at the server?
>>Funny you should ask... it seems a little slow.  As I mentioned, there are two drives to write between locally.  It's not as fast as it should be.  Certainly faster than over the network though.

Anything else going on at the server?
>>Not really, no real errors in the event log to speak of.  

Can you past output of "show interface fast 0/x" x=port connected to server..
>>I could get that info, but it might be a couple of days.  This is occurring at a client site, and I won't be able to make it back out until this weekend.

Have you checked the disk manager utility to see if all the disks are marked "healthy"?
>>Yes, Healthy, even did a defrag on both drives.

Also, are you seeing any disk related errors in the event log (system event log)?
>>No disk related errors.

Are the hard drives the exact same model, size, etc?
>>The HD's in the RAID array are the exact same.

I had similar issues with two IDE drived mirrored with a Promise card (can't remember the actual model number).  Anyhow, I had all kinds of problems with one of the drives always going down and really slow copying between that machine and anywhere. The only solution I came up with was to plug one drive into the promise card and plug the other one to be used in the set into the IDE controller on the mother board.  Once I did that, the drive failure and performance problems vanished.  I seem to remember seeing something on the promise site that hinted I should try this.  Give it a shot and see if it works.
>>I like the sound of this solution, though I still suspect it's tied into Server 2003 a little bit, but given the local slowness, I still can't rule out hardware problem.  Will the Promise RAID array allow me to switch one of the RAID drives to the IDE controller without modifying any kind of configuration?  I would really like to try that if possible.  

Things to check:
* crc errors on that switch port, where server is connected. may be failing port, bad contact, bad cable. can be bad network card on server, too.
>>I'll check, but the symptoms are a continuation of things going wrong even before the switch was implemented.

* is antivirus software running on server? although it may have impact when copying large amount of small files (had this problem with Kaspersky antivirus)
>>No antivirus running while doing tests.

>>Further note - Tried the driver and BIOS update for the Promise card.  No fix.
Also had a scare while implementing the reg hack/GPO fix I mentioned earlier that I found in another post.  Something we disabled relative to the digital signatures and encryption stopped clients from being able to access network drives.  They could log in, but not access drives.  Then when we went back and looked at the Registry, the values we had changed to zero were 1 again.  Odd.  The next hour was spent trying to get everything back to some kind of almalgamated state where it wasn't identical to how it was before the changes, but wasn't exactly the full change we started with, and everything seemed to work again.  

Is the promise card handling all of the drive management or are you using Windows Server to create the raid array?  If Windows, then you should be able to move the drive to an open IDE cable and try it out.  Good luck!

Promise is handling.

> Make sure that all Nic's and Ports are set to 100 Full, NOT AUTO.
> It makes a difference, just because it detects at 100 full it can still have issues.

Unfortunately, I've seen cases where the opposite was true.  It really does depend on the mix of switch and NIC manufacturers.

I've also seen very poor performance due to faulty/mistreated cables.

So I'd try swapping out first the cable between server and switch, and then the server's NIC (for another brand, preferably something like 3com or Intel).

What about the default gateway and network settings on the server.  One time while playing with my network I had setup a test default gateway which then bounced back to a different gateway and back to the local network.  Proabably not the same issue but maybe the packets are getting routed around a bit more than you want.

I finally used a packet sniffer to see all of the hops (then hit myself in the head).

What is the speed, when you copy files locally? It should be 7-10 Mb/s (tested with Total Commander, 300 MB file). Computer was W2K Server, SATA RAID1 on integrated Promise controller.

Have you tried updating network card drivers? Also, can you test another computer from the same location, where server is now? It should show, where the problem is - in your server configuration, or in network (cables, switch, combination of switch and NIC).

What about the default gateway and network settings on the server.  One time while playing with my network I had setup a test default gateway which then bounced back to a different gateway and back to the local network.  Proabably not the same issue but maybe the packets are getting routed around a bit more than you want.
>>Settings here are fine

What is the speed, when you copy files locally? It should be 7-10 Mb/s (tested with Total Commander, 300 MB file). Computer was W2K Server, SATA RAID1 on integrated Promise controller.
>>7-10 Mb(its)/s... not MB(ytes)/s?  Are you sure?  Drives like this typically have a data transfer rate of 100+ MB(ytes)/s to the processor.  Why so slow from one drive to another?

Have you tried updating network card drivers? Also, can you test another computer from the same location, where server is now? It should show, where the problem is - in your server configuration, or in network (cables, switch, combination of switch and NIC).
>>I've effectively taken the switch out of the equation.  It's not the switch.  

7-10 MB/s, you are correct.

OK, if it is not switch, it might be cables still.

Please try out Microsoft document 321098 for solving file copy network problems with Windows 2003 Active Directory Servers. I had the same performance problem with 2 of my customers and this speeds up file copying a lot. Just follow the document on the Microsoft site above.

Best Regards

Admin, I want to try last fix, but can't until 9/30/04 (after hours) please do not close this case.

infombp, how do I get to that document?

Here's the document referenced:
http://support.microsoft.com/default.aspx?scid=kb;[LN];321098

Don't worry, nobody will close this question on you (unless it gets classified as abandoned about 3-4 months from now....)

On support.microsoft.com website you can search and type the number.
Regards

So, it turns out that the issue was a combination of Bad NIC + some default NIC settings.

Even when the NICS and the Ports on the switch were forced to 100 full, there were issues.  I replaced the NIC with a newer model, that was actually made for Server 2003, and it was like a lightbulb was turned on.  The old NIC said it was 100 Full, the switch agreed, no errors, it just couldn't get through put.

So there you go.  You can't always rule out hardware, even when you think you've ruled out hardware.

Bad Hardware Sucks when it shows no problems. Even on good hardware, I have seen where setting at 100 instead of Auto is a better thing. Normally the tell tale sign is lots of errors.