Slow network performance Server 2008 R2

I have a single Dell PE 2950, 2 CPU, 16GB Ram, 2 embedded Broadcom ports(teamed), Intel Gigabit ET Quad port card(only one port in use).

Problem:  I am only able to achieve 20% of any one connection to this server.

Here is what I have tried

netsh interface tcp set global autotuninglevel=disabled
netsh interface tcp set global chimney=disabled
netsh interface tcp set global rss=disabled
and a couple registry edits
All drivers for both cards are the latest versions.
None of this worked with the Broadcom so I assumed the Broadcom was junk and installed the Intel card.  Same results.

Here is the strange part: With the previous tcp settings disabled I get ~20MBps on the Intel port.  If I open the properties of the Intel card and CHANGE the RSS setting the connection will gradually increase to ~80-90 MBps for about 20 minutes and gradually go back to ~20 MBps.  I can repeat this process over and over.  It does not matter what RSS is set to.  Just the fact that the setting changed causes the increase in bandwidth.

Anyone???