Windows Server 2008 - Memory Leak - SMB2

Published:
 It would seem that when you are copying files between two Windows 2008 x64 servers it first copies the file into memory before sending the file to it’s destination.  The larger the file you try to copy the more memory it will consume and the more it will increase the likelihood of us running out of memory. This problem does not manifest when files are copied between different Windows Server versions (2003 to 2008). I’ll include a technical explanation below.

  To protect us going forward we are implementing a moratorium on copying large files through Windows Explorer from any Windows 2008 servers  when the destination OS is any of the following: Vista (any version), Windows 7 (any version), and Windows 2008.

Technical Explanation

Attached, find an image from some perfmon counters running on Primary Database server during the problem period that shows the Cache Bytes increasing (file copy consuming available memory) and the Available Memory Bytes decreasing.
 
 Test Resulted in Large Memory Consumption

Large file copy within Windows Explorer between 2 Windows 2008 Servers (The OS decides to use SMB2)
 
Test Resulted in No Memory Consumption Problems

 Large file copy within Windows Explorer between 1 Windows 2003 Server and 1 Windows 2008 Server

Large file copy within Windows Explorer between 2 Windows 2008 Servers with SMB2 disabled

Large file copy between 2 Windows 2008 Servers using ESEUTIL to copy the file
 
  According to Microsoft the Windows Explorer is performing as designed when executing a copy of a large file. When Windows Server determines that it has 2, Windows 2008 servers communicating, it uses SMB2 to facilitate the transfer, which leverages a buffered file copy. There are many utilities that suffer from this problem as well (OS commands COPY, XCOPY, and ROBOCOPY (2008 only)).
  The only safe work around when moving files between 2 Windows 2008 Servers is to use an Exchange utility called ESEUTIL. One of the methods this utility supports is what’s called an unbuffered copy, ie 0 impact on available memory’s bottom line.
1
4,599 Views

Comments (4)

Author

Commented:
We came across this issue because it crippled our SQL Servers in production. A large file was being transferred (a database Backup) from one SQL Server to another. This had massive impact upon the available memory for the the SQL Server process.
Ted BouskillSoftware Development Manager
CERTIFIED EXPERT
Top Expert 2009

Commented:
Ah, you should have mentioned that specific example.

Have you set any limits to the memory SQL is allowed to use?  Sadly SQL server will try to consume all physical memory and doesn't care if swapping starts to occur (which will kill performance)

What method are you using to do the file copy?  Is it a batch file or did you setup an SSIS automated task to execute the copy?
CERTIFIED EXPERT

Commented:
athelu,

Congratulations! Your article has been published.

ericpete
Page Editor
Pretty sure if you turn off superfetch in the system services it fixes this issue as well..

Have a question about something in this article? You can receive help directly from the article author. Sign up for a free trial to get started.