SMB3.0 Copy from FreeBSD to Windows 2012 R2

Posted on 2015-01-05
Last Modified: 2016-02-11

I was hoping someone could help explain this for me.

I have a Windows 2012 Server, I have setup perfmon to record the Disk Read Bytes/Sec and Disk Write Bytes/sec of a local disk.  I have the lowest sample set to 1 second.

I am copying a file from the local disk to an Isilon Node (running free BSD) everything is perfect, numbers look good.

The problem is copying from Isilon to the local disk.  I see values that 'hiccup'.  Typically I get several hundred megabytes for a few seconds and then 0, then back high then zero again.  The high numbers are always at least one in a row up to 6 (so 1 second to 6 seconds) the drop is always 1 second....    

NOTE: I have verified that I am connected and transferring with SMB 3.0.

I am not seeing this with SMB 2.0 copies so I am thinking this is something to do with SMB 3.0??
Question by:adamc817
  • 5
  • 3
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 40533899
What is the network connection? Gigabit will only get you 100MBps.
Typically I get several hundred megabytes for a few seconds and then 0

Author Comment

ID: 40533959
Sorry, setup as 10gig SFP+
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 40533992
I'm not familiar with Isilon, but if you are using ZFS in freeBSD, it can have this behavior. If you have a zil/slog ssd setup as a write cache, you can stream write pretty fast as it flushes to disk every 5 seconds or so. However the reading from disk can be affected by a few things:
1. if some of the data is in the arc (ram cache)
2. if some of the data is in the l2arc (ssd cache)
3. if it's going to disk, and you are using the disk, the write cache flush can block reads for a moment.

Id suggest copying something into a ramdisk on the Isilon and using that as the source to test everything except the disk subsystem.
Connect further...control easier

With the ATEN CE624, you can now enjoy a high-quality visual experience powered by HDBaseT technology and the convenience of a single Cat6 cable to transmit uncompressed video with zero latency and multi-streaming for dual-view applications where remote access is required.


Author Comment

ID: 40534235
Hi Aaron,

No SSD's all platters....

wondering how I can write to the ram cache?  some zfs commands?  I will google later tonight but would love an example or place to look.  Thanks!
LVL 39

Accepted Solution

Aaron Tomosky earned 500 total points
ID: 40534278
From what I know of building my own FreeBSD and ZFS storage systems, this type of stutter on reads without an SSD is normal. To create a ramdisk for testing, reference this:

One of the benefits of having branded gear is you can call them. I'd suggest asking EMC if this is normal behavior

Edit: Are you running the current version of OneFS?

Generally with performance testing new systems, it's best to isolate the disk from the network and cpu etc... With storage systems there are so many variables you can tweak and so many moving parts involved, isolating the subsystem part that is the bottleneck is paramount.
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 40534882
Thanks for the points, but I'd really like to know what the end result is when you hear back from emc.

Author Comment

ID: 40535742
OK, so you might not like the answer....

So previously I was copying from Isilon to HD on Windows 2012 R2.  I thought maybe the local disk array is causing the slow down?

So I made a RAMDisk on the Windows Machine, I did the copy and it was the same but was unable to get perfmon stats of a 'virtual' drive.

So I then tried to look at an SMB Counter on perfmon, no luck.  Finally tried the Network Adapter, here I saw steady 'bytes received/sec'.  So I thought I would try that again using the disk sub system, still steady bytes/received. (Always in the 400 MB/s, which was the average write MB/sec for the disk array).

So finally to sum it up I made a counter that recorded both the bytes written to disk and bytes received from the 10 gig adapter.  Turns out I still got drops in writes to the disk but maintained a steady reads on the adapter.  When I compared the bytes received vs bytes written to disk over a given period, they were off by about .1 bytes!

My conclusion is that there is a hold up either on the RAID card or WIndows 2012 that waits until there is enough data to write before writing it.  I was comparing this to a WIn 2008 R2 machine (SMB2) so I did the wrong initial test :(

I do appreciate your comments though, it will really help in the future.  I have to say the best help I have received here!
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 40535826
Glad you got it figured out! I had assumed you were doing SMB 2.0 on the same hardware so I didn't even consider ther receiving side hardware.

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This is the first one of a series of articles I’ll be writing to address technical issues that are always referred to as network problems. The network boundaries have changed, therefore having an understanding of how each piece in the network  puzzl…
This article offers some helpful and general tips for safe browsing and online shopping. It offers simple and manageable procedures that help to ensure the safety of one's personal information and the security of any devices.
This video Micro Tutorial explains how to clone a hard drive using a commercial software product for Windows systems called Casper from Future Systems Solutions (FSS). Cloning makes an exact, complete copy of one hard disk drive (HDD) onto another d…
Windows 8 comes with a dramatically different user interface known as Metro. Notably missing from the new interface is a Start button and Start Menu. Many users do not like it, much preferring the interface of earlier versions — Windows 7, Windows X…

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question