Why do these SMB/CIFS transaction repeat over and over?

Posted on 2006-07-21
Last Modified: 2008-01-09
The following trace was taken while accessing a database program over the wire.  In troubleshooting slow performance I noticed that it seems like the same sequence is being repeated over and over.  The following was repeated 23 times before it moved on.  I know that SMB/CIFS is very chatty, but can some one explain why it is behaving this way?  This program was written in DbaseIII and please don't tell me they need to change, this is the program they are going to use.  Also, I have been searching high and low for tips on reading traces, if anyone has a good reference I would be greatful to be pointed in the right direction.

6669      client      SERVER      226       20.84448       20.84902      SMB            NT Create AndX Request, Path: \MCRDSD\RecSchool\x test\Mcwin\MCAIMS.INI
6670      SERVER      client      193       20.84932       20.85385      SMB            NT Create AndX Response, FID: 0x079a
6671      client      SERVER      129       20.85401       20.85853      SMB            Locking AndX Request, FID: 0x079a
6672      SERVER      client      97       20.85931       20.86383      SMB            Locking AndX Response
6673      client      SERVER      130       20.86392       20.86845      SMB            Trans2 Request, QUERY_FILE_INFO, FID: 0x079a, Query File Standard Info
6674      SERVER      client      142       20.86854       20.87306      SMB            Trans2 Response, QUERY_FILE_INFO
6675      client      SERVER      117       20.87317       20.87769      SMB            Read AndX Request, FID: 0x079a, 223 bytes at offset 0
6676      SERVER      client      341       20.87823       20.88279      SMB            Read AndX Response, FID: 0x079a, 223 bytes
6677      client      SERVER      129       20.88292       20.88745      SMB            Locking AndX Request, FID: 0x079a
6678      SERVER      client      97       20.88727       20.89179      SMB            Locking AndX Response
6679      client      SERVER      99       20.89189       20.89641      SMB            Close Request, FID: 0x079a
6680      SERVER      client      93       20.89676       20.90127      SMB            Close Response
6681      client      SERVER      226       20.90157       20.90611      SMB            NT Create AndX Request, Path: \MCRDSD\RecSchool\x test\Mcwin\MCAIMS.INI
6682      SERVER      client      193       20.90623       20.91076      SMB            NT Create AndX Response, FID: 0x079b
6683      client      SERVER      129       20.91091       20.91544      SMB            Locking AndX Request, FID: 0x079b
6684      SERVER      client      97       20.91524       20.91976      SMB            Locking AndX Response
6685      client      SERVER      130       20.91984       20.92436      SMB            Trans2 Request, QUERY_FILE_INFO, FID: 0x079b, Query File Standard Info
6686      SERVER      client      142       20.92423       20.92876      SMB            Trans2 Response, QUERY_FILE_INFO
6687      client      SERVER      117       20.92885       20.93338      SMB            Read AndX Request, FID: 0x079b, 223 bytes at offset 0
6688      SERVER      client      341       20.93295       20.93751      SMB            Read AndX Response, FID: 0x079b, 223 bytes
6689      client      SERVER      129       20.93765       20.94218      SMB            Locking AndX Request, FID: 0x079b
6690      SERVER      client      97       20.94199       20.94651      SMB            Locking AndX Response
6691      client      SERVER      99       20.94659       20.95111      SMB            Close Request, FID: 0x079b
6692      SERVER      client      93       20.95099       20.95551      SMB            Close Response
6693      client      SERVER      226       20.95593       20.96047      SMB            NT Create AndX Request, Path: \MCRDSD\RecSchool\x test\Mcwin\MCAIMS.INI
6694      SERVER      client      193       20.96119       20.96572      SMB            NT Create AndX Response, FID: 0x079c
6695      client      SERVER      129       20.96589       20.97041      SMB            Locking AndX Request, FID: 0x079c
6696      SERVER      client      97       20.97093       20.97545      SMB            Locking AndX Response
6697      client      SERVER      130       20.97552       20.98004      SMB            Trans2 Request, QUERY_FILE_INFO, FID: 0x079c, Query File Standard Info
6698      SERVER      client      142       20.97992       20.98445      SMB            Trans2 Response, QUERY_FILE_INFO
6699      client      SERVER      117       20.98454       20.98907      SMB            Read AndX Request, FID: 0x079c, 223 bytes at offset 0
6700      SERVER      client      341       20.98888       20.99345      SMB            Read AndX Response, FID: 0x079c, 223 bytes
6701      client      SERVER      125       20.99357       20.99809      SMB            Write AndX Request, FID: 0x079c, 3 bytes at offset 93
6702      SERVER      client      105       20.99793       21.00245      SMB            Write AndX Response, FID: 0x079c, 3 bytes
6703      client      SERVER      129       21.00266       21.00718      SMB            Locking AndX Request, FID: 0x079c
6704      SERVER      client      97       21.00668       21.01120      SMB            Locking AndX Response
6705      client      SERVER      99       21.01129       21.01580      SMB            Close Request, FID: 0x079c
6706      SERVER      client      93       21.01714       21.02166      SMB            Close Response
Question by:bb91915
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 6
LVL 57

Expert Comment

ID: 17156806
Does "\MCRDSD\RecSchool\x test\Mcwin\MCAIMS.INI"  mean anything to you?

It is a file that does reside on the file/databse server?  What is in this file?

Offhand I would say that this is sometype of configuration file and either the application or DBaseIII needs to read it mutiple times in order to do the work required.  You said that this was read 23 times, is there something in the application that is done 23 times?  Do you have 23 tables that are being used by the application?

What is "Slow?"  Is it taking hours when people feel it should take minutes or is it taking 10 seconds and they want it to take 2?

Author Comment

ID: 17158637
Yes, the ini file does reside on the file server that is located across a WAN link.  This particular transaction is running at about 20 minutes versus 2 minutes locally.  There are numerous tables contained within this database and I would bet that you are correct about it having to repeat numerous times because of the ini file.  I will see exactly what the ini file contains first thing Monday morning.  Knowing that this type of application was built for LAN use only, I have to show the customer why it runs so slow over a WAN link and thanks to you, I think I have the answer.


LVL 57

Expert Comment

ID: 17158691
What is the speed of the WAN link?  The 3 series of operation you show here only took about .2 seconds.  If we round it up to .3, that is .1 second per "function.  So 23 times would be 2.3 seconds.  I don't think this is causing the problem.  However running a datbase application over a WAN link will be slower than locally, even if the WAN link was a pair of T3's.

I would focus on the volume of data that is being passed over the WAN link, how busy the WAN link is, and the latency.   You might be able to do things like increase the TCP windows size (assuming that this is TCP based).

Tweaking the DB query could cause less data to be returned, thus speeding up the response also.
Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.


Author Comment

ID: 17160331

The WAN link is a DS3 (44mbps) with a 9ms latency (one-way).  We use a 65kb window size and the circuit utilization is well under 20% and running at full duplex.  I am going to focus on the ini file and what it has the application doing and to find out exactly how many tables it is traversing to get the data.   The amount of data changes on what query they are running and of course the more data it is getting the slower it will be.   I will also see if the customer can tweak their queries as well.  Thanks for the input.

LVL 57

Expert Comment

ID: 17161851
Well I would say that you can get much better on the WAN link.  The other question would be how DBaseIII is accessing the databases.  It has been years since I have even heard that name.  But, if it is accessing the databases as a file share on a remote system, that means NETBIOS type file sharing.  Which is slow on any type of connection that has more than about 2-3 ms of latency.  If it is using its own networking functions, like having the SQL execute on a remote server and just having the results returned, then you need to trace how that flow goes.

Look to see if the data returned is one long stream of data or a bunch of smaller stream.  A bunch of smaller streams could cause performance issues no matter how fast the pipe is, if the smaller streams are done serially.  This is because normally each stream is a unique TCP connection, which means connection setup and tear down for each stream.

Author Comment

ID: 17169591
This is all done in one session with a majority of the frames containing less than 100bytes of data.  I also looked at the ini file and it only points to the the default path where the files are located, the location of the temp directory where all work is locally.  So I guess the ini file idea did not pan out.  Looking into the SMB NT Create AndX Request/Responses I found and I quote,
 " A Create and X SMB tells a server to open a designated file and return a file handle.  The request
   includes flags to indicate the type of access required by the client.  Another set of flags is used to
   request an opportunistic lock (oplock).  "Opportunistic" means the server is free to hold off on the
   request unitl it can lock the file.  If the client doesn't free the oplock, the only way to break it is to tear
   down the entire session".
What I need do now is tear apart each frame and see if this is what is happening.  Looking at it from a distance, this does seem to appear what is happening.  I will let you know what I find.

LVL 57

Expert Comment

ID: 17170430
Off hand it appears as if this is setup as though it was file sharing the database instead of a client/server model.

Is a full copy of dBASE III running on the client and accessing the server as a file server?

Or is it using ODBC to talk to the server as a database server?

Author Comment

ID: 17170621
Only the executable is located on the client, the remainder is located on the file server.  As far as if they are using ODBC to talk to the server, I don't know at this point but I will try and find out.  After digging into the traces a bit further, I can see that the client is requesting either an Exclusive or Batch Oplock and the server is responding with "No oplock granted".  If I understand it correctly when the client is  not authorized to place a lock on the file so it has to do the following over and over until it has all of the information:

Opening the command procedure.
Seeking to the next line in the file.
Reading the line from the file.
Closing the file.
Executing the command.

If Oplock is granted the client can go through the process without reestablishing this line of communication.  I got the information from the following link:
LVL 57

Accepted Solution

giltjr earned 250 total points
ID: 17170718
Could be that somebody else has the file locked already.  If multiple people are running the same program at the same time, only one will be able to get exclusive use of it.  If the file is locked while the program is running, then if it takes two minutes to run the program, nobody else can run it for two mintes.  If it takes 20 minutes to run the program, then nobody else can run it for 20 minutes.

Is the MCAIMS.INI file dBASE III related ini file, or an application related any file?  If it is application related you may want to see if there is away to put it on the remote PC and have the application access it locally.

Author Comment

ID: 17170935
I would have to say no to the file being locked by someone else, only one person was accessing it at the time and I know this because no one else knew that we put the files on the server, plus the fact that the file server is not granting an oplock on the file.  I am not sure I understand your question about the ini file, but what I do know is when the application is launched, it reads the ini file to see where to go to get the files.
LVL 57

Expert Comment

ID: 17184202
Thanks for the points.  Is the problem resloved?

I will try and re-ask my question about the ini file.  Is the ini file for the application?  Or for dBASE III?

If for the application, then you might be able to put the file on the comptuer that is running the application.

If it is for dBASEIII then you may need to leave it where dBASEIII is running, however in this case it sounds like dBASEIII may be running on the users compute.

Author Comment

ID: 17189859
Yes and no.  The ini file is for DBase III and only really points the client to the location of the database files.  I believe the problem is with opportunistic locking, but will not know for sure until we enable it on the server if that is possible.  I ran another capture where the server had oplock enable but the difference is, besides the oplocks, is that one server is a windows platform and the other is a paired down version of redhat, meaning it is part of a SAN solution where it basically acts as an appliance and is running a proprietrary system called DART.  I do want to thank you for the help, you  made me look where I wouldn't have looks before asking the question.  Thanks.

Featured Post

Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Workplace bullying has increased with the use of email and social media. Retain evidence of this with email archiving to protect your employees.
Invest in your employees with these five simple steps to improve employee engagement and retention.
The viewer will learn common shortcuts with easy ways to remember them. The viewer will then learn where to find all of the keyboard shortcuts, how to create/change them, and how to speed up their workflow.
This video will demonstrate how to find the puppet warp tool from the edit menu and where to put the points to edit.

732 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question