Problem with Samba share configuration slowing down our source control program

I tried your suggestion.

I did not disable logging, because I have found not entries in either the messages or secure log during these delay times. I did find these entries in /var/lob/samba/log.smbd:

[2009/11/04 13:35:52, 1] smbd/service.c:close_cnum(1230)
  192.168.101.63 (192.168.101.63) closed connection to service sysroot
[2009/11/04 13:35:53, 0] smbd/server.c:main(944)
  smbd version 3.0.33-3.7.el5 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2008
[2009/11/04 13:36:33, 1] smbd/service.c:make_connection_snum(1033)
  192.168.101.61 (192.168.101.61) connect to service sysroot initially as user nobody (uid=99, gid=99) (pid 29840)
[2009/11/04 13:49:30, 1] smbd/service.c:close_cnum(1230)
  192.168.101.61 (192.168.101.61) closed connection to service sysroot
[2009/11/04 13:49:30, 0] smbd/server.c:main(944)
  smbd version 3.0.33-3.7.el5 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2008
[2009/11/04 13:50:09, 1] auth/auth_util.c:create_token_from_username(1156)
  lookup_name_smbconf for guest failed

After I made the changes you suggested (except the turning off logging, which I reserve for a last resort...)
I stopped Accurev and restarted it and I got an error popup that looked like the attached picture. Previously, I just had long delays after initial startup, the pop-up error is new. Once I click OK on the pop-up, things still work, but with the same long delay.

One thing that I have consistently had problems with is browsing to a Samba share by name.  
If I want to look at a file on /etc/samba/ for instance, on a the Linux samba share 'sysroot' (see the code snippet, where I pasted my smb.conf file). I can do it two ways:

1. browse to workgroup Linux click on ASSETx and then to \sysroot\etc\samba\
OR
2. browse directly to \\192.168.101.65\sysroot\etc\samba\

Method 1 ALWAYS takes a long time and often results in the browser pane getting hung up and having to be killed. I usually never even get to see the ASSETx object.

Method two (which is how I do it in Accurev) always works instantly and I've been using this samba conf file with minor variations on a dozen Linux boxes here every day for 4 years with no issues at all.  Using this method, file copies, editor reads and writes, BeyondCompare, AraxisMerge, etc. ALL work flawlessly.

But with this one application, Accurev, I have these extremely long delays.

I am behind a really secure firewall, so I have all the Linux security disabled and always run as root. They are  strictly engineering development platforms.  I wrestled with Samba and Windows ADS for weeks until it made me insane, so I removed all the security s**t and opened the shares up to the public and since then, everything has run smoothly (until Accurev).

So I wonder if I should make all the files in my primary share owned by root and set up smb.conf share to force user = root, rather than guest?

I'm going to try that and see what happens.

        workgroup = LINUX
        netbios name = ASSETx
        server string = Samba Server
        security = SHARE
        socket options = TCP_NODELAY SO_RCVBUF=16384 SO_SNDBUF=16384
        hosts allow = 192.168. 
[homes]
        comment = Home Directories
        valid users = %S
        read only = No
        create mask = 0666
        directory mask = 0775 
[printers]
        comment = All Printers
        path = /var/spool/samba
        printable = Yes
        browseable = No 
[source]
        path = /source
        read only = No
        guest ok = Yes 
[data]
        path = /data
        read only = No
        guest ok = Yes 
[datalogs]
        path = /datalogs
        read only = No
        guest ok = Yes 
[root]
        path = /root
        read only = No
        guest ok = Yes 
[sysroot]
        path = /
        read only = No
        guest ok = Yes
        force user = guest 
[development]
        path = /development
        read only = No
        guest ok = Yes 
[beacon]
        path = /beacon
        read only = No
        guest ok = Yes 
[workspaces]
        path = /workspaces
        read only = No
        guest ok = Yes
                                              

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:
43:
44:
45:
46:
47:
48:
49:
50:
51:
52:
53:
54:
55:
56:
57:
58:
59:
60:

The issue won't be with the OS, and I suspect it isn't even with Samba... I still maintain the issue is a networking timeout of some type.

I won't go into the problems with sharing / as sysroot -- I assume you're not stupid, just "trying everything"

Try adding the following lines to your [global] section:

        smb ports = 445 139
        name resolve order = host wins bcast

This will force Samba to listen on (and prefer communications on) port 445 over the older port 139.
Then it will resolve host named based on your /etc/hosts file first -- and you can easily dummy up entries for your entire LAN to make sure lookups (rather, reverse lookups) are uber-fast!

Awaiting results....

Dan
IT4SOHO

Dan, that seems to have helped a little.
I think the slowdown is reduced to something closer to 20 or 30 times slower, rather than 75 times slower!

Got any more ideas? I think that the slowdown should be 4 or 5 times slower, which would be manageable, and is something like the slowdown I'd expect based on what we've seen for doing other things, like moving gigabytes of files across the Samba transport layer versus from one Windows folder to another.

I'll try setting up the hosts file and get back to you.

Thanks a bunch - Frank

Strangely, the gains I thought we picked up yesterday have mysteriously vanished.
I'm back to 70 times slower :-)
I thought your suggestions were great and should have worked, but somehow yesterday must have been an exceptional coincidence.

Frank....

Let's try to clear something up here... understand that there is a different set of problems
 a) setting up the TCP connection, and
 b) using (transferring data) on that connection.

I've been assuming that the issue is in the first part -- setting up the connection. Perhaps the issue is in part b, the actual transfer of the data. This would lead me to look at the NIC card itself (I know, for example, that some older linux drivers for some of the realtek chipsets could be very slow), or the infrastructure (as in, the switch/router that is connecting the devices (in this instance, I've had situations where clients try to send fast data through the router's switch  -- in other words, file transfers on the LAN that route through the router -- that have had this type of a problem. Most "small business" routers are not designed for high speed data throughput... they're routers, not switches. In my case, replacing the router with a "real" switch with some decent throughput capabilities solved our problem).

This is a rather curious issue... makes one think!!!

Dan
IT4SOHO

I don't think transferring the data is the problem. I have 4 years of data with a dozen nodes that have run across different distros of Linux (RH 9, Fedora 3,4,5 and 6 and CentOS 5.0 and 5.3) Different kernels (2.4 and 2.6)

And the data transfer rate of files (many terabytes have been transferred by this time, at the rate of several hundred megabytes per day) between Linux (on about 12 different nodes every day) to and from a Microsoft Exchange based server with some slowdown relative to the speed of transferring within the same computer, but the slowdown between Linux and MS Exchange Server is no slower that between My PC and the Exchange server. Of course we pay a time penalty for using Ethernet instead of SATA, but NOT a factor of 70+

In every case, the data path is the same regardless of whether I am using Microsoft File Copy, BeyondCompare, scp, tftp, ftp, telnet or whatever...

The only place we've seen this slowdown is with this particular database application (Accurev). Nowhere else have we seen this. I can't understand how this could possibly be using a different set of routings than the other applications. I will ask my IT manager again, but he assures me it's the same.

The same NICs were in use for all these applications. The same protocol settings. Everything the same.

I really liked and appreciated the suggestions you made before, I think they were on the right track. It seemed that one day when I tried it i saw a measurable improvement, but the next day that improvement vanished. I just need to know where to look to try and figure out where the other network configuration items are that might be fruitful places to look.

I really think that a likely place to look is this question:
Why is it impossible for me to use named file servers with Samba? Why is the only way I can resolve node identities by using raqw IP addresses? That has consistently been a problem and I'd just bet that part of the problem I am seeing with Accurev is the result of attempts within that database to use named server nodes on the Linux workgroup. I just don't know how to probe this part of our system to see why this is a problem or to determine if that is what Accurev is inf fact doing.

Thanks again. I appreciate your input and would like to continue the discussion.

Ditch Accurev, and get VersionOne! Sometimes you have to measure value and performance with the Oh so important "help"! you get from customer service.   It sounds to me that Accurev sat on those hands to help you! Did they even offer to get their technical grade programmers involved in overcoming our issue?   It sounds like some, "front" user/salesman who doesn't know Squat, says.   "Too bad, I got paid, so go away"!   I'd be pissed!  Get accurev on the phone, and demand for them to get you in touch with thier "real" programmers!    Because my guess is: that database is attempting to use named server nodes on the Linux workgroup!     That's why you got the glitch/improvement because when you first attempted to resolve it, the "cache" of your page ignored the inital refresh.   That's why you saw an improvement!    Get Accurev on the phone again, and demand a "real" programmer to help configure Accurev's stream browser S@%T to disallow it's behavior.   Bottom line.  Make them Bite Back!

I actually think you may like this solution better:

http://www.axosoft.com/ontime/try_ontime_free

aijreal:
Thanks for hanging in there with me.
We are not going to change from Accurev.
We invested a lot in the learning curve and really like the stream-based architecture.
I used to be a ClearCase user, and trust me on this one, ClearCase is WAY more difficult to use than any of the competition.
But still, migration to another system, after we have a huge source repository --> we just aren't going to go there.
This review says it all.
http://www.bxgrant.com/entry/14
He talks about the problems with Accurev and contrasts it with subversion. He is spot on in everything he says.
We have also made good use of the Accuwork tool that is packaged with Accurev. It allows us to track bugs in a very powerful way.
But this Windows <--> Samba <--> Linux connection is simply problematic.
It is problematic not only because of Accurev, but just using named Linux shares through a Windows explorer is a complete bust. I am thoroughly convinced that these two problems are connected and that Windows is the culprit. If Accurev is using named shares, we are sunk.

I am going to try going at this a different way. Perhaps I will try to mount a shared file area on my laptop from Linux.
Then I don't have to host the workspaces on Linux, I can leave them on XP and access them from Linux when I am compiling.

Thanks - Frank