SMB processes consume 100% of CPU cycles

The linux server has been joined to the ADS for about six months now.  On thursday last week, the Linux server stopped talking to the Windows domain after a reboot.

Now all SMB processes go into an infinite loop when attempting to do any authentication with the server.  This include smb processes from the cups printer daemon.

I have attached the output from the: net -d10 ads join command.

Below is the smb.conf file that has been running for six months:

[global]
        log file = /var/log/samba/%m.log
        load printers = yes
        smb passwd file = /etc/samba/smbpasswd
#       name resolve order = dns wins lmhosts bcast
        idmap gid = 16777216-33554431
        socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
#       wins server = joyce.domain.local
        wins server = 172.24.2.2
        winbind trusted domains only = yes
        encrypt passwords = yes
        realm = DOMAIN.LOCAL
        winbind use default domain = no
        template shell = /bin/false
        dns proxy = no
        netbios name = MONTANA
        cups options = raw
        server string = AddonSoftware Server
        idmap uid = 16777216-33554431
#       password server = joyce.domain.local
        password server = 172.24.2.2
        local master = no
        workgroup = WORKGROUP
        os level = 20
        printcap name = /etc/printcap
        security = ads
        max log size = 50
        mangled names = no
        debug level = 8

Thank you in advance,

Andy

[2008/02/08 15:42:53, 5] lib/debug.c:debug_dump_status(391)
  INFO: Current debug levels:
    all: True/10
    tdb: False/0
    printdrivers: False/0
    lanman: False/0
    smb: False/0
    rpc_parse: False/0
    rpc_srv: False/0
    rpc_cli: False/0
    passdb: False/0
    sam: False/0
    auth: False/0
    winbind: False/0
    vfs: False/0
    idmap: False/0
    quota: False/0
    acls: False/0
    locking: False/0
    msdfs: False/0
    dmapi: False/0
[2008/02/08 15:42:53, 3] param/loadparm.c:lp_load(5033)
  lp_load: refreshing parameters
  lp_load: refreshing parameters
[2008/02/08 15:42:53, 3] param/loadparm.c:init_globals(1424)
  Initialising global parameters
[2008/02/08 15:42:53, 3] param/params.c:pm_process(572)
  params.c:pm_process() - Processing configuration file "/etc/samba/smb.conf"
[2008/02/08 15:42:53, 3] param/loadparm.c:do_section(3772)
  Processing section "[global]"
  doing parameter log file = /var/log/samba/%m.log
  doing parameter load printers = yes
  doing parameter smb passwd file = /etc/samba/smbpasswd
  doing parameter idmap gid = 16777216-33554431
  doing parameter socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
  doing parameter wins server = 172.24.2.2
  doing parameter winbind trusted domains only = yes
  doing parameter encrypt passwords = yes
  doing parameter realm = DOMAIN.LOCAL
  doing parameter winbind use default domain = no
  doing parameter template shell = /bin/false
  doing parameter dns proxy = no
  doing parameter netbios name = MONTANA
[2008/02/08 15:42:53, 4] param/loadparm.c:handle_netbios_name(3129)
  handle_netbios_name: set global_myname to: MONTANA
  doing parameter cups options = raw
  doing parameter server string = AddonSoftware Server
  doing parameter idmap uid = 16777216-33554431
  doing parameter password server = 172.24.2.2
  doing parameter local master = no
  doing parameter workgroup = aaw
  doing parameter os level = 20
  doing parameter printcap name = /etc/printcap
  doing parameter security = ads
  doing parameter max log size = 50
  doing parameter mangled names = no
  doing parameter debug level = 8
[2008/02/08 15:42:53, 4] param/loadparm.c:lp_load(5064)
  pm_process() returned Yes
[2008/02/08 15:42:53, 7] param/loadparm.c:lp_servicenumber(5202)
  lp_servicenumber: couldn't find homes
[2008/02/08 15:42:53, 10] param/loadparm.c:set_server_role(4308)
  set_server_role: role = ROLE_DOMAIN_MEMBER
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset UCS-2LE
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset UCS-2LE
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset UTF-16LE
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset UTF-16LE
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset UCS-2BE
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset UCS-2BE
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset UTF-16BE
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset UTF-16BE
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset UTF8
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset UTF8
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset UTF-8
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset UTF-8
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset ASCII
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset ASCII
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset 646
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset 646
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset ISO-8859-1
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset ISO-8859-1
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(105)
  Attempting to register new charset UCS2-HEX
[2008/02/08 15:42:53, 5] lib/iconv.c:smb_register_charset(113)
  Registered charset UCS2-HEX
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/charcnv.c:charset_name(82)
  Substituting charset 'UTF-8' for LOCALE
[2008/02/08 15:42:53, 5] lib/util.c:init_names(309)
  Netbios name list:-
  my_netbios_names[0]="MONTANA"
[2008/02/08 15:42:53, 2] lib/interface.c:add_interface(81)
  added interface ip=172.24.2.6 bcast=172.24.2.255 nmask=255.255.255.0
[2008/02/08 15:42:53, 5] lib/gencache.c:gencache_init(61)
  Opening cache file at /var/cache/samba/gencache.tdb
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(226)
  Returning valid cache entry: key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL, value
= Default-First-Site-Name, timeout = Mon Jan 18 21:14:07 2038
[2008/02/08 15:42:53, 5] libads/dns.c:sitename_fetch(679)
  sitename_fetch: Returning sitename for DOMAIN.LOCAL: "Default-First-Site-N
ame"
[2008/02/08 15:42:53, 4] libsmb/namequery_dc.c:ads_dc_name(73)
  ads_dc_name: domain=AAW
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(226)
  Returning valid cache entry: key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL, value
= Default-First-Site-Name, timeout = Mon Jan 18 21:14:07 2038
[2008/02/08 15:42:53, 5] libads/dns.c:sitename_fetch(679)
  sitename_fetch: Returning sitename for DOMAIN.LOCAL: "Default-First-Site-N
ame"
[2008/02/08 15:42:53, 6] libads/ldap.c:ads_find_dc(294)
  ads_find_dc: looking for realm 'DOMAIN.LOCAL'
[2008/02/08 15:42:53, 8] libsmb/namequery.c:get_sorted_dc_list(1626)
  get_sorted_dc_list: attempting lookup for name DOMAIN.LOCAL (sitename Defa
ult-First-Site-Name) using [ads]
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(212)
  Cache entry with key = SAF/DOMAIN/DOMAIN.LOCAL couldn't be found
[2008/02/08 15:42:53, 5] libsmb/namequery.c:saf_fetch(133)
  saf_fetch: failed to find server for "DOMAIN.LOCAL" domain
[2008/02/08 15:42:53, 3] libsmb/namequery.c:get_dc_list(1489)
  get_dc_list: preferred server list: ", 172.24.2.2"
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(226)
  Returning valid cache entry: key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL, value
= Default-First-Site-Name, timeout = Mon Jan 18 21:14:07 2038
[2008/02/08 15:42:53, 5] libads/dns.c:sitename_fetch(679)
  sitename_fetch: Returning sitename for DOMAIN.LOCAL: "Default-First-Site-N
ame"
[2008/02/08 15:42:53, 10] libsmb/namequery.c:remove_duplicate_addrs2(435)
  remove_duplicate_addrs2: looking for duplicate address/port pairs
[2008/02/08 15:42:53, 4] libsmb/namequery.c:get_dc_list(1599)
  get_dc_list: returning 1 ip addresses in an ordered list
[2008/02/08 15:42:53, 4] libsmb/namequery.c:get_dc_list(1600)
  get_dc_list: 172.24.2.2:389
[2008/02/08 15:42:53, 5] libads/ldap.c:ads_try_connect(180)
  ads_try_connect: sending CLDAP request to 172.24.2.2 (realm: DOMAIN.LOCAL)
[2008/02/08 15:42:53, 10] libads/dns.c:sitename_store(640)
  sitename_store: realm = [DOMAIN.LOCAL], sitename = [Default-First-Site-Nam
e], expire = [2147483647]
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_set(140)
  Adding cache entry with key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL; value = Def
ault-First-Site-Name and timeout = Mon Jan 18 21:14:07 2038
   (944976674 seconds ahead)
[2008/02/08 15:42:53, 3] libads/ldap.c:ads_connect(394)
  Connected to LDAP server 172.24.2.2
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(226)
  Returning valid cache entry: key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL, value
= Default-First-Site-Name, timeout = Mon Jan 18 21:14:07 2038
[2008/02/08 15:42:53, 5] libads/dns.c:sitename_fetch(679)
  sitename_fetch: Returning sitename for DOMAIN.LOCAL: "Default-First-Site-N
ame"
[2008/02/08 15:42:53, 10] libads/ldap.c:ads_closest_dc(149)
  ads_closest_dc: ADS_CLOSEST flag set
[2008/02/08 15:42:53, 10] libads/kerberos.c:create_local_private_krb5_conf_for_d
omain(612)
  create_local_private_krb5_conf_for_domain: fname = /var/cache/samba/smb_krb5/k
rb5.conf.AAW, realm = DOMAIN.LOCAL, domain = AAW
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(212)
  Cache entry with key = SAF/DOMAIN/DOMAIN.LOCAL couldn't be found
[2008/02/08 15:42:53, 5] libsmb/namequery.c:saf_fetch(133)
  saf_fetch: failed to find server for "DOMAIN.LOCAL" domain
[2008/02/08 15:42:53, 3] libsmb/namequery.c:get_dc_list(1489)
  get_dc_list: preferred server list: ", 172.24.2.2"
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(226)
  Returning valid cache entry: key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL, value
= Default-First-Site-Name, timeout = Mon Jan 18 21:14:07 2038
[2008/02/08 15:42:53, 5] libads/dns.c:sitename_fetch(679)
  sitename_fetch: Returning sitename for DOMAIN.LOCAL: "Default-First-Site-N
ame"
[2008/02/08 15:42:53, 10] libsmb/namequery.c:remove_duplicate_addrs2(435)
[2008/02/08 15:42:53, 10] libsmb/namequery.c:remove_duplicate_addrs2(435)
  remove_duplicate_addrs2: looking for duplicate address/port pairs
[2008/02/08 15:42:53, 4] libsmb/namequery.c:get_dc_list(1599)
  get_dc_list: returning 1 ip addresses in an ordered list
[2008/02/08 15:42:53, 4] libsmb/namequery.c:get_dc_list(1600)
  get_dc_list: 172.24.2.2:389
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(212)
  Cache entry with key = SAF/DOMAIN/DOMAIN.LOCAL couldn't be found
[2008/02/08 15:42:53, 5] libsmb/namequery.c:saf_fetch(133)
  saf_fetch: failed to find server for "DOMAIN.LOCAL" domain
[2008/02/08 15:42:53, 3] libsmb/namequery.c:get_dc_list(1489)
  get_dc_list: preferred server list: ", 172.24.2.2"
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(226)
  Returning valid cache entry: key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL, value
= Default-First-Site-Name, timeout = Mon Jan 18 21:14:07 2038
[2008/02/08 15:42:53, 5] libads/dns.c:sitename_fetch(679)
  sitename_fetch: Returning sitename for DOMAIN.LOCAL: "Default-First-Site-N
ame"
[2008/02/08 15:42:53, 10] libsmb/namequery.c:remove_duplicate_addrs2(435)
  remove_duplicate_addrs2: looking for duplicate address/port pairs
[2008/02/08 15:42:53, 4] libsmb/namequery.c:get_dc_list(1599)
  get_dc_list: returning 1 ip addresses in an ordered list
[2008/02/08 15:42:53, 4] libsmb/namequery.c:get_dc_list(1600)
  get_dc_list: 172.24.2.2:389
[2008/02/08 15:42:53, 10] libads/kerberos.c:get_kdc_ip_string(563)
  get_kdc_ip_string: Returning  kdc = 172.24.2.2
 
[2008/02/08 15:42:53, 5] libads/kerberos.c:create_local_private_krb5_conf_for_do
main(681)
  create_local_private_krb5_conf_for_domain: wrote file /var/cache/samba/smb_krb
5/krb5.conf.AAW with realm DOMAIN.LOCAL KDC = 172.24.2.2
[2008/02/08 15:42:53, 4] libsmb/namequery_dc.c:ads_dc_name(131)
  ads_dc_name: using server='JOYCE.DOMAIN.LOCAL' IP=172.24.2.2
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(226)
  Returning valid cache entry: key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL, value
= Default-First-Site-Name, timeout = Mon Jan 18 21:14:07 2038
[2008/02/08 15:42:53, 5] libads/dns.c:sitename_fetch(679)
  sitename_fetch: Returning sitename for DOMAIN.LOCAL: "Default-First-Site-N
ame"
[2008/02/08 15:42:53, 6] libads/ldap.c:ads_find_dc(294)
  ads_find_dc: looking for realm 'DOMAIN.LOCAL'
[2008/02/08 15:42:53, 8] libsmb/namequery.c:get_sorted_dc_list(1626)
  get_sorted_dc_list: attempting lookup for name DOMAIN.LOCAL (sitename Defa
ult-First-Site-Name) using [ads]
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(212)
  Cache entry with key = SAF/DOMAIN/DOMAIN.LOCAL couldn't be found
[2008/02/08 15:42:53, 5] libsmb/namequery.c:saf_fetch(133)
  saf_fetch: failed to find server for "DOMAIN.LOCAL" domain
[2008/02/08 15:42:53, 3] libsmb/namequery.c:get_dc_list(1489)
  get_dc_list: preferred server list: ", 172.24.2.2"
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_get(226)
  Returning valid cache entry: key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL, value
= Default-First-Site-Name, timeout = Mon Jan 18 21:14:07 2038
[2008/02/08 15:42:53, 5] libads/dns.c:sitename_fetch(679)
  sitename_fetch: Returning sitename for DOMAIN.LOCAL: "Default-First-Site-N
ame"
[2008/02/08 15:42:53, 10] libsmb/namequery.c:remove_duplicate_addrs2(435)
  remove_duplicate_addrs2: looking for duplicate address/port pairs
[2008/02/08 15:42:53, 4] libsmb/namequery.c:get_dc_list(1599)
  get_dc_list: returning 1 ip addresses in an ordered list
[2008/02/08 15:42:53, 4] libsmb/namequery.c:get_dc_list(1600)
  get_dc_list: 172.24.2.2:389
[2008/02/08 15:42:53, 5] libads/ldap.c:ads_try_connect(180)
  ads_try_connect: sending CLDAP request to 172.24.2.2 (realm: DOMAIN.LOCAL)
[2008/02/08 15:42:53, 10] libads/dns.c:sitename_store(640)
  sitename_store: realm = [DOMAIN.LOCAL], sitename = [Default-First-Site-Nam
e], expire = [2147483647]
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_set(140)
  Adding cache entry with key = AD_SITENAME/DOMAIN/DOMAIN.LOCAL; value = Def
ault-First-Site-Name and timeout = Mon Jan 18 21:14:07 2038
   (944976674 seconds ahead)
[2008/02/08 15:42:53, 3] libads/ldap.c:ads_connect(394)
  Connected to LDAP server 172.24.2.2
[2008/02/08 15:42:53, 10] libads/ldap.c:ads_closest_dc(149)
  ads_closest_dc: ADS_CLOSEST flag set
[2008/02/08 15:42:53, 10] libsmb/namequery.c:saf_store(74)
  saf_store: domain = [AAW], server = [172.24.2.2], expire = [1202507873]
[2008/02/08 15:42:53, 10] lib/gencache.c:gencache_set(140)
  Adding cache entry with key = SAF/DOMAIN/AAW; value = 172.24.2.2 and timeout =
 Fri Feb  8 15:57:53 2008
   (900 seconds ahead)

Open in new window

dlaincAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Pétur Ingi EgilssonSoftware Engineer -- ConsultantCommented:
Could this possibly be a problem on the windows side?
0
NopiusCommented:
Ensure that your /etc/resolv.conf file has not been changed since initial samba configuration. You should use MS DNS as a nameserver.

As root user:
try to preserve /var/cache/samba/gencache.tdb
then, stop samba service, remove this file, start samba again,
try to authenticate against your AD with
kinit Administrator@DOMAIN.LOCAL
then try to join with 'net ads join'
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
dlaincAuthor Commented:
The resolv.conf file had not been changed when this problem occurred.

I am using the MS DNS as the DNS.
kinit Administrator@DOMAIN.LOCAL works properly

Apparently deleting the gencache.tdb, redoing the kinit and doint the net ads join has resolved the problem.

Thank you for your assistance.

Andy
0
dlaincAuthor Commented:
Do you know what sort of event could corrupt the gencache.tdb file?
0
NopiusCommented:
Thank you for points, dlianc.

> Do you know what sort of event could corrupt the gencache.tdb?

It may be incorrect server or Samba process shutdown. Also it may be due to incorrect usage of this file by some samba processes (say concurrent write without a lock).
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Server Software

From novice to tech pro — start learning today.