Solved

Mac OSX Spotlight indexing takes forever

Posted on 2010-09-14
34
4,196 Views
Last Modified: 2013-11-08
Hi,

I have a user running OSX 10.6.4 with a remote home directory on my windows server.  Every time she logs into her machine, the spotlight search will reindex her files and it takes about 5 hours which means she cannot use it for most of the day.  Her user profile is very large, about 40gb.  Is it just the sheer size of her profile that makes it take so long?  I've tried excluding her 20gb of music from the spotlight search, but there is still about 10gb of email and 10gb of files that take forever to index.  Any ideas?

0
Comment
Question by:tferro999
  • 14
  • 11
  • 8
  • +1
34 Comments
 
LVL 21

Expert Comment

by:chapmanjw
ID: 33674747
Since OS X is not in control of the remote file share, it has to look at every single file to look for changes.  If you need spotlight search enabled for the remote files, you might be better using synchronization rather than connecting remotely. (Like with Windows Live Sync, http://sync.live.com).  This way, OS X is in control of the files and Spotlight doesn't have to look at every single file during the index. Otherwise, just take the home directory out of the spotlight index.
0
 

Author Comment

by:tferro999
ID: 33675197
Is there any chance that this can be caused by corrupt files.  I read that if your logs show errors for the mdworker process, they recommend to delete the offending files.  My log indeed shows a ton of errors like this...

mdworker[2989] Invalid char for PropertyName in line 18

I have no clue how to find the file it is referring to in line 18.
0
 
LVL 53

Expert Comment

by:strung
ID: 33675478
It sounds like the spotlight index is corrupt. I suggest you delete it and let OS X rebuild it.

See:  http://www.macintouch.com/tigerreview/spotlight.html#fixes
0
 
LVL 53

Expert Comment

by:strung
ID: 33675488
0
 
LVL 53

Expert Comment

by:strung
ID: 33675494
0
 
LVL 53

Expert Comment

by:strung
ID: 33676368
0
 
LVL 53

Expert Comment

by:strung
ID: 33676379
Actually, I am surprised it is indexing a remote Windows volume.  I had understood that Spotlight worked only on Mac volumes.
0
 
LVL 12

Expert Comment

by:nxnw
ID: 33678422
Let me preface the following by a caution that I know nothing about AD. With that out of the way:

1. If this user is always on the same machine (you said this is the only mac on the network), it might be best to set up a local account on her machine.

Pretty sure that spotlight will, by default, index network homes (but not other remote volumes unless you use coercion and trickery), Even with network homes under under OS X server — which has server side spotlight indexing — there are bottlenecks created by caches being written to and read from a remote server. There are ways to redirect the cache files to the local disk, but I understand that it can be an adventure if you are running Office Mac applications. Setting up the user as a mobile account (I know nothing about AD and am not sure if it can be done in your setting, though) would be better.

A further alternative (can be done with OD and, I suspect also with AD) would be to configure the network user to have a home folder at /Users/

This enables the network user to have a home directory in the same place as would be the case for local account - on the local hard drive in the /Users/ folder. To do this with OS X server, after configuring the network user with Workgroup Manager, you copy the home directory into the local drive in a way that maintains the privileges (network user is owner, etc), then log in as before with the network username and password.
0
 

Author Comment

by:tferro999
ID: 33687068
I've tried rebuilding the index and it still re-indexes after each login.

In response to nxnw's suggestions...

Security audits mandate that these home directories must not be stored on the local machine so I can't go with local profiles.  At the moment i'm experimenting with nhr.dmg to redirect the local caches.  I'll let you guys know if it helps.
0
 
LVL 12

Expert Comment

by:nxnw
ID: 33687457
OK, some other thoughts, then:

Out of an abundance of caution, how did you go about rebuilding the suspect index? I'm not sure the usual utilities will get the index for a network home, because it is kept in a different place (/var/db/spotlight/ I think).

Also, have you tried making the network home private (in system preferences/spotlight) as a temporary workaround to stop the endless indexing?

Finally, troublemaker that I am: If a local home directory, that could be encrypted, violates security audit standards, how about unencrypted local cache files and spotlight indexes?
0
 
LVL 12

Expert Comment

by:nxnw
ID: 33687471
Other possibilities:
- a corrupt file that spotlight is trying to index (would likely show up in system logs)
- a privilege issue with the directory containing the index
0
 

Author Comment

by:tferro999
ID: 33692788
The user and creator/owner have full control over the .spotlight-v100 dir so I dont think it's that unless there is some other group/user i'm missing.

How would I go about finding the corrupt file that the index is choking on.  All it tells me is ..

mdworker[2989] Invalid char for PropertyName in line 18

without any real file name or location.  Is there a way I can track it down?  I've ran repair permissions and repair disk without any luck.
0
 
LVL 53

Expert Comment

by:strung
ID: 33692925
0
 
LVL 53

Expert Comment

by:strung
ID: 33692930
0
 

Author Comment

by:tferro999
ID: 33692959
Responding to your earlier post.

1.) I rebuilt the index by logging in as the user and running
sudo mdutil -E /

I'm getting permission denied when trying to browse the /var/db locations, will try in single user mode

2.) Since the users are adamant about using the spotlight, i've just had them use a password protected screen saver instead of logging off

3.) I do realize that one could get some info from the local cache files.  I would prefer an encrypted local profile and remote file shares for ease of backup but i'm just following orders whether they make sense or not. :)
0
 
LVL 53

Expert Comment

by:strung
ID: 33693054
If it turns out the problem is the .ds store files, here is how to stop the Mac from creating them on the network volume:

http://hints.macworld.com/article.php?story=2005070300463515
0
 
LVL 12

Expert Comment

by:nxnw
ID: 33693294
"The user and creator/owner have full control over the .spotlight-v100 dir "

That isn't where the index is. It is in /var/db/

You can get at it in terminal by opening a session as a superuser with the command:

sudo bash

If rebuilding the index doesn't fix it, the reference provided by Strung in  33692925 looks really good.

0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 

Author Comment

by:tferro999
ID: 33702968
I've tried rebuilding that index under /var/db and it doesn't help.  The network accounts are still reindexing the entire profile each time they login.

The full command
lsof | grep mdimport | grep /Users/`whoami`
doesnt do anything for me, but "lsof | grep mdimport " reports the following


mdworker  229 tony3  txt      REG       14,2     51136     1168897 /System/Library/Spotlight/Application.mdimporter/Contents/MacOS/Application
mdworker  229 tony3  txt      REG       14,2     84352      778011 /System/Library/Spotlight/Image.mdimporter/Contents/MacOS/Image
mdworker  229 tony3  txt      REG       14,2     61632      830265 /System/Library/Spotlight/iCal.mdimporter/Contents/MacOS/iCal
mdworker  229 tony3  txt      REG       14,2     60320     1138787 /System/Library/Spotlight/Audio.mdimporter/Contents/MacOS/Audio
mdworker  229 tony3  txt      REG       14,2     50560      777637 /System/Library/Spotlight/Archives.mdimporter/Contents/MacOS/Archives
mdworker  229 tony3  txt      REG       14,2     51696      778020 /System/Library/Spotlight/PDF.mdimporter/Contents/MacOS/PDF
mdworker  229 tony3  txt      REG       14,2    145072     1133194 /System/Library/Spotlight/RichText.mdimporter/Contents/MacOS/RichText
mdworker  229 tony3  txt      REG       14,2     95552     1133774 /System/Library/Spotlight/vCard.mdimporter/Contents/MacOS/vCard
mdworker  229 tony3  txt      REG       14,2     77200     1146905 /System/Library/Spotlight/Mail.mdimporter/Contents/MacOS/Mail
mdworker3 234 tony3  txt      REG       14,2   2894560      790256 /Library/Spotlight/Microsoft Office.mdimporter/Contents/MacOS/Microsoft Office
mdworker3 234 tony3  txt      REG       14,2   2360240      790247 /Library/Spotlight/Microsoft Office.mdimporter/Contents/Frameworks/MetroFramework.framework/Versions/12/MetroFramework
mdworker3 234 tony3    9r     REG       14,2     30450      790258 /Library/Spotlight/Microsoft Office.mdimporter/Contents/Resources/Microsoft Office.rsrc

The logs create right after login and after the rebuilding begins are as follows


9/17/10 8:38:33 AM      mds[29]      (Normal) DiskStore: Rebuilding index for /Network/Servers/xyz.edu/macprofiles$/tony3
9/17/10 8:38:33 AM      mds[29]      (Normal) DiskStore: Creating index for /Network/Servers/xyz.edu/macprofiles$/tony3
9/17/10 8:39:35 AM      mdworker32[321]      kCGErrorFailure: Set a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged.
9/17/10 8:39:41 AM      mdworker[320]      socket(PF_ROUTE) failed: Operation not permitted
9/17/10 8:39:41 AM      sandboxd[324]      mdworker(320) deny system-socket
9/17/10 8:39:41 AM      sandboxd[324]      mdworker(320) deny network-outbound /private/var/run/mDNSResponder
9/17/10 8:39:41 AM      sandboxd[324]      mdworker(320) deny network-outbound /private/var/run/mDNSResponder
9/17/10 8:39:41 AM      sandboxd[324]      mdworker(320) deny network-outbound /private/var/run/mDNSResponder
9/17/10 8:39:41 AM      sandboxd[324]      mdworker(320) deny network-outbound /private/var/run/mDNSResponder
9/17/10 8:39:41 AM      sandboxd[324]      mdworker(320) deny network-outbound /private/var/run/mDNSResponder
9/17/10 8:39:41 AM      sandboxd[324]      mdworker(320) deny network-outbound /private/var/run/mDNSResponder
9/17/10 8:39:41 AM      mdworker[320]      socket(PF_ROUTE) failed: Operation not permitted
9/17/10 8:39:41 AM      sandboxd[324]      mdworker(320) deny system-socket
0
 
LVL 12

Expert Comment

by:nxnw
ID: 33703490
The first group from the lsof command are all local. Not relevant.

The second group, I'm pretty sure, means that mdworker is trying to index the network home directory, but the internal firewall is blocking it. If that is correct, the user may be able to use spotlicht, but his searches will not find documents added to his home directory since the problem started.

I'm not sure if turning off the internal firewall (system preferences/security) will completely turn off sandboxd, but it's worth exploring (on the path to a solution)
0
 

Author Comment

by:tferro999
ID: 33705395
The firewall was already turned off.  Not sure where to go from here.
0
 
LVL 12

Expert Comment

by:nxnw
ID: 33709871
I think sandboxd does more than just the firewall, and the logs do indicate that it is blocking network access to spotlight indexing.

Three shots in the dark,
- run the 10.6.4 combo installer, in case it is being caused by system corruption of some kind;
- deep clean your caches, in case it is being caused by corrupt caches;
- turn off Time Machine.
0
 

Author Comment

by:tferro999
ID: 33726299
Time machine was off, the Onyx cache cleaning didnt help, and it can't be system corruption because its happening on 13 different machines.

I'm waiting on a response from some of the Apple and MS forums.
0
 
LVL 12

Expert Comment

by:nxnw
ID: 33726730
Ahh. I was thrown off by "I have a user running OSX 10.6.4 with a remote home directory" and have been assuming it was a single machine. Almost all of the suggestions provided above would be very unlikely in a scenario of 13 different machines. The fact that this was happening on multiple machines was a critical piece of information.

In the circumstances, please provide more info:

• [b]can you confirm that recent changes are not being indexed[b] (which would be the case if I am interpreting the log entry correctly)?

• [b]Are all of the affected macs running 10.6.4? 10.6.x?[b]

• [b]Are the 13 affected machines all of your macs?[b]

I suspect that your problem is due to a bug in the OS. The above may confirm this and possibly suggest a workaround. If it is a bug, it should be reported to Apple.

Finally, if the actual problem is — in fact — that spotlight is failing to index OS X network homes on a windows server, your problem should be described accordingly. A more accurate description will help get you a solution.
0
 

Author Comment

by:tferro999
ID: 33727061
Sorry for not being more descriptive, after entering this problem on several forums things get a little blurry :)

I'm pretty sure the recent changes are actually being indexed, but i'll verify.  My test account is only 3gb or so and I let it completely index and the spotlight was able to find an email that came in that day.  Give me 30 min and i'll try this again.

All the affected macs are running 10.6.4, and the 13 macs are my entire test population at this point (If I can get this stupid thing fixed, i've got 10 more to add)
0
 

Author Comment

by:tferro999
ID: 33727880
I have confirmed that recent changes are being indexed by spotlight.  I'm able to find text from emails sent this morning after logging in and waiting around 45 min to index everything.

The issue that remains is why it insists on re-indexing everything every time time the user logs in.
0
 
LVL 12

Expert Comment

by:nxnw
ID: 33728103
And that is the same machine that is getting these errors in the log?

mdworker(320) deny network-outbound /private/var/run/mDNSResponder
mdworker[320]      socket(PF_ROUTE) failed: Operation not permitted
sandboxd[324]      mdworker(320) deny system-socket

By the way, how does the mac know where to look for the network homes? Do you login to the network homes using a FQDN? Are the correct local DNS servers shown in network settings?
0
 

Author Comment

by:tferro999
ID: 33728256
The macbook back in my office gave me those errors.  The clients in a different building were similar.  I can provide some of their specific log errors tomorrow.  In Active Directory, the user account has a tab for "profiles". You can specify the home folder to connect to a network volume.

My test account uses...
\\servername\macprofiles$\tony3

The actual users have...
\\servername.xxx.edu\macprofiles$\tony3
The problem exists no matter if I use the FQDN or not.




0
 
LVL 12

Expert Comment

by:nxnw
ID: 33728466
Can you use \\servername.xxx.edu\macprofiles$\tony3 and see if you get the same errors? I was wondering about DNS because of this error:
"mdworker(320) deny network-outbound /private/var/run/mDNSResponder"
which indicates that mdworker was trying to use bonjour (aka mDNSResponder)

It would be good to ensure that the errors are not a red herring.

Also, I am assuming that your test machine is also having the same problems with a lengthy period of poor performance while spotlight appears to be indexing. Please advise if this assumption is wrong.
0
 

Author Comment

by:tferro999
ID: 33843255
I've had good luck with using a brand new profile and not importing the old one, only the data itself.  I'll give it a few more days to test it before posting the results.

The indexing seems to go very quickly.
0
 
LVL 12

Expert Comment

by:nxnw
ID: 34450781
So what was different?
0
 

Author Comment

by:tferro999
ID: 34451185
Never was able to resolve this issue.  We went back to forcing local home directories and just passing the authentication through AD without the remote home directories.  I guess it wasnt meant to be.
0
 
LVL 12

Accepted Solution

by:
nxnw earned 500 total points
ID: 34451459
Forcing local home directories has advantages, as was discussed previously. You had indicated that it was forbidden, however.

Did you ever do this:
Can you use \\servername.xxx.edu\macprofiles$\tony3 and see if you get the same errors? I was wondering about DNS because of this error:
"mdworker(320) deny network-outbound /private/var/run/mDNSResponder"
which indicates that mdworker was trying to use bonjour (aka mDNSResponder)

It would be good to ensure that the errors are not a red herring.

Also, I am assuming that your test machine is also having the same problems with a lengthy period of poor performance while spotlight appears to be indexing. Please advise if this assumption is wrong.
0
 

Author Comment

by:tferro999
ID: 34451506
Ya, we tried using IP as well as fqdn with no change.  My test machine was having the same lengthy indexing process as the user machines with a new profile as well as an imported one.  After enough complaints from the users we got the OK from their security watchdogs to go back to the local profiles.

I've since left this job so I guess the issue is closed :)
0
 

Author Closing Comment

by:tferro999
ID: 34451512
tried everything, thanks for the help
0

Featured Post

Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

Join & Write a Comment

In certain circumstances, a Microsoft Office mac application may fail to save a document to a network server, with the following error message: "There has been a network or file permission error. The network connection may be lost." An article at…
The error "There was an error performing the update" occurred on a Mac OS X client workstation running  Symantec AntiVirus for Mac (http://www.symantec.com/business/products/purchasing.jsp?pcid=pcat_security&pvid=825_1) - the Enterprise product vers…
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now