Sharepoint 2010 Event Id 14 Every Hour

SharePoint 2010 Farm SP2 CU June 2015

I had to delete the Search Service Application due to many errors after a Server restart.

I recreated the Search Service Application

Set up the content sources added the web applications set the schedule for full and incremental

Also set up a crawl rule to exclude http://tgcs013:29261

Ran a re index and a full crawl  get no errors from the crawl log

but I get this error every hour on the server application event log.


Log Name:      Application
Source:        Microsoft-SharePoint Products-SharePoint Foundation Search
Date:          7/24/2015 8:56:23 PM
Event ID:      14
Task Category: Gatherer
Level:         Warning
Keywords:      
User:          OUR\spserv
Computer:      TGCS013.FQDN.com
Description:
The start address sts4://tgcs013:29261/contentdbid={ac89cd5e-197e-46c6-b681-11fa35974595} cannot be crawled.

Context: Application 'Search_index_file_on_the_search_server', Catalog 'Search'

Details:
      Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled.   (0x80041205)
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-SharePoint Products-SharePoint Foundation Search" Guid="{1C415899-58B3-4BFC-9236-105E7FD38719}" />
    <EventID>14</EventID>
    <Version>14</Version>
    <Level>3</Level>
    <Task>174</Task>
    <Opcode>0</Opcode>
    <Keywords>0x2000000000000000</Keywords>
    <TimeCreated SystemTime="2015-07-25T00:56:23.244781700Z" />
    <EventRecordID>38238</EventRecordID>
    <Correlation />
    <Execution ProcessID="920" ThreadID="7396" />
    <Channel>Application</Channel>
    <Computer>TGCS013.our.network.tgcsnet.com</Computer>
    <Security UserID="S-1-5-21-3054588571-1341459584-784128302-3140" />
  </System>
  <EventData>
    <Data Name="string0">sts4://tgcs013:29261/contentdbid={ac89cd5e-197e-46c6-b681-11fa35974595}</Data>
    <Data Name="string1">

Context: Application 'Search_index_file_on_the_search_server', Catalog 'Search'

Details:
      Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled.   (0x80041205)</Data>
  </EventData>
</Event>


What am I missing
LVL 23
Thomas GrassiSystems AdministratorAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Thomas GrassiSystems AdministratorAuthor Commented:
Shailaja

thanks for responding

yes i did

The issue is with the Central Administration site which is in the event id 14 message

The article does not apply to Central Administation

I setup a Crawl Rule to exclude the Central Adminstration but that seems not to clear the error every hour on the hour.

Thoughts
Shailaja KumarCommented:
I don't think its the SharePoint Central Administration database, usually the CA content database will follow this naming convention "SharePoint_AdminContent_GUID" unless you have changed it. If possible can you check again if its the CA or any other content database, and try to disable the loop back check.
Protecting & Securing Your Critical Data

Considering 93 percent of companies file for bankruptcy within 12 months of a disaster that blocked access to their data for 10 days or more, planning for the worst is just smart business. Learn how Acronis Backup integrates security at every stage

Thomas GrassiSystems AdministratorAuthor Commented:
Shailaja

This is why I was thinking it was the CA  based on the start address information here

The start address sts4://tgcs013:29261/contentdbid={ac89cd5e-197e-46c6-b681-11fa35974595} cannot be crawled.

Are you saying that the database in question in this then?

contentdbid={ac89cd5e-197e-46c6-b681-11fa35974595}


How do I determine which this is?

I only have one Content Database

WSS_Content_069b8e47-103c-432f-a684-c78a714b6e4e

I listed my databases see attached


Thoughts


PS

I made the Registry Change for Loopback a long time ago it has no effect on this issue
Sharepoint-Databases.png
Thomas GrassiSystems AdministratorAuthor Commented:
Hi

Update

I did some checking on the databases

I ran get-spdatabase and get-spcontentdatabase     see the attached file

The start address sts4://tgcs013:29261/contentdbid={ac89cd5e-197e-46c6-b681-11fa35974595} cannot be crawled.

The guid in the above message is SharePoint_AdminContent_ac89cd5e-197e-46c6-b681-11fa35974595

So the message is correct , this SharePoint_AdminContent_ac89cd5e-197e-46c6-b681-11fa35974595 cannot be crawled

How to exclude this db from being crawled?
spdatabase.txt
Walter CurtisSharePoint AEDCommented:
Content sources are base on the web application root url's, not on content databases. Unless you explicitly set up the central administration web application as a content source, search will not crawl central administration.

The access denied message indicated that either the user used for search does not have access to the web application, or the password for the user is incorrect.

Hope that helps
Thomas GrassiSystems AdministratorAuthor Commented:
SneekCo

When I run this command get-spcontentdatabase
Id               : 1516326e-116c-4115-93ec-7e17376fe632
Name             : WSS_Content_069b8e47-103c-432f-a684-c78a714b6e4e
WebApplication   : SPWebApplication Name=SharePoint - 80
Server           : tgcs010\sharepoint
CurrentSiteCount : 3

Id               : 1189699f-74b7-449f-973c-cd4837b52041
Name             : WSS_Content_My_Sites
WebApplication   : SPWebApplication Name=SharePoint - My Site
Server           : tgcs010\sharepoint
CurrentSiteCount : 1

That proves that Central Administration was not set up as a content source correct?

If that is true then why is the error message event ID 14 pointing to the
SharePoint_AdminContent_ac89cd5e-197e-46c6-b681-11fa35974595 database?

That is Central Administration trying to be crawled unless the error message is misleading


The access denied message indicated that either the user used for search does not have access to the web application, or the password for the user is incorrect.


As far as the user account for search goes I believe it is the same for the other two content databases being searched and they work? Where do I check this?

Thoughts
Walter CurtisSharePoint AEDCommented:
Here a few basics that are important -

Search crawls are based "search" content sources. Content databases are not the same. The PowerShell command "Get-SPContentDatabase"  has nothing to do with search content sources. One search content source can crawl multiple content databases from multiple web apps.

For access, the search account needs full read access to the content of the web app that is being crawled. The security on the content database is configured by SharePoint automatically in a standard installation.

Yes, the error message is misleading and can lead to confusion.

Hope that helps
Thomas GrassiSystems AdministratorAuthor Commented:
Sneekco

Thanks for that

So then How do I check to see 1. which account is doing the search and 2 where do I check the security to make sure it has full read access?
Walter CurtisSharePoint AEDCommented:
In central administration there is a page named Farm Search Administration. Within the section (page) there is a section named "System Status" and there you will find "Default content access account".

 For granting access to the web app for you crawl account, go to Application Management --> Manage Web applications --> highlight the web app you want to grant access to, (do them one at a time), click in the ribbon on the icon "user Policy" --> Click on the button "Add Users" and then add you crawl (search) account with full read permission. Then the account will be have access to the entire web app (sites).
Thomas GrassiSystems AdministratorAuthor Commented:
Sneekco

Thanks found the information

I created a new user account on the domain  our\spcontent
Register the  account as managed accounts in SharePoint Central Admin, Security, Configure Managed Accounts:

Then made spcontent the Default content access account in
Central Administration > Search Service Application> Search Administration

Then Central Administration > Manage Web Applications

highlighted the Web Application "SharePoint - My Site" and on the Ribbon bar selected User Policy in user Policy
added spcontent with Full Read permissions

Did that for my other Web Application "SharePoint - 80"

Still getting Event Id 14 and in the error it s using the our\spserv account not our\spcontent account
see error in first posting

Do I need to restart something?
Walter CurtisSharePoint AEDCommented:
This shows that there is a content source using the sts4 protocol to crawl and it is crawling what appears to the the ca site.

The start address sts4://tgcs013:29261/contentdbid={ac89cd5e-197e-46c6-b681-11fa35974595} cannot be crawled.

When you set up the search service application, there should have been a content source named "Local SharePoint Sites".  Per default your SharePoint -80 and My Sites should be a part of the content source and they should use the HTTP protocol not sts4. If you don have a content source with sts4 protocol, you can delete that, it is not necessary. Then the error message, (which is just actually a warning and you will always get warnings, impossible to clean totally), will go away.
Thomas GrassiSystems AdministratorAuthor Commented:
SneekCo

Yes that's why these messages are so misleading.

In my Central Administration > Search Service Application> Edit Content Source named "Local SharePoint sites"

In Start Addresses I have
http://mysite.tgcsnet.com
http://sharepointtgcs.com
sps3://sharepointtgcs.com


No sts4 listed at all


Thoughts?
Walter CurtisSharePoint AEDCommented:
That looks good. The sps3 is for the user profile service. Not sure why the sts4 was ever in there if it was. No need as it is simply a different rendition for the http protocol and is not needed to be used.
Thomas GrassiSystems AdministratorAuthor Commented:
SneekCo

I added the sps3://sharepointtgcs.com when I got another message saying it needed to be added.

Should I remove it ?

Yes where in the world is the STS4 coming from?
Walter CurtisSharePoint AEDCommented:
The sps3:// is correct. The concern I see is the .com url. If this is internal only, the .com will cause problems because it will be trying to the outside world.
Thomas GrassiSystems AdministratorAuthor Commented:
What is the correct method ?

Please provide example

I attached my Alternate Access Mappings
Sharepoint-2010-AAM-2.png
Walter CurtisSharePoint AEDCommented:
Is http://sharepointtgcs.com a public facing site, with a registered DNS entry with a naming service, connected directly to the internet. If so, turn it off quick before any data you have on there is hacked.

If you just used that name and it is internal, it will be problematic because anytime anyone tries to get to it, even internally, your dns will try and hit a public name server. The users will probably get unable to connect errors in their broswer.

This is really going off of the topic of the original question, so might be a good time to close this questions, and start a new one - How to Extend a Web App

Hope that helps
Thomas GrassiSystems AdministratorAuthor Commented:
Ok the lets fix the question

Why is event Id 14 repeating every hour

I was giving you information you was requesting
Walter CurtisSharePoint AEDCommented:
Yes, thanks for the information. For the original questions, we don't need to progress to aam, dns, web extensions or how the site was designed :-)
Thomas GrassiSystems AdministratorAuthor Commented:
So where do we go next?

This error is filling up my event log probably missing other issues that are just as important

Need this resolved

Thoughts
Shailaja KumarCommented:
Thomas,

Can you try opening the SQL database for the CA site and check under "Security", if there is a search app pool account added? Please don't change anything in the database.

I am not sure about this, but no harm in trying
Thomas GrassiSystems AdministratorAuthor Commented:
Which database there are 20 of them

I found it SharePoint_AdminContent_8aef1eb7-2efb-456a-80c4-e8a83c732bc5

Security > Users    found our\SPSERV

Owned Schemas       OUR\spserv

Role Members          db_owner           WSS_Content_Application_Pools


What is your thinking on this?
Thomas GrassiSystems AdministratorAuthor Commented:
Everyone still getting this event over and over

Need suggestions

Thanks
Walter CurtisSharePoint AEDCommented:
Here's the bottom line. This is a warning. It is annoying to see things in the event viewer, but that is the SharePoint life. You will always get warnings and information that don't require any action. Actual errors and critical errors require additional, but there are several bogus errors in SharePoint, and when you find the official Microsoft fix, it might even say so. It's not a perfect product. Maybe try and forget about the warning and check for errors and critical errors.

Just my two cents worth. This doesn't come out of laziness, but out of 17 years working the MS stack.
Thomas GrassiSystems AdministratorAuthor Commented:
Shailaja Kumar/ SneekCo

Today I was looking at this and when I was looking at another issue I stumbled across reason why this is happening

I disabled the Job definition
SharePoint Foundation Search Refresh
This runs hourly soon s I disabled this job the Event Id 14 stopped.

So the question still stands what is wrong?

I just disabled a function that needs to run for now,


Anyone with any thoughts?
Did I miss a step when I deleted the Service and recreated it?
Walter CurtisSharePoint AEDCommented:
I have a few questions that may help with troubleshooting:
When you disable the hourly job for Search Refresh, did you see any changes in your farm?
Were there any side effects, such as search crawls failing? Incomplete crawls?
Why do you say the function needs to run? Is it possible that it never should have been running in the first place? Have you checked the documentation to see if it should run or not? Do you have Fast Search enabled (installed) on your farm?
Is the farm in prod? For how long? What steps have you taken as far as resetting the search index and re-indexing? Exactly how was the search application created?

The answers to these questions may help determine what is causing the warning.
Thomas GrassiSystems AdministratorAuthor Commented:
sneekCo

I will answer the above later

I found these instructions to rebuild the Search Service which I am going to do again

https://smartrider.wordpress.com/2012/05/31/how-to-rebuild-search-service-application-in-sharepoint-2010/

Only question I have is on step 9

Add the existing portal address in the content sources and setup Full Crawl and Incremental crawl.
I know how to setup the crawl and incremental crawl
What does he mean by "Add the existing portal address in the content sources" ?????

Not sure about that once I figure that step out then I should be able to rebuild a successfull serach service

Thoughts
Walter CurtisSharePoint AEDCommented:
When you go to the Search Configuration page of Central Admin, in the left hand navigation you see a link name "Content Sources". That is where you set up your content sources. You will see a content source already present named "Local SharePoint Sites". This may be the only content source you need, so just kick off a full crawl and see what happens.
Thomas GrassiSystems AdministratorAuthor Commented:
Sneekco

After a complete rebuild of the Search Service application I am now getting this and other errors over and over.

Event Id 6398 over and over

Type :            Critical
Date :            8/7/2015
Time :            3:10:24 PM
Event :            6398
Source :            Microsoft-SharePoint Products-SharePoint Foundation
Category :      Timer
User :            xxxx\spfarm
Computer :      serv013.fqdn.com
Description:
The Execute method of job definition Microsoft.Office.Server.Search.Administration.CrawlReportJobDefinition (ID 0739d3f1-c076-4197-a3b3-2b37739107a5) threw an exception. More information is included below.

Cannot open database "Search_Service_Application_DB_9e3d8f54cf3b44beb28e880e324b3449" requested by the login. The login failed.
Login failed for user 'xxx\spfarm'.


Did I miss a step?

here is what I did in order

Rebuild Search Service Application

Delete the Search Service Application

1. Open SharePoint Management Shell (as an administrator)

2. Locate the GUID for the search service application by typing this
get-spserviceApplication |?{$_.name -eq "search service application"}

PS C:\> get-spserviceApplication |?{$_.name -eq "search service application"}
Name                  : Search Service Application
Id                    : 4e60d798-f907-4135-9e1e-20b6d2888dfc
ServiceName           : SearchQueryAndSiteSettingsService
QueryTopologies       : {2f6b159e-5370-4d23-8ebe-22a471a66090}
PropertyStores        : {Search_Service_Application_PropertyStoreDB_77f36f891eaa41dfbeb1d19ffd30bc82}
CrawlTopologies       : {1e14fc27-60fe-4373-9a76-6d2526f838a7}
CrawlStores           : {Search_Service_Application_CrawlStoreDB_e9f36e0ff1b34f36a054c17a9bc3da24}
SearchAdminDatabase   : SearchAdminDatabase Name=Search_Service_Application_DB_9e3d8f54cf3b44beb28e880e324b3449
Status                : Online
SearchApplicationType : Regular
DefaultSearchProvider : SharepointSearch
Properties            : {Microsoft.Office.Server.Utilities.SPPartitionOptions}
PS C:\>

3. Copy the names of the Databases associated with the Search Service Application

PropertyStoreDB - Search_Service_Application_PropertyStoreDB_77f36f891eaa41dfbeb1d19ffd30bc82
                  Integrity 12:18, Rebuild Index 1:18, Clean Up History 3:18, Full Backup 2:18, Maintenance 4:18 Sunday
CrawlStoreDB    - Search_Service_Application_CrawlStoreDB_e9f36e0ff1b34f36a054c17a9bc3da24
                  Integrity 12:10, Rebuild Index 1:10, Clean Up History 3:10, Full Backup 2:10, Maintenance 4:10 Sunday
DB              - Search_Service_Application_DB_9e3d8f54cf3b44beb28e880e324b3449
                  Integrity 12:22, Rebuild Index 1:22, Clean Up History 3:22, Full Backup 2:22, Maintenance 4:22 Sunday

4. Run the following stsadm command

stsadm -o deleteconfigurationobject -id 4e60d798-f907-4135-9e1e-20b6d2888dfc

PS C:\> stsadm -o deleteconfigurationobject -id 4e60d798-f907-4135-9e1e-20b6d2888dfc

Operation completed successfully.

PS C:\>


5. Delete the following Databases from SQL Management Studio on TGCS010. The Exact names that were copied from step 3.

      PropertyStoreDB - Search_Service_Application_PropertyStoreDB_77f36f891eaa41dfbeb1d19ffd30bc82
      CrawlStoreDB    - Search_Service_Application_CrawlStoreDB_e9f36e0ff1b34f36a054c17a9bc3da24
      DB              - Search_Service_Application_DB_9e3d8f54cf3b44beb28e880e324b3449

6. Delete the index files from C:\Program Files\Microsoft Office Servers\14.0\Data\Office Server\Applications
      Delete the entire contents of this folder they will be recreated automatically


           Configure the new Search Service Application

7. Go to Central Administration > Manage Service Applications

8. Click New Search Service Application

9. Provide a name to the application, select the managed account and click OK

Name Search Service Application
Managed Account xxx\spsearch  password

      Wait for the confirmation

10. Add the existing portal address in the content sources

11. Setup FullCrawl and Incremental Crawl

12. Run full crawl

   

The 6398 error is pointing to the old search db

Thoughts
Thomas GrassiSystems AdministratorAuthor Commented:
Closing this out as unsolved but still awarding point because of the good effort.

I decided to Completed rebuild the Farm from the beginning.

Hopefully this will help

Thanks for all your help
Walter CurtisSharePoint AEDCommented:
Thanks
Thomas GrassiSystems AdministratorAuthor Commented:
Walter CurtisSharePoint AEDCommented:
I am tied up in some heavy work load, but will take a look when I have a chance!
Shailaja KumarCommented:
Hi Thomas,

What is the status now? still getting errors?
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SharePoint

From novice to tech pro — start learning today.