Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win


MS SQl 2005 full-text index (FTI)

Posted on 2010-08-12
Medium Priority
Last Modified: 2012-05-10

Ive searched the net high and low but there doesnt seem to be much discussion on the pros and cons of FTI.
Q1. can shed some light on this matter.

Ive been asked to create a search facility that needs to include Internet like search (i.e. "this and that", this and that -Those, this or that +Those, contains etc) and i cant do such search requirements using normal sql. The search is going to be against 15+ tables and numerous columns with no more than 300,000 records.

Q2. I know by adding FTI there is a possibity of performance issues but only if you run into million+ records (i wont have that problem), Is this true? Also, how does does it affect database back up or performance when a search is executed.

Q3. If the advise is not to use FTI, any recommendations? An idea i had was to create a temp table which will hold all the data that would be searchable from all differnet tables/columns and search against those using normal type searching (i.e. and, or and like). Thoughts?

Sorry for having so many questions, i hope you guys can help me out.


Question by:gHell
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 4

Accepted Solution

GLoad earned 450 total points
ID: 33426034
Full text indexing is the only way to get FREETEXT type searches and other imperfect string matches. ie, ,you can get a number to rank the relevance for search terms for "my house" as being a match for "my mate's house"

LIKE will not work for that scenario.

Full text indexes do have some overhead, but they are handled out of process from SQL server (it actually uses the microsoft text indexing engine). The main performance hit comes from the building and updating ot those indexes and tables that have millions of rows can take days to build the index the first time. Having said that, they are much faster than the like approaches that try to offer the same functionality.

Having said that I've found them to be maintained very well and provided you have a decent normalized schema with all your indexes in the right place, it is a great tool for user friendly searches.

Key points:

- Create your indexes before you roll out the code that uses them.
- Play with quering the index using the different options available in full text indexes to make sure your relevance is good.
- In SQL server 2005, full text indexes are backed up with the database. 2000 and 2008 have slightly different handling of them.. see http://www.sqlmag.com/article/sql-server/full-text-search-in-backup-recovery.aspx for a bit more info.
LVL 75

Expert Comment

by:Anthony Perkins
ID: 33426047
>>can shed some light on this matter.<<
Full-Text Search is not for everyone.  It does require some additional setup and to some extent some maintenance.  One caveat is that many think it is replacement for using LIKE except on steroids.  While it does to regular searches as does LIKE it cannot search for words ending in something.  For example, using LIKE you can do this to find all the instances that start with Experts such as ExpertsExchange:
WHERE YourColumn LIKE 'Experts%'

and with Full-Text Search that is as simple as:
WHERE CONTAINS(YourColumn, '"Experts*"')

What you cannot do with Full-Text Search is the equivalent of:
WHERE YourColumn LIKE '%Exchange'

But Full-Text Search does far more than that and the best advice I can give is to read up on it as much as possible and try it out.

>>I know by adding FTI there is a possibity of performance issues but only if you run into million+ records (i wont have that problem), Is this true?<<
It is quite the opposite.  The reason you add Full-Text search is to improve performance, especially if you have large datasets.

>> If the advise is not to use FTI, any recommendations? <<
This really depends, if you find Full-Text Search to be too constraining this may be your only choice.  But I would only roll your own as a last resort.

Author Comment

ID: 33437007
Sorry, i didn't explain myself properly, i meant pros/cons server side rather than its usability. I.E disk space, maintenance, most importantly, can the search still operate while indexes are being updated.

One last thing. i have other database on the same server, how will they get affected?

Get your Conversational Ransomware Defense e‑book

This e-book gives you an insight into the ransomware threat and reviews the fundamentals of top-notch ransomware preparedness and recovery. To help you protect yourself and your organization. The initial infection may be inevitable, so the best protection is to be fully prepared.

LVL 75

Assisted Solution

by:Anthony Perkins
Anthony Perkins earned 300 total points
ID: 33437072
Disk space should be minimum.  It only saves unique keywords and many words are excluded (noise words).  Without knowing exactly what you are saving it is difficult to be more precise.  That can only come from experience with your data.  The good news is that it does not take long to figure that out in a test environment.

Full-Text Search indexes can be updated in the background and on a continual basis, so there is no problem there.  One caveat you should be aware of, and that is if you insert 10K rows at one time, it may take a few minutes depending on many factors before the index is totally updated.  So while you can still continue to search for keywords in previous rows, the keywords for the new rows may not be immediately available.

Other databases will not be affected.

Author Comment

ID: 33437084
thank you both for your advise.

Author Closing Comment

ID: 33437110
thank you for you help.
LVL 75

Expert Comment

by:Anthony Perkins
ID: 33437159
I realize you are new here, but please read up on the EE Guidelines regarding grading standards (http://www.experts-exchange.com/help.jsp#hs=29&hi=403):

What's the right grade to give?
Grading at Experts Exchange is not like school. It's more like the "10-point Must" system in professional boxing; in other words, an answer is worth an A, unless it doesn't resolve your issue. If it requires you to do a little more research, or figure out one more piece of code, then it's worth a B. If you think it's not worth a B, the custom is to offer the Experts an opportunity to earn a better grade.

Giving a higher grade has no impact on your Available Points.

Author Comment

ID: 33439086
thank you for letting me know, following the guidelines this should have been and A. I dont think i can change it now but ill be sure to assign grades appropriately in future.
LVL 75

Expert Comment

by:Anthony Perkins
ID: 33439122
That is very fair. Thank you.

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

by Mark Wills Attending one of Rob Farley's seminars the other day, I heard the phrase "The Accidental DBA" and fell in love with it. It got me thinking about the plight of the newcomer to SQL Server...  So if you are the accidental DBA, or, simp…
In SQL Server, when rows are selected from a table, does it retrieve data in the order in which it is inserted?  Many believe this is the case. Let us try to examine for ourselves with an example. To get started, use the following script, wh…
Video by: ITPro.TV
In this episode Don builds upon the troubleshooting techniques by demonstrating how to properly monitor a vSphere deployment to detect problems before they occur. He begins the show using tools found within the vSphere suite as ends the show demonst…
In response to a need for security and privacy, and to continue fostering an environment members can turn to for support, solutions, and education, Experts Exchange has created anonymous question capabilities. This new feature is available to our Pr…

610 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question