Solved

Using SOUNDEX to match individual words within Text fields

Posted on 2007-04-10
6
741 Views
Last Modified: 2008-01-09
I have a table which contains  a Text fields containing course information.

I am constructing a 'fuzzy search' stored procedure which will match words within this field based on what was entered into a web front end.

The part I am finding difficulty with is matching on mis-spelled words

I thought I could use SOUNDEX for this but it only appears to be finding matches where the first word of the Text field is the word I am looking for.

So, for example, my search value is 'compter course' (note it is mis-spelled)

There are records in my DB which have text field value of 'Computers for beginners' and 'An introduction to computers'

My SOUNDEX query partly does the job but only the first record is matched :(

Is there a way to use SOUNDEX across all words in the text field or is ther another approach I can look at?

I am using SQL 2005

Thanks
0
Comment
Question by:WNottsC
6 Comments
 
LVL 11

Expert Comment

by:Ved Prakash Agrawal
ID: 18881525
http://databases.about.com/od/development/l/aasoundex.htm

just try to look this may help you.. not sure it wil solve your problem or not..
I am looking your problem and give the feedback very soon...

0
 
LVL 143

Assisted Solution

by:Guy Hengel [angelIII / a3]
Guy Hengel [angelIII / a3] earned 200 total points
ID: 18881529
you cannot use soundex efficiently for this.
you might consider using full-text search for this kind of query.
look up
CONTAINS
CONTAINSTABLE
FREETEXTTABLE
in the books online.
0
 
LVL 23

Expert Comment

by:Racim BOUDJAKDJI
ID: 18881547
<<you cannot use soundex efficiently for this.>>
Agreed.  I would also ask: why use soundex at all.
0
The Eight Noble Truths of Backup and Recovery

How can IT departments tackle the challenges of a Big Data world? This white paper provides a roadmap to success and helps companies ensure that all their data is safe and secure, no matter if it resides on-premise with physical or virtual machines or in the cloud.

 
LVL 50

Accepted Solution

by:
Lowfatspread earned 300 total points
ID: 18881639
you can use soundex for some limited "misspelled word"  problems but i agree with angelliii and racimo its usage will not be easy ...

you need to filter for noise words and build soundex values for each remaining word and compare those...

full text searching is probably a better solution these days...

however your problem really seems  to stem from allowing a free format input in the firstplace... if your are really trying to allow for searching a course catalogue
why is a simple cross reference / category selection not the simplest approach...

ie type the course to an agreed set of course key words   COMPUTING / BASIC / PROGRAMMING / THEORY / ....

then just let the userr select from the defined set of Keywords...

0
 

Author Comment

by:WNottsC
ID: 18881815
Thanks

My SP uses all the other methods mentioned above already, none of them produce a match on 'compting' except soundex.  I already do some tidying up of noise words etc as well, the SP is growing into a bit of a monster.

I agree the design of the DB & application is not perfect but its what I've been given to work with
0
 

Author Comment

by:WNottsC
ID: 18896180
Update

Fortunately the data used here is only refreshed every few weeks so Ive created an overnight which splits the words in the search field into separate fields in a new table, then a soundex match is performed on each field in turn.

Hopefully this will catch enough typos to let us escape creating a thesaurus which I really, really want to avoid
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
GeoClustering  and AOG 25 41
How to handle time out inside the stored procedure 10 24
SQL Server Trigger 8 32
Copy data to New Year 9 29
Introduction SQL Server Integration Services can read XML files, that’s known by every BI developer.  (If you didn’t, don’t worry, I’m aiming this article at newcomers as well.) But how far can you go?  When does the XML Source component become …
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
Via a live example combined with referencing Books Online, show some of the information that can be extracted from the Catalog Views in SQL Server.
Using examples as well as descriptions, and references to Books Online, show the documentation available for datatypes, explain the available data types and show how data can be passed into and out of variables.

680 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question