Solved

Fuzzy logic in SQL

Posted on 2013-01-21
8
2,145 Views
Last Modified: 2016-02-11
WonkinlastyearnoGuitarLotsfloorlotsonly      
WonkinlastyearnoGuitarLots
Hi experts, I need to calculte the match percentage of these two, is there a way to do it? I have heard about fuzzy logic but have no clue as to how to do it, please help
0
Comment
Question by:sqlcurious
8 Comments
 
LVL 75

Expert Comment

by:Anthony Perkins
Comment Utility
Hi experts, I need to calculte the match percentage of these two, is there a way to do it?
You cannot do that with T-SQL, you will have to resort to writing a CLR function to accomplish that.
0
 
LVL 27

Expert Comment

by:tliotta
Comment Utility
What do those names reference? I.e., what is the meaning of the data and what is the data definition?

Tom
0
 
LVL 5

Expert Comment

by:RehanYousaf
Comment Utility
Please provide with sample data and logic you want to use
0
 

Author Comment

by:sqlcurious
Comment Utility
Please look at the attached sample data, and I have the logic in place to figure out the exact matches but I  want the logic to see if there are any close matches in terms of percentage -
For ex: DescrCode 1 and 2 has exact match Description1 and Description2 , DescCode 3 - 6 match > 50% , 7 and 8 less than 50% , this is just tentative percentages but can be more definitive.
Please let me know if you need more info, thanks
sample-data.txt
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 

Author Comment

by:sqlcurious
Comment Utility
this is the correct sample file, pls ignore previous one, thanks
sample-data.txt
0
 
LVL 6

Accepted Solution

by:
liija earned 500 total points
Comment Utility
Could be done with SQL - but might become pretty complex code.
The easiest way is to use SQL Server Integration Services. There's a component that you might use on this, Fuzzy Lookup.

Here's a decent step-by-step guide - even if you have never used SSIS before:
http://oakdome.com/programming/SSIS_FuzzyLookup.php
0
 

Author Closing Comment

by:sqlcurious
Comment Utility
thanks!
0
 
LVL 5

Expert Comment

by:RehanYousaf
Comment Utility
Well there are different ways of doing it e.g

1. As  acperkins said, you can use CLR function to accomplish and the example is
http://anastasiosyal.com/POST/2009/01/11/18.ASPX?

2. As liija said, use SSIS package to do the job

3. If you have SQL Server 2008 R2 then a another option is to install Master Data Services
http://msdn.microsoft.com/en-us/library/ee633742(v=sql.105).aspx

sample code will be something like
DECLARE 
	@Name VARCHAR(100), 
	@Name2 VARCHAR(100)

SET @Name = 'rehan yousaf'
SET @Name2 = 'rayhaan usuf'

SELECT 
	@Name AS [@Name], 
	@Name2 AS [@Name2],
    MDS.mdq.Similarity(@Name, @Name2, 0, 0.85, 0) AS Levenshtein,
    MDS.mdq.Similarity(@Name, @Name2, 1, 0.85, 0) AS Jaccard,
    MDS.mdq.Similarity(@Name, @Name2, 2, 0.85, 0) AS JaroWinkler,
    MDS.mdq.Similarity(@Name, @Name2, 3, 0.85, 0) AS LongestCommonSubsequence

Open in new window


I have also attached the screenshot for option 3 as well
FuzzyLogic01.JPG
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

In this article I will describe the Backup & Restore method as one possible migration process and I will add the extra tasks needed for an upgrade when and where is applied so it will cover all.
International Data Corporation (IDC) prognosticates that before the current the year gets over disbursing on IT framework products to be sent in cloud environs will be $37.1B.
Via a live example, show how to shrink a transaction log file down to a reasonable size.
Viewers will learn how to use the INSERT statement to insert data into their tables. It will also introduce the NULL statement, to show them what happens when no value is giving for any given column.

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now