asked on

SQL - How to filter data and pick one record

I have made this up to illustrate what I need. I have records of the following format:
Id, field1, field2
And the sample data is as follow
1, D1, D2
1, DD1, DD2
2, D12, D22
3, D13, D23
3, DD13, DD23
4, D14, D24
5, D15, D25
5, DD15, DD25
6, D16 , D26
7, D17, D27

I have a set of rules to go through Id that have duplicate records. The final result is to have one record per unique id. My question is to how I write the SQL so I can examine a group of records for a particular ID using a set of rules.

Here is some of the rules I have
1. Fields1 and Fields2 are equal pick first 1
2. Fields1 and Fields2 equal the set value choose the next record

Lee Wadwell

I do not understand the rules ...
- your example does not have records that have "Fields1 and Fields2 are equal" that I can tell
- "Fields1 and Fields2 equal the set value" ... what is the "set value"?

Please provide an expected result.

tommym121

ASKER

Sorry,
My table has
autonum, Id, field1, field2
This is the Data
1, 1, D1, D2
2, 1, D1, D2
3, 2, D12, D22
4, 3, D1, D3
5, 3, D1, D3
6, 4, D14, D24
7, 5, D15, D25
8, 5, DD15, DD25
9, 6, D16 , D26
10, 7, D17, D27

Let say I have these two rules
1. Fields1 and Fields2 are equal pick first 1
2. Fields1 and Fields2 equal the DD15 and DD25 correspondingly

This is the result I expect with the data I have
autonum, Id, field1, field2
1, 1, D1, D2
3, 2, D12, D22
4, 3, D1, D3
6, 4, D14, D24
8, 5, DD15, DD25
9, 6, D16 , D26
10, 7, D17, D27

Let me know if you other questions

Lee Wadwell

How about this:

IF OBJECT_ID('tempdb..#tempTable') IS NOT NULL
	DROP TABLE #tempTable

create table #tempTable (
    autonum INT,
    id      INT,
    field1  varchar(10),
    field2  varchar(10)
);
insert into #tempTable values
(1, 1, 'D1', 'D2'),
(2, 1, 'D1', 'D2'),
(3, 2, 'D12', 'D22'),
(4, 3, 'D1', 'D3'),
(5, 3, 'D1', 'D3'),
(6, 4, 'D14', 'D24'),
(7, 5, 'D15', 'D25'),
(8, 5, 'DD15', 'DD25'),
(9, 6, 'D16' , 'D26'),
(10, 7, 'D17', 'D27');

-- select * from #tempTable;

select a.*
  from #tempTable a
  left join #tempTable b
         on a.id = b.id 
            and (   (a.field1 = b.field1 and a.field2 = b.field2 and a.autonum > b.autonum)
                 or (a.field1 != b.field1 and a.field2 != b.field2
                     and right(a.field1,len(a.field1)-patindex('%[^a-zA-Z]%',a.field1)+1) = right(b.field1,len(b.field1)-patindex('%[^a-zA-Z]%',b.field1)+1)
                     and right(a.field2,len(a.field2)-patindex('%[^a-zA-Z]%',a.field2)+1) = right(b.field2,len(b.field2)-patindex('%[^a-zA-Z]%',b.field2)+1)
                     and a.autonum < b.autonum))
 where b.id is null

Open in new window

I assumed for rule 2 - you meant the number in the values where equal.

tommym121

ASKER

lwadwell,

Thanks. One question I have is there anyway not to use the autonum in the select. I put the autonum so I can indicate to you which records are being choose.

Lee Wadwell

Without the autonum ... how do you determine which is the 'first 1' or the 'next 1'?

An arbitrary one could be added with a row_number() function prior using inline views if necessary.

tommym121

ASKER

Yes. I think I will need the row_number() as you suggested. How do you apply the row_number() function? Would you be able to elaborate?

ASKER CERTIFIED SOLUTION

Lee Wadwell

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

SOLUTION

Guy Hengel [angelIII / a3]

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

tommym121

ASKER

lwadwell,

I try to disassemble your code to understand how it will work on my situation.

What is 'Where b.id is null' mean? When I try to remove it it give me all the data including all the duplicate.

Lee Wadwell

The query uses a left join ... a left join will return null to the where clause where a match is not found ... you wanted the records that were not matched.

tommym121

ASKER

Thanks