SQL - How to filter data and pick one record

I have made this up to illustrate what I need. I have records of the following format:
Id, field1, field2
And the sample data is as follow
1, D1, D2
1, DD1, DD2
2, D12, D22
3, D13, D23
3, DD13, DD23
4, D14, D24
5, D15, D25
5, DD15, DD25
6, D16 , D26
7, D17, D27

I have a set of rules to go through Id that have duplicate records.  The final result is to have one record per unique id.   My question is to how I write the SQL so I can examine a group of records for a particular ID using a set of rules.

Here is some of the rules I have
1.  Fields1 and Fields2 are equal pick first 1
2.  Fields1 and Fields2 equal the set value choose the next record
tommym121Asked:
Who is Participating?
 
lwadwellCommented:
IF OBJECT_ID('tempdb..#tempTable') IS NOT NULL
	DROP TABLE #tempTable;

create table #tempTable (
    id      INT,
    field1  varchar(10),
    field2  varchar(10)
);
insert into #tempTable values
(1, 'D1',  'D2'),
(1, 'D1',  'D2'),
(2, 'D12', 'D22'),
(3, 'D1',  'D3'),
(3, 'D1',  'D3'),
(4, 'D14', 'D24'),
(5, 'D15', 'D25'),
(5, 'DD15','DD25'),
(6, 'D16', 'D26'),
(7, 'D17', 'D27');

-- select * from #tempTable;

;with temp_with_seq as (
select id, field1, field2, row_number()over(order by id) autonum 
from #tempTable
)
select a.*
  from temp_with_seq a
  left join temp_with_seq b
         on a.id = b.id 
            and (   (a.field1 = b.field1 and a.field2 = b.field2 and a.autonum > b.autonum)
                 or (a.field1 != b.field1 and a.field2 != b.field2
                     and right(a.field1,len(a.field1)-patindex('%[^a-zA-Z]%',a.field1)+1) = right(b.field1,len(b.field1)-patindex('%[^a-zA-Z]%',b.field1)+1)
                     and right(a.field2,len(a.field2)-patindex('%[^a-zA-Z]%',a.field2)+1) = right(b.field2,len(b.field2)-patindex('%[^a-zA-Z]%',b.field2)+1)
                     and a.autonum < b.autonum))
 where b.id is null;

Open in new window

0
 
lwadwellCommented:
I do not understand the rules ...
  - your example does not have records that have "Fields1 and Fields2 are equal" that I can tell
  - "Fields1 and Fields2 equal the set value" ... what is the "set value"?

Please provide an expected result.
0
 
tommym121Author Commented:
Sorry,
My table has
autonum, Id, field1, field2
This is the Data
1, 1, D1, D2
2, 1, D1, D2
3, 2, D12, D22
4, 3, D1, D3
5, 3, D1, D3
6, 4, D14, D24
7, 5, D15, D25
8, 5, DD15, DD25
9, 6, D16 , D26
10, 7, D17, D27

Let say I have these two rules
1.  Fields1 and Fields2 are equal pick first 1
2.  Fields1 and Fields2 equal the DD15 and DD25 correspondingly

This is the result I expect with the data I have
autonum, Id, field1, field2
1, 1, D1, D2
3, 2, D12, D22
4, 3, D1, D3
6, 4, D14, D24
8, 5, DD15, DD25
9, 6, D16 , D26
10, 7, D17, D27

Let me know if you other questions
0
Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

 
lwadwellCommented:
How about this:
IF OBJECT_ID('tempdb..#tempTable') IS NOT NULL
	DROP TABLE #tempTable

create table #tempTable (
    autonum INT,
    id      INT,
    field1  varchar(10),
    field2  varchar(10)
);
insert into #tempTable values
(1, 1, 'D1', 'D2'),
(2, 1, 'D1', 'D2'),
(3, 2, 'D12', 'D22'),
(4, 3, 'D1', 'D3'),
(5, 3, 'D1', 'D3'),
(6, 4, 'D14', 'D24'),
(7, 5, 'D15', 'D25'),
(8, 5, 'DD15', 'DD25'),
(9, 6, 'D16' , 'D26'),
(10, 7, 'D17', 'D27');

-- select * from #tempTable;

select a.*
  from #tempTable a
  left join #tempTable b
         on a.id = b.id 
            and (   (a.field1 = b.field1 and a.field2 = b.field2 and a.autonum > b.autonum)
                 or (a.field1 != b.field1 and a.field2 != b.field2
                     and right(a.field1,len(a.field1)-patindex('%[^a-zA-Z]%',a.field1)+1) = right(b.field1,len(b.field1)-patindex('%[^a-zA-Z]%',b.field1)+1)
                     and right(a.field2,len(a.field2)-patindex('%[^a-zA-Z]%',a.field2)+1) = right(b.field2,len(b.field2)-patindex('%[^a-zA-Z]%',b.field2)+1)
                     and a.autonum < b.autonum))
 where b.id is null

Open in new window

I assumed for rule 2 - you meant the number in the values where equal.
0
 
tommym121Author Commented:
lwadwell,

Thanks.   One question  I have is there anyway not to use the autonum in the select.  I put the autonum so I can indicate to you which records are being choose.
0
 
lwadwellCommented:
Without the autonum ... how do you determine which is the 'first 1' or the 'next 1'?

An arbitrary one could be added with a row_number() function prior using inline views if necessary.
0
 
tommym121Author Commented:
Yes. I think I will need the row_number() as you suggested. How do you apply the row_number() function?  Would you be able to elaborate?
0
 
Guy Hengel [angelIII / a3]Billing EngineerCommented:
0
 
tommym121Author Commented:
lwadwell,

I try to disassemble your code to understand how it will work on my situation.

What is 'Where b.id is null' mean?  When I try to remove it it give me all the data including all the duplicate.
0
 
lwadwellCommented:
The query uses a left join ... a left join will return null to the where clause where a match is not found ... you wanted the records that were not matched.
0
 
tommym121Author Commented:
Thanks
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.