Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Duplicate records in tables

Posted on 2003-11-20
12
Medium Priority
?
1,286 Views
Last Modified: 2013-12-09
Hi there,

I have a couple of tables that have duplicate records.
I do NOT want to delete the duplicate records.
All I want to do, is add a primary key to the table so that the records can be uniquely identified.

What would be the easiest way to add a Primary Key (or a field) to the tables without loosing the duplicate records? (hope this is possible).
A query, small program would really help.

If I add a field to the table and try to make it a primary key,
Interbase manager generates an error.

The table have about 3000 000 records.
Im using interbase 5.1 and 5.5.
Programming in Borland C++ Builder 5

Thanks


0
Comment
Question by:morees
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 3
  • 2
12 Comments
 
LVL 10

Accepted Solution

by:
kacor earned 400 total points
ID: 9787769
Hi morees,

1. create domains if needed

2. create NewTable with the existing structure
    create table NewTable
    (
    ID_field       integer NOT NULL, //this is your new column for primary key
    Field2 FieldType2,
    ... etc
    CONSTRAINT PK_ID_Field PRIMARY KEY (ID_field)
    );    
3. create generator
    create generator xxxx;
4. create trigger      //to increment the PK

    set term ^ ;
    CREATE TRIGGER xxxx for NewTable
    ACTIVE BEFORE INSERT POSITION 0
    AS
      BEGIN
            NEW.ID_Field = GEN_ID( xxxx, 1 );
      END
     ^
    set term ;
5. copy your data to the new table

   insert into NewTable
        select *
        from OldTable

I hope it helps

wbr

Janos
0
 

Author Comment

by:morees
ID: 9788068
Thanks for the reply kacor...

I tried what you said I must end then I get the following error:

"Unsuccessful execution caused by a system error that precludes
successful execution of subsequent statements.
internal gds software consistency check (wrong record length (183))."

In the table there are 2 records which are identical.
All the field values are exactly the same. for example

# | code | price ...
----------------------
1  | 123  | 12.45 ...
1  | 123  | 12.45 ...

There are a couple of records like this.

I think this might be why I get the error.
Is there a way around this (I hope so)?

Cheers

0
 
LVL 10

Expert Comment

by:kacor
ID: 9789241
Hi moores,

You know in a database the system permits no record duplication because every record must differ - min. one character - from the other. I told therefore to insert a new field which would be unique - so every record could differ each from other. What I can imagine: you have to make a stored procedure to search for duplicated record in the old database/table and if this search found something duplicated, in a given textual column places a suffix for example "dummy" but only for the second/third/etc. "copy". If it would be repeated more time this would be added twice, threetimes, etc. so every record could differ (eg. every record would be unique in this sense). After this tuning you could copy the "old" (but renewed) table into the new table, thereafter you can delete from this tuned column every occurence of the added word (in this case "dummy"). This is the only possibility I mean. May be some more experienced gurus could suggest something better, now this was what I could imagine.

have success!

wbr

Janos
0
Will your db performance match your db growth?

In Percona’s white paper “Performance at Scale: Keeping Your Database on Its Toes,” we take a high-level approach to what you need to think about when planning for database scalability.

 
LVL 10

Expert Comment

by:kacor
ID: 9789258
but of course I am waiting your message about

Janos
0
 
LVL 10

Expert Comment

by:kacor
ID: 9789404
As now I read the complet question and the comments, I see that you wanted to be every record unique. Sorry for the mistake. I felt this was my idea :-)))))

again sorry  Janos
0
 

Author Comment

by:morees
ID: 9795480
kacor,

you know this error that I got when I tried your lolution

(Unsuccessful execution caused by a system error that precludes
successful execution of subsequent statements.
internal gds software consistency check (wrong record length (183)).")   <-- this error,

well I tried to browse through the records and the same error came up just by going through the records.
I tried doing a backup and recovery but got the same error.
So I had to recover an old database and tried your solution again. I had to add a 'dummy' column in the old table just so that the column count matched in the 2 tables and geuss what it worked.
Thanx for 'what should have been a quick solution', but took longer because of a stuffed upped table and a poor database.

Cheers
0
 
LVL 10

Expert Comment

by:kacor
ID: 9795507
hi morees,

the pleasure is mine

Janos
0
 
LVL 6

Expert Comment

by:BAlexandrov
ID: 9795612
just a comment on that - same can be achieved with adding an ID collumn to same table
Just add an integer collumn, set unique values and then make it primary keys.
you can achieve it with

CREATE GENERATOR gen_yourtable_id;

update yourtable
set id = gen_id(gen_yourtable_id,1)

Update in interbase is performed record by record like iterating with cursor (delete also) so this will work.

Bojidar Alexandrov
0
 

Author Comment

by:morees
ID: 9795917
BAlexandro
I tried what you said and it worked aswell.
The only difference was that I had to set the new field to be the primary field after I updated the table, and not before the update.
It generates an error if you try to make it a primary field before the update

Thanks

0
 
LVL 10

Expert Comment

by:kacor
ID: 9808968
Hi Bojidar,

I offered this solution because in his question morees said, he tried to insert a new column

wbr

Janos
0
 
LVL 6

Expert Comment

by:BAlexandrov
ID: 9809010
that's right - may be you have to read my answer again?
0
 
LVL 10

Expert Comment

by:kacor
ID: 9809080
Sorry Bojidar, it escaped my attention

wbr Janos
0

Featured Post

Survive A High-Traffic Event with Percona

Your application or website rely on your database to deliver information about products and services to your customers. You can’t afford to have your database lose performance, lose availability or become unresponsive – even for just a few minutes.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This post contains step-by-step instructions for setting up alerting in Percona Monitoring and Management (PMM) using Grafana.
This post looks at MongoDB and MySQL, and covers high-level MongoDB strengths, weaknesses, features, and uses from the perspective of an SQL user.
In this video, Percona Director of Solution Engineering Jon Tobin discusses the function and features of Percona Server for MongoDB. How Percona can help Percona can help you determine if Percona Server for MongoDB is the right solution for …
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question