Solved

Duplicate records in tables

Posted on 2003-11-20
12
1,274 Views
Last Modified: 2013-12-09
Hi there,

I have a couple of tables that have duplicate records.
I do NOT want to delete the duplicate records.
All I want to do, is add a primary key to the table so that the records can be uniquely identified.

What would be the easiest way to add a Primary Key (or a field) to the tables without loosing the duplicate records? (hope this is possible).
A query, small program would really help.

If I add a field to the table and try to make it a primary key,
Interbase manager generates an error.

The table have about 3000 000 records.
Im using interbase 5.1 and 5.5.
Programming in Borland C++ Builder 5

Thanks


0
Comment
Question by:morees
  • 7
  • 3
  • 2
12 Comments
 
LVL 10

Accepted Solution

by:
kacor earned 100 total points
ID: 9787769
Hi morees,

1. create domains if needed

2. create NewTable with the existing structure
    create table NewTable
    (
    ID_field       integer NOT NULL, //this is your new column for primary key
    Field2 FieldType2,
    ... etc
    CONSTRAINT PK_ID_Field PRIMARY KEY (ID_field)
    );    
3. create generator
    create generator xxxx;
4. create trigger      //to increment the PK

    set term ^ ;
    CREATE TRIGGER xxxx for NewTable
    ACTIVE BEFORE INSERT POSITION 0
    AS
      BEGIN
            NEW.ID_Field = GEN_ID( xxxx, 1 );
      END
     ^
    set term ;
5. copy your data to the new table

   insert into NewTable
        select *
        from OldTable

I hope it helps

wbr

Janos
0
 

Author Comment

by:morees
ID: 9788068
Thanks for the reply kacor...

I tried what you said I must end then I get the following error:

"Unsuccessful execution caused by a system error that precludes
successful execution of subsequent statements.
internal gds software consistency check (wrong record length (183))."

In the table there are 2 records which are identical.
All the field values are exactly the same. for example

# | code | price ...
----------------------
1  | 123  | 12.45 ...
1  | 123  | 12.45 ...

There are a couple of records like this.

I think this might be why I get the error.
Is there a way around this (I hope so)?

Cheers

0
 
LVL 10

Expert Comment

by:kacor
ID: 9789241
Hi moores,

You know in a database the system permits no record duplication because every record must differ - min. one character - from the other. I told therefore to insert a new field which would be unique - so every record could differ each from other. What I can imagine: you have to make a stored procedure to search for duplicated record in the old database/table and if this search found something duplicated, in a given textual column places a suffix for example "dummy" but only for the second/third/etc. "copy". If it would be repeated more time this would be added twice, threetimes, etc. so every record could differ (eg. every record would be unique in this sense). After this tuning you could copy the "old" (but renewed) table into the new table, thereafter you can delete from this tuned column every occurence of the added word (in this case "dummy"). This is the only possibility I mean. May be some more experienced gurus could suggest something better, now this was what I could imagine.

have success!

wbr

Janos
0
 
LVL 10

Expert Comment

by:kacor
ID: 9789258
but of course I am waiting your message about

Janos
0
 
LVL 10

Expert Comment

by:kacor
ID: 9789404
As now I read the complet question and the comments, I see that you wanted to be every record unique. Sorry for the mistake. I felt this was my idea :-)))))

again sorry  Janos
0
 

Author Comment

by:morees
ID: 9795480
kacor,

you know this error that I got when I tried your lolution

(Unsuccessful execution caused by a system error that precludes
successful execution of subsequent statements.
internal gds software consistency check (wrong record length (183)).")   <-- this error,

well I tried to browse through the records and the same error came up just by going through the records.
I tried doing a backup and recovery but got the same error.
So I had to recover an old database and tried your solution again. I had to add a 'dummy' column in the old table just so that the column count matched in the 2 tables and geuss what it worked.
Thanx for 'what should have been a quick solution', but took longer because of a stuffed upped table and a poor database.

Cheers
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 10

Expert Comment

by:kacor
ID: 9795507
hi morees,

the pleasure is mine

Janos
0
 
LVL 6

Expert Comment

by:BAlexandrov
ID: 9795612
just a comment on that - same can be achieved with adding an ID collumn to same table
Just add an integer collumn, set unique values and then make it primary keys.
you can achieve it with

CREATE GENERATOR gen_yourtable_id;

update yourtable
set id = gen_id(gen_yourtable_id,1)

Update in interbase is performed record by record like iterating with cursor (delete also) so this will work.

Bojidar Alexandrov
0
 

Author Comment

by:morees
ID: 9795917
BAlexandro
I tried what you said and it worked aswell.
The only difference was that I had to set the new field to be the primary field after I updated the table, and not before the update.
It generates an error if you try to make it a primary field before the update

Thanks

0
 
LVL 10

Expert Comment

by:kacor
ID: 9808968
Hi Bojidar,

I offered this solution because in his question morees said, he tried to insert a new column

wbr

Janos
0
 
LVL 6

Expert Comment

by:BAlexandrov
ID: 9809010
that's right - may be you have to read my answer again?
0
 
LVL 10

Expert Comment

by:kacor
ID: 9809080
Sorry Bojidar, it escaped my attention

wbr Janos
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

This article describes some very basic things about SQL Server filegroups.
Shadow IT is coming out of the shadows as more businesses are choosing cloud-based applications. It is now a multi-cloud world for most organizations. Simultaneously, most businesses have yet to consolidate with one cloud provider or define an offic…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now