Solved

Duplicate inserts on mysql

Posted on 2011-03-08
12
387 Views
Last Modified: 2012-06-27
Hi,

I am trying to solve an issue with my application but can't find the way:

I have the following mysql table:

entry_id       int(11)       Autoincrement                
camp_id       int(11)                       
date       datetime                
ip       varchar(15)

Now, many times records are saved at exactly the same time with the same details but the entry_id.

So the exact record could be stored several times, with the same camp_id, date and ip. However, I would like to avoid saving it more than once.

I tried INSERT IGNORE, but as the entry_id is different it does not work. Is there a way to use INSERT IGNORE but only checking the camp_id, date and ip?

Thanks!

0
Comment
Question by:ApOG
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
  • 3
12 Comments
 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 35076988
You could do a If not exists (select * from table where whatever = whatever)
Begin
Insert into ...
End
0
 

Author Comment

by:ApOG
ID: 35077251
But does that kind of query executes entirely before another query is executed? or it is treated like a single query? Since otherwise another query may insert the data after the "internal" query and before the main one...
0
 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 35077397
Ok. So back to th original question. Insert ignore works off of unique indexes. So just define a unique index on the three columns you care about and insert ignore will handle the rest
0
Forrester Webinar: xMatters Delivers 261% ROI

Guest speaker Dean Davison, Forrester Principal Consultant, explains how a Fortune 500 communication company using xMatters found these results: Achieved a 261% ROI, Experienced $753,280 in net present value benefits over 3 years and Reduced MTTR by 91% for tier 1 incidents.

 

Author Comment

by:ApOG
ID: 35077430
I don't think that will work since nor of the columns are unique (except the entry_id, which is an autoincrement), the three columns must match the new insert in order to be ignored...
0
 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 35078952
Well if you want those three other columns to be part of a unique index (across all 3, not individually unique) you can do that and insert ignore will work. It's the only way I can think of a solution in your situation. It's clean, simple, doesn't use alot of CPU, and will function in a high transaction environment.
0
 
LVL 143

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 35082927
>Now, many times records are saved at exactly the same time with the same details but the entry_id.

are you sure it's not about the web interface "submit/save" button clicked twice, and hence the underlying code being saved twice?

>I tried INSERT IGNORE, but as the entry_id is different it does not work
INSERT IGNORE will check the primary key, indeed.

so, 2 things:
* ensure that you have the submit button disabled on it's click event in the first place
* do an UPDATE instead of INSERT, and if the update returns 0 rows, do the INSERT (you might consider to put this logic into a stored procedure to avoid to clutter the front-end code)
0
 

Author Comment

by:ApOG
ID: 35083245
Thanks angellll!

The thing is this are visitors coming to a site and apparently sometimes by some reason the code that saves the pageview is loaded twice, even more sometimes. So there's no way to "disable" the double data insert. The only option is to do it via mysql.

The solution you suggest uses 2 queries, therefore, if the update is performed and return 0 rows and then the insert is made, another insert may occur after the update, so I'll have 2 inserts...

Is there some kind of UPDATE that if there's no record found to update, it inserts it? but with a single command..??
0
 
LVL 143

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 500 total points
ID: 35084064
not if you use transaction control... however only with Innodb, not with MyISAM ..

the other option is to make the entry_id not a primary key nor a unique index, but just a plain index.
instead, create 1 single primary key/unique constraint/unique index  including all 3 columns
camp_id                        
date    
ip      

then, the insert ignore (or insert on duplicate key) shall work as you need.
you might then include a "count" column that would be 1 for the insert, and the ON DUPLICATE ... could update it to increment by 1 ... so you could account the "duplicate" views :)
0
 

Author Comment

by:ApOG
ID: 35084504
It sounds good!!

I have already setup 3 separate indexes on each column, plain indexes... Do I need to remove those indexes first?

(I use those indexes to get faster selects)
0
 
LVL 143

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 35084561
> Do I need to remove those indexes first?
no, they can stay.
0
 

Author Closing Comment

by:ApOG
ID: 35085000
Thanks!
0
 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 35085039
Wow, I guess you just said it clearer or something because that is exactly the same solution I suggested.
0

Featured Post

Webinar: MariaDB® Server 10.2: The Complete Guide

Join Percona’s Chief Evangelist, Colin Charles as he presents MariaDB Server 10.2: The Complete Guide on Tuesday, June 27, 2017 at 7:00 am PDT / 10:00 am EDT (UTC-7).

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

As a database administrator, you may need to audit your table(s) to determine whether the data types are optimal for your real-world data needs.  This Article is intended to be a resource for such a task. Preface The other day, I was involved …
In this series, we will discuss common questions received as a database Solutions Engineer at Percona. In this role, we speak with a wide array of MySQL and MongoDB users responsible for both extremely large and complex environments to smaller singl…
If you're a developer or IT admin, you’re probably tasked with managing multiple websites, servers, applications, and levels of security on a daily basis. While this can be extremely time consuming, it can also be frustrating when systems aren't wor…
Add bar graphs to Access queries using Unicode block characters. Graphs appear on every record in the color you want. Give life to numbers. Hopes this gives you ideas on visualizing your data in new ways ~ Create a calculated field in a query: …

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question