Solved

MYSQL

Posted on 2016-09-09
11
46 Views
Last Modified: 2016-09-30
how to insert 100000 data in 1 minutes in mysql table
0
Comment
Question by:vijendra solanki
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
11 Comments
 
LVL 9

Expert Comment

by:Brian Tao
ID: 41791222
You have several options, among them here are 2 that I can think of
1) Use a program or a script to do the insert.
2) Manually prepare those 100000 SQL insert statements, paste them into whatever UI you're using and run.  Believe me, they will execute and finish in less than 1 minute. ^^
0
 
LVL 25

Accepted Solution

by:
Tomas Helgi Johannsson earned 500 total points
ID: 41793937
Hi!

To insert massive amount of data in the shortest time as possible the best way to do that is to
use batch insert / bulk data loading using your favorite programming language. A size of 1000 rows/batch is a good starting point and usually gives you the best performance although in some cases smaller or larger batch size is better depending on your data and/or workload.

There are several parameters you will need to adjust like max_allowed_packet to a size to fit your batch size.

See the following urls for more info.
http://www.geeksengine.com/database/data-manipulation/bulk-insert.php
http://dev.mysql.com/doc/refman/5.6/en/insert-speed.html
https://dev.mysql.com/doc/refman/5.6/en/optimizing-innodb-bulk-data-loading.html
http://www.journaldev.com/2494/jdbc-batch-insert-update-mysql-oracle
http://derwiki.tumblr.com/post/24490758395/loading-half-a-billion-rows-into-mysql

Regards,
    Tomas Helgi
0
 
LVL 29

Expert Comment

by:Pawan Kumar
ID: 41801222
Please use below

--

;WITH SingleDigits(Number) AS
(
    SELECT Number
    FROM (VALUES (1), (2), (3), (4), (5), (6), (7), (8),
    (9), (0)) AS X(Number)
)
,Series AS
(
    SELECT (d1.Number+1) + (10*d2.Number) + (100*d3.Number) + (1000*d4.Number) Number
    from
    SingleDigits as d1,
    SingleDigits as d2,
    SingleDigits as d3,
    SingleDigits as d4
)
SELECT * FROM Series 




--

Open in new window

0
Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

 
LVL 9

Expert Comment

by:Brian Tao
ID: 41823238
I disagree closing by accepting Pawan Kumar Khowal's comment, because WITH doesn't even work in MySQL (and the question is for MySQL).
0
 
LVL 9

Expert Comment

by:Brian Tao
ID: 41823243
I would propose to split the points to me (Brian Tao) and TomasHelgi, as the 2 solutions actually work.
0
 
LVL 29

Expert Comment

by:Pawan Kumar
ID: 41823247
Ohh my bad ! thought it was SQL Server.
0
 
LVL 29

Expert Comment

by:Pawan Kumar
ID: 41823768
Disagree. Brain Tao solutions are not good (Loop oriented and not managed) . I think points should go to TomasHelgi.
0
 
LVL 29

Expert Comment

by:Pawan Kumar
ID: 41823780
Even If i change "With" from a SELECT my method would be the best one. Even with in few seconds it will insert the 100000 records.
0
 
LVL 9

Expert Comment

by:Brian Tao
ID: 41823878
I'm fine if all 500 points goes to Tomas Helgi, but I do have something to say in response to Pawan Kumar Khowal's last 2 comments:
  1. I don't see why "loop oriented" is not good.  It solves the problem.
  2. Pawan Kumar Khowal's method doesn't work in MySQL as "WITH" is not supported, but the question is for MySQL.
  3. Not only does it not work in MySQL, but also it doesn't answer OP's question at all!  There's nothing mentioning that the OP wants to insert 0 to 9999 (or you may modify the SQL code to insert 99999, whatever) into the table. He wants to insert 100000 data, remember that? It may be 100000 names, 100000 addresses or 100000 who knows what.  The method "would be the best one"? Oh well~
0
 
LVL 29

Expert Comment

by:Pawan Kumar
ID: 41823907
Dear Brain,

Lets just close this. Technical discussions ... :)

Bye !
1

Featured Post

Forrester Webinar: xMatters Delivers 261% ROI

Guest speaker Dean Davison, Forrester Principal Consultant, explains how a Fortune 500 communication company using xMatters found these results: Achieved a 261% ROI, Experienced $753,280 in net present value benefits over 3 years and Reduced MTTR by 91% for tier 1 incidents.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

As a database administrator, you may need to audit your table(s) to determine whether the data types are optimal for your real-world data needs.  This Article is intended to be a resource for such a task. Preface The other day, I was involved …
Introduction This article is intended for those who are new to PHP error handling (https://www.experts-exchange.com/articles/11769/And-by-the-way-I-am-New-to-PHP.html).  It addresses one of the most common problems that plague beginning PHP develop…
Monitoring a network: why having a policy is the best policy? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the enormous benefits of having a policy-based approach when monitoring medium and large networks. Software utilized in this v…
Do you want to know how to make a graph with Microsoft Access? First, create a query with the data for the chart. Then make a blank form and add a chart control. This video also shows how to change what data is displayed on the graph as well as form…

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question