Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

How to Import Large Database to MySQL

Posted on 2011-03-23
6
Medium Priority
?
1,050 Views
Last Modified: 2012-05-11
Hi!

Have a large Access database to transition to MySQL.  Have written VBA code to write the SQL statements (CREATE then INSERT) for all tables and records being converted.  Problem is the file size.  If all data is in a single file, the .sql file is 177 meg.  If I split into 57 separate tables, the largest is still 67 meg.

I'm testing locally on WAMP server.  The MySQL import will only allow 2K (that's K, not Meg) files.

How can I change the MySQL import file size limit?  Does MySQL, in general, have a limit on the file size or number of records that can be imported?

Thanks,
Bruce
0
Comment
Question by:springthorpeSoftware
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35199072
Set mySQLs variable 'max_allowed_packet' to '1024M' in your config file, restart and you will be good to go.

I am running several mySQL databases with plenty GBs. This is not the matter.
0
 

Accepted Solution

by:
springthorpeSoftware earned 0 total points
ID: 35200393
Ferrosti,

Changed the MySQL "my.ini" file to 1024M, then closed and restarted WAMP.
Still got 'file too large' error when using the MySQL GUI to import.
Was able to resolve, however, by using the MySQL DOS Console and the 'source' command.  Found out elsewhere that it will take unlimited file size.

Thanks for replying, though!

Bruce
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35200644
Sorry, I did not see that you try this using a GUI.
Which one are you trying? Web-based or some mySQL Workbench tools or similar?
0
What Is Blockchain Technology?

Blockchain is a technology that underpins the success of Bitcoin and other digital currencies, but it has uses far beyond finance. Learn how blockchain works and why it is proving disruptive to other areas of IT.

 
LVL 2

Expert Comment

by:ASta
ID: 35206736
or move data record by record

MoveTable("Table1");
...
MoveTable("Table99");


proc MoveTable(tableName)
  select from access
  while not eof
    insert into mysql
    goto Next record
    Show Progess, etc
  end;
end;

Add transactions by taste

:)))
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35207695
It would be even faster to export it as .csv and bulk insert into mySQL.
http://dev.mysql.com/doc/refman/5.0/en/mysqlimport.html
0
 

Author Closing Comment

by:springthorpeSoftware
ID: 35489819
Self-resolved.
0

Featured Post

Moving data to the cloud? Find out if you’re ready

Before moving to the cloud, it is important to carefully define your db needs, plan for the migration & understand prod. environment. This wp explains how to define what you need from a cloud provider, plan for the migration & what putting a cloud solution into practice entails.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This post contains step-by-step instructions for setting up alerting in Percona Monitoring and Management (PMM) using Grafana.
In this blog post, we’ll look at how ClickHouse performs in a general analytical workload using the star schema benchmark test.
In this video, Percona Solution Engineer Dimitri Vanoverbeke discusses why you want to use at least three nodes in a database cluster. To discuss how Percona Consulting can help with your design and architecture needs for your database and infras…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
Suggested Courses

722 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question