Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

How to Import Large Database to MySQL

Posted on 2011-03-23
6
Medium Priority
?
1,053 Views
Last Modified: 2012-05-11
Hi!

Have a large Access database to transition to MySQL.  Have written VBA code to write the SQL statements (CREATE then INSERT) for all tables and records being converted.  Problem is the file size.  If all data is in a single file, the .sql file is 177 meg.  If I split into 57 separate tables, the largest is still 67 meg.

I'm testing locally on WAMP server.  The MySQL import will only allow 2K (that's K, not Meg) files.

How can I change the MySQL import file size limit?  Does MySQL, in general, have a limit on the file size or number of records that can be imported?

Thanks,
Bruce
0
Comment
Question by:springthorpeSoftware
  • 3
  • 2
6 Comments
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35199072
Set mySQLs variable 'max_allowed_packet' to '1024M' in your config file, restart and you will be good to go.

I am running several mySQL databases with plenty GBs. This is not the matter.
0
 

Accepted Solution

by:
springthorpeSoftware earned 0 total points
ID: 35200393
Ferrosti,

Changed the MySQL "my.ini" file to 1024M, then closed and restarted WAMP.
Still got 'file too large' error when using the MySQL GUI to import.
Was able to resolve, however, by using the MySQL DOS Console and the 'source' command.  Found out elsewhere that it will take unlimited file size.

Thanks for replying, though!

Bruce
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35200644
Sorry, I did not see that you try this using a GUI.
Which one are you trying? Web-based or some mySQL Workbench tools or similar?
0
Veeam Task Manager for Hyper-V

Task Manager for Hyper-V provides critical information that allows you to monitor Hyper-V performance by displaying real-time views of CPU and memory at the individual VM-level, so you can quickly identify which VMs are using host resources.

 
LVL 2

Expert Comment

by:ASta
ID: 35206736
or move data record by record

MoveTable("Table1");
...
MoveTable("Table99");


proc MoveTable(tableName)
  select from access
  while not eof
    insert into mysql
    goto Next record
    Show Progess, etc
  end;
end;

Add transactions by taste

:)))
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35207695
It would be even faster to export it as .csv and bulk insert into mySQL.
http://dev.mysql.com/doc/refman/5.0/en/mysqlimport.html
0
 

Author Closing Comment

by:springthorpeSoftware
ID: 35489819
Self-resolved.
0

Featured Post

How to Use the Help Bell

Need to boost the visibility of your question for solutions? Use the Experts Exchange Help Bell to confirm priority levels and contact subject-matter experts for question attention.  Check out this how-to article for more information.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this series, we will discuss common questions received as a database Solutions Engineer at Percona. In this role, we speak with a wide array of MySQL and MongoDB users responsible for both extremely large and complex environments to smaller singl…
In this article, I’ll talk about multi-threaded slave statistics printed in MySQL error log file.
In this video, Percona Solution Engineer Dimitri Vanoverbeke discusses why you want to use at least three nodes in a database cluster. To discuss how Percona Consulting can help with your design and architecture needs for your database and infras…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
Suggested Courses

877 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question