Solved

How to Import Large Database to MySQL

Posted on 2011-03-23
6
1,044 Views
Last Modified: 2012-05-11
Hi!

Have a large Access database to transition to MySQL.  Have written VBA code to write the SQL statements (CREATE then INSERT) for all tables and records being converted.  Problem is the file size.  If all data is in a single file, the .sql file is 177 meg.  If I split into 57 separate tables, the largest is still 67 meg.

I'm testing locally on WAMP server.  The MySQL import will only allow 2K (that's K, not Meg) files.

How can I change the MySQL import file size limit?  Does MySQL, in general, have a limit on the file size or number of records that can be imported?

Thanks,
Bruce
0
Comment
Question by:springthorpeSoftware
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35199072
Set mySQLs variable 'max_allowed_packet' to '1024M' in your config file, restart and you will be good to go.

I am running several mySQL databases with plenty GBs. This is not the matter.
0
 

Accepted Solution

by:
springthorpeSoftware earned 0 total points
ID: 35200393
Ferrosti,

Changed the MySQL "my.ini" file to 1024M, then closed and restarted WAMP.
Still got 'file too large' error when using the MySQL GUI to import.
Was able to resolve, however, by using the MySQL DOS Console and the 'source' command.  Found out elsewhere that it will take unlimited file size.

Thanks for replying, though!

Bruce
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35200644
Sorry, I did not see that you try this using a GUI.
Which one are you trying? Web-based or some mySQL Workbench tools or similar?
0
NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.

 
LVL 2

Expert Comment

by:ASta
ID: 35206736
or move data record by record

MoveTable("Table1");
...
MoveTable("Table99");


proc MoveTable(tableName)
  select from access
  while not eof
    insert into mysql
    goto Next record
    Show Progess, etc
  end;
end;

Add transactions by taste

:)))
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35207695
It would be even faster to export it as .csv and bulk insert into mySQL.
http://dev.mysql.com/doc/refman/5.0/en/mysqlimport.html
0
 

Author Closing Comment

by:springthorpeSoftware
ID: 35489819
Self-resolved.
0

Featured Post

[Webinar] Code, Load, and Grow

Managing multiple websites, servers, applications, and security on a daily basis? Join us for a webinar on May 25th to learn how to simplify administration and management of virtual hosts for IT admins, create a secure environment, and deploy code more effectively and frequently.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Foreword This article was written many years ago, in the days when PHP supported the MySQL extension (http://php.net/manual/en/function.mysql-connect.php).  Today (http://php.net/manual/en/migration70.removed-exts-sapis.php) you would not use MySQL…
Containers like Docker and Rocket are getting more popular every day. In my conversations with customers, they consistently ask what containers are and how they can use them in their environment. If you’re as curious as most people, read on. . .
A short tutorial showing how to set up an email signature in Outlook on the Web (previously known as OWA). For free email signatures designs, visit https://www.mail-signatures.com/articles/signature-templates/?sts=6651 If you want to manage em…

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question