Solved

How to Import Large Database to MySQL

Posted on 2011-03-23
6
1,038 Views
Last Modified: 2012-05-11
Hi!

Have a large Access database to transition to MySQL.  Have written VBA code to write the SQL statements (CREATE then INSERT) for all tables and records being converted.  Problem is the file size.  If all data is in a single file, the .sql file is 177 meg.  If I split into 57 separate tables, the largest is still 67 meg.

I'm testing locally on WAMP server.  The MySQL import will only allow 2K (that's K, not Meg) files.

How can I change the MySQL import file size limit?  Does MySQL, in general, have a limit on the file size or number of records that can be imported?

Thanks,
Bruce
0
Comment
Question by:springthorpeSoftware
  • 3
  • 2
6 Comments
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35199072
Set mySQLs variable 'max_allowed_packet' to '1024M' in your config file, restart and you will be good to go.

I am running several mySQL databases with plenty GBs. This is not the matter.
0
 

Accepted Solution

by:
springthorpeSoftware earned 0 total points
ID: 35200393
Ferrosti,

Changed the MySQL "my.ini" file to 1024M, then closed and restarted WAMP.
Still got 'file too large' error when using the MySQL GUI to import.
Was able to resolve, however, by using the MySQL DOS Console and the 'source' command.  Found out elsewhere that it will take unlimited file size.

Thanks for replying, though!

Bruce
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35200644
Sorry, I did not see that you try this using a GUI.
Which one are you trying? Web-based or some mySQL Workbench tools or similar?
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 2

Expert Comment

by:ASta
ID: 35206736
or move data record by record

MoveTable("Table1");
...
MoveTable("Table99");


proc MoveTable(tableName)
  select from access
  while not eof
    insert into mysql
    goto Next record
    Show Progess, etc
  end;
end;

Add transactions by taste

:)))
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35207695
It would be even faster to export it as .csv and bulk insert into mySQL.
http://dev.mysql.com/doc/refman/5.0/en/mysqlimport.html
0
 

Author Closing Comment

by:springthorpeSoftware
ID: 35489819
Self-resolved.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A lot of articles have been written on splitting mysqldump and grabbing the required tables. A long while back, when Shlomi (http://code.openark.org/blog/mysql/on-restoring-a-single-table-from-mysqldump) had suggested a “sed” way, I actually shell …
I have been using r1soft Continuous Data Protection (http://www.r1soft.com/linux-cdp/) for many years now with the mySQL Addon and wanted to share a trick I have used several times. For those of us that don't have the luxury of using all transact…
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …
Two types of users will appreciate AOMEI Backupper Pro: 1 - Those with PCIe drives (and haven't found cloning software that works on them). 2 - Those who want a fast clone of their boot drive (no re-boots needed) and it can clone your drive wh…

810 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question