Solved

How to Import Large Database to MySQL

Posted on 2011-03-23
6
1,037 Views
Last Modified: 2012-05-11
Hi!

Have a large Access database to transition to MySQL.  Have written VBA code to write the SQL statements (CREATE then INSERT) for all tables and records being converted.  Problem is the file size.  If all data is in a single file, the .sql file is 177 meg.  If I split into 57 separate tables, the largest is still 67 meg.

I'm testing locally on WAMP server.  The MySQL import will only allow 2K (that's K, not Meg) files.

How can I change the MySQL import file size limit?  Does MySQL, in general, have a limit on the file size or number of records that can be imported?

Thanks,
Bruce
0
Comment
Question by:springthorpeSoftware
  • 3
  • 2
6 Comments
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35199072
Set mySQLs variable 'max_allowed_packet' to '1024M' in your config file, restart and you will be good to go.

I am running several mySQL databases with plenty GBs. This is not the matter.
0
 

Accepted Solution

by:
springthorpeSoftware earned 0 total points
ID: 35200393
Ferrosti,

Changed the MySQL "my.ini" file to 1024M, then closed and restarted WAMP.
Still got 'file too large' error when using the MySQL GUI to import.
Was able to resolve, however, by using the MySQL DOS Console and the 'source' command.  Found out elsewhere that it will take unlimited file size.

Thanks for replying, though!

Bruce
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35200644
Sorry, I did not see that you try this using a GUI.
Which one are you trying? Web-based or some mySQL Workbench tools or similar?
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 2

Expert Comment

by:ASta
ID: 35206736
or move data record by record

MoveTable("Table1");
...
MoveTable("Table99");


proc MoveTable(tableName)
  select from access
  while not eof
    insert into mysql
    goto Next record
    Show Progess, etc
  end;
end;

Add transactions by taste

:)))
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 35207695
It would be even faster to export it as .csv and bulk insert into mySQL.
http://dev.mysql.com/doc/refman/5.0/en/mysqlimport.html
0
 

Author Closing Comment

by:springthorpeSoftware
ID: 35489819
Self-resolved.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Popularity Can Be Measured Sometimes we deal with questions of popularity, and we need a way to collect opinions from our clients.  This article shows a simple teaching example of how we might elect a favorite color by letting our clients vote for …
Password hashing is better than message digests or encryption, and you should be using it instead of message digests or encryption.  Find out why and how in this article, which supplements the original article on PHP Client Registration, Login, Logo…
This tutorial gives a high-level tour of the interface of Marketo (a marketing automation tool to help businesses track and engage prospective customers and drive them to purchase). You will see the main areas including Marketing Activities, Design …
Edureka is one of the fastest growing and most effective online learning sites.  We are here to help you succeed.

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now