[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 478
  • Last Modified:

Importing large text files into MYSQL

I am importing a series of text files given to me by the local chamber of commerce.  There are about 30 files.  Some are very small and some are very, very large.  The small ones import with no issue but the very, very large ones are timing out at 650 seconds each time.  Not sure if this is a limit imposed by MYSQL or if there is a way to batch the file so it only processes so many records at one time and can chug through these large files.
0
pwbigdawg
Asked:
pwbigdawg
1 Solution
 
pwbigdawgAuthor Commented:
Exact error is Error code 2013. Lost connection to MySql server during Query.  The duration each time is 600.500 sec.

Query being used is:

LOAD DATA INFILE 'C:\\DIRNAME\\FILENAME.TXT'
    INTO TABLE TABLENAME
      fields terminated BY '\t'
0
 
johanntagleCommented:
It looks like you have a wait_timeout setting of 600 seconds.  You can confirm this by doing a "show variables like 'wait_timeout';"

You can change it just for your current session via something like "set wait_timeout=1200;"
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now