MySQL Infile Load Data

I have a large raw data that in the csv format (acutally a comma, not semi colon)

I have the first line and the last line should be ignored because they are header.
also it has a date field too....i do not know how to import it...

Suppose I have the folliwng data

Anyone can help me how to write the infile script?
STR,G_A-1_20100818_0001.csv
"12345678901","1234567","12345678901234","","12345678901234","","1001","1234","","100","18-08-2010 01:20:23","123456789012","123456","123456","","123456789012","","1009","","","MOB123ABC123A12345678ABC1234ACBD2","","","","","9","",""
END,G_A-1_20100818_0001.csv, 000001

Open in new window

mawinghoAsked:
Who is Participating?
 
kingjelyConnect With a Mentor Commented:

Hi, if your using mysql Query browser, you can

load data local infile
'drive:\path\filename.csv'
into table Tablename
fields terminated by ','
lines terminated by '\r\n';


You can optionally ignore lines with in this,
[IGNORE number LINES], where you put in the first and last linenumber.

Have a look here,  
http://dev.mysql.com/doc/refman/5.0/en/load-data.html

Kj



0
 
kingjelyConnect With a Mentor Commented:
Also, Mawingho,
before you load the data infile, you have to have created the table that will hold the data.

Have you done that?
0
 
mawinghoAuthor Commented:
Yes, created.
However, the number of line is different from different files....
so how can I just ignore the last line?
Also how to handle date format and also the number value?
Because the field "" will be determined as string rather than integer...
so it returns an error...
0
Easily Design & Build Your Next Website

Squarespace’s all-in-one platform gives you everything you need to express yourself creatively online, whether it is with a domain, website, or online store. Get started with your free trial today, and when ready, take 10% off your first purchase with offer code 'EXPERTS'.

 
kingjelyConnect With a Mentor Commented:
As far as im aware, You can't skip lines at the end of the file (only at the beginning).

You'll need to either alter the file before you load it or
delete the last line after you load the data infile.

For deleting the last line once it has been loaded,

If you have an auto-increment primary key column, you can:
DELETE FROM yourtable
ORDER BY primarykey_column DESC LIMIT 1;





0
 
kingjelyConnect With a Mentor Commented:

Also, you need '/' not '\' for the drive path.

load data local infile
'drive:/path/filename.csv'
into table Tablename
fields terminated by ','
lines terminated by '\r\n';
0
 
kingjelyConnect With a Mentor Commented:
Hi, second part of the next question,

So you want to remove ALL the "" around all fields as they are all Numbers or date fields, is this correct?
0
 
mawinghoAuthor Commented:
yes
0
 
mawinghoAuthor Commented:
I have created a table with all VARCHAR fields now....
I will convert them back to number later...
0
 
mawinghoAuthor Commented:
1. By the way, my friends said if I use MyISAM engine, there are 3 files will be stored in my directory...
2. also, I specified the datafile store in D:\
but I discovered that it stored in C:\ now and I do not know where it is stored...
0
 
kingjelyConnect With a Mentor Commented:

So now you have your table with all fields with ALL "",
Which you would like to remove from the table now?

You can update the fields, and replace the " with  nothing ''

UPDATE table SET FIELD1= replace(FIELD1,'"','');
UPDATE table SET FIELD2= replace(FIELD2,'"','');
UPDATE table SET FIELD3= replace(FIELD3,'"','');
UPDATE table SET FIELD4= replace(FIELD4,'"','');
UPDATE table SET FIELD5= replace(FIELD5,'"','');
UPDATE table SET FIELD6= replace(FIELD6,'"','');
UPDATE table SET FIELD7= replace(FIELD7,'"','');

0
 
kingjelyConnect With a Mentor Commented:

Try using '/'s not '\'s in the drive path.



a few comments above, comment, # 33461494


Also, you need '/' not '\' for the drive path.

load data local infile
'drive:/path/filename.csv'
into table Tablename
fields terminated by ','
lines terminated by '\r\n';
0
 
kingjelyConnect With a Mentor Commented:



2. also, I specified the datafile store in D:\
but I discovered that it stored in C:\ now and I do not know where it is stored...

You may not need

load data LOCAL infile

you may just need

load data infile - local will go to your local machine.

Also, make sure your  drive:/path/filename.csv  is using '/'  not '\'

0
 
mawinghoAuthor Commented:
but I already load all the data into my database, what should I do now?
0
 
mawinghoAuthor Commented:
how to move those data to D: now?
0
 
kingjelyConnect With a Mentor Commented:


In regards to what exactly?
If you loaded the data,

Can you select fields from yourtable?



Go to Mysql query browser, and connect to your database


SELECT * from YOURTABLE


0
 
mawinghoAuthor Commented:
yes, I can
but I do not want those data store in my C:\
0
 
kingjelyConnect With a Mentor Commented:

Why do you think it is stored on your C:\  ?

Do you know where it is stored?
Maybe you can look in

C:\bin

if you have mysql command installed

If not, you could go to

START>SEARCH and search for the file name, on your C:\ to see if it exists?

0
 
kingjelyConnect With a Mentor Commented:

if your refering to this question..

< 1. By the way, my friends said if I use MyISAM engine, there are 3 files will be stored in my directory...

I am not particularly familliar with MyISAM, I use innoDB so you may need to ask a new Question to continue, someone else may beable to help you further with this ;)

Kj

0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.