MySQL Infile Load Data

I have a large raw data that in the csv format (acutally a comma, not semi colon)

I have the first line and the last line should be ignored because they are header.
also it has a date field too....i do not know how to import it...

Suppose I have the folliwng data

Anyone can help me how to write the infile script?
STR,G_A-1_20100818_0001.csv
"12345678901","1234567","12345678901234","","12345678901234","","1001","1234","","100","18-08-2010 01:20:23","123456789012","123456","123456","","123456789012","","1009","","","MOB123ABC123A12345678ABC1234ACBD2","","","","","9","",""
END,G_A-1_20100818_0001.csv, 000001

Open in new window

mawinghoAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

kingjelyCommented:

Hi, if your using mysql Query browser, you can

load data local infile
'drive:\path\filename.csv'
into table Tablename
fields terminated by ','
lines terminated by '\r\n';


You can optionally ignore lines with in this,
[IGNORE number LINES], where you put in the first and last linenumber.

Have a look here,  
http://dev.mysql.com/doc/refman/5.0/en/load-data.html

Kj



0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
kingjelyCommented:
Also, Mawingho,
before you load the data infile, you have to have created the table that will hold the data.

Have you done that?
0
mawinghoAuthor Commented:
Yes, created.
However, the number of line is different from different files....
so how can I just ignore the last line?
Also how to handle date format and also the number value?
Because the field "" will be determined as string rather than integer...
so it returns an error...
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

kingjelyCommented:
As far as im aware, You can't skip lines at the end of the file (only at the beginning).

You'll need to either alter the file before you load it or
delete the last line after you load the data infile.

For deleting the last line once it has been loaded,

If you have an auto-increment primary key column, you can:
DELETE FROM yourtable
ORDER BY primarykey_column DESC LIMIT 1;





0
kingjelyCommented:

Also, you need '/' not '\' for the drive path.

load data local infile
'drive:/path/filename.csv'
into table Tablename
fields terminated by ','
lines terminated by '\r\n';
0
kingjelyCommented:
Hi, second part of the next question,

So you want to remove ALL the "" around all fields as they are all Numbers or date fields, is this correct?
0
mawinghoAuthor Commented:
yes
0
mawinghoAuthor Commented:
I have created a table with all VARCHAR fields now....
I will convert them back to number later...
0
mawinghoAuthor Commented:
1. By the way, my friends said if I use MyISAM engine, there are 3 files will be stored in my directory...
2. also, I specified the datafile store in D:\
but I discovered that it stored in C:\ now and I do not know where it is stored...
0
kingjelyCommented:

So now you have your table with all fields with ALL "",
Which you would like to remove from the table now?

You can update the fields, and replace the " with  nothing ''

UPDATE table SET FIELD1= replace(FIELD1,'"','');
UPDATE table SET FIELD2= replace(FIELD2,'"','');
UPDATE table SET FIELD3= replace(FIELD3,'"','');
UPDATE table SET FIELD4= replace(FIELD4,'"','');
UPDATE table SET FIELD5= replace(FIELD5,'"','');
UPDATE table SET FIELD6= replace(FIELD6,'"','');
UPDATE table SET FIELD7= replace(FIELD7,'"','');

0
kingjelyCommented:

Try using '/'s not '\'s in the drive path.



a few comments above, comment, # 33461494


Also, you need '/' not '\' for the drive path.

load data local infile
'drive:/path/filename.csv'
into table Tablename
fields terminated by ','
lines terminated by '\r\n';
0
kingjelyCommented:



2. also, I specified the datafile store in D:\
but I discovered that it stored in C:\ now and I do not know where it is stored...

You may not need

load data LOCAL infile

you may just need

load data infile - local will go to your local machine.

Also, make sure your  drive:/path/filename.csv  is using '/'  not '\'

0
mawinghoAuthor Commented:
but I already load all the data into my database, what should I do now?
0
mawinghoAuthor Commented:
how to move those data to D: now?
0
kingjelyCommented:


In regards to what exactly?
If you loaded the data,

Can you select fields from yourtable?



Go to Mysql query browser, and connect to your database


SELECT * from YOURTABLE


0
mawinghoAuthor Commented:
yes, I can
but I do not want those data store in my C:\
0
kingjelyCommented:

Why do you think it is stored on your C:\  ?

Do you know where it is stored?
Maybe you can look in

C:\bin

if you have mysql command installed

If not, you could go to

START>SEARCH and search for the file name, on your C:\ to see if it exists?

0
kingjelyCommented:

if your refering to this question..

< 1. By the way, my friends said if I use MyISAM engine, there are 3 files will be stored in my directory...

I am not particularly familliar with MyISAM, I use innoDB so you may need to ask a new Question to continue, someone else may beable to help you further with this ;)

Kj

0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
MySQL Server

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.