Solved

Importing from .csv with table lookups

Posted on 2011-09-28
4
235 Views
Last Modified: 2012-08-13
I have a CSV file that I want to import data from into an existing MySQL database. However, during the import I need to look certain values in order to insert the records correctly into the database. For example, I need to lookup the CompanyID using the CompanyName which is found in both the DB and the CSV file. I have done stuff like this in the past for Microsoft SQL Server using SSIS but wasn't sure if there was a similar type of tool for MySQL.
0
Comment
Question by:mikedgibson
  • 2
4 Comments
 
LVL 42

Accepted Solution

by:
dqmq earned 500 total points
ID: 36717439
Import the CSV "as is" to a staging table.  Then populate the final table by joining the staging table to the lookup table(s) to retrieve the foreign keys.  

0
 
LVL 23

Expert Comment

by:nemws1
ID: 36717758
How are you doing the import now?  Are you using LOAD DATA LOCAL INFILE?

Usually in these cases, I'll write a short program to parse the CSV file and load it, and within this script I'll do any lookups I need to do (along with any other data munging/fixing that I might need to do, like removing extra spaces from TEXT fields or making sure numbers are formatted the way I want them).

However, you can certainly do as dqmq suggests and that might work better for you, especially if you have a very large number of rows that you need to import on a regular basis.
0
 
LVL 21

Expert Comment

by:theGhost_k8
ID: 36814383
I think this can be done along with LOAD DATA itself:

consider you have b.csv with ID values and you need to populate value from another table. You can do it as follows:


load data local infile 'c:/b.csv' into table table1 lines terminated by '\n' (ID) set value=(SELECT value-to-be-set from table2 where table2.ID=table1.ID);

Refer This: http://kedar.nitty-witty.com/blog/load-delimited-data-csv-excel-into-mysql-server
0
 
LVL 21

Expert Comment

by:theGhost_k8
ID: 36940763
@mikedgibson!
I'm fine with no-points but I assume solution I provided will work as well!!!
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
Many companies are looking to get out of the datacenter business and to services like Microsoft Azure to provide Infrastructure as a Service (IaaS) solutions for legacy client server workloads, rather than continuing to make capital investments in h…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question