?
Solved

Importing from .csv with table lookups

Posted on 2011-09-28
4
Medium Priority
?
240 Views
Last Modified: 2012-08-13
I have a CSV file that I want to import data from into an existing MySQL database. However, during the import I need to look certain values in order to insert the records correctly into the database. For example, I need to lookup the CompanyID using the CompanyName which is found in both the DB and the CSV file. I have done stuff like this in the past for Microsoft SQL Server using SSIS but wasn't sure if there was a similar type of tool for MySQL.
0
Comment
Question by:mikedgibson
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 42

Accepted Solution

by:
dqmq earned 2000 total points
ID: 36717439
Import the CSV "as is" to a staging table.  Then populate the final table by joining the staging table to the lookup table(s) to retrieve the foreign keys.  

0
 
LVL 23

Expert Comment

by:nemws1
ID: 36717758
How are you doing the import now?  Are you using LOAD DATA LOCAL INFILE?

Usually in these cases, I'll write a short program to parse the CSV file and load it, and within this script I'll do any lookups I need to do (along with any other data munging/fixing that I might need to do, like removing extra spaces from TEXT fields or making sure numbers are formatted the way I want them).

However, you can certainly do as dqmq suggests and that might work better for you, especially if you have a very large number of rows that you need to import on a regular basis.
0
 
LVL 21

Expert Comment

by:K V
ID: 36814383
I think this can be done along with LOAD DATA itself:

consider you have b.csv with ID values and you need to populate value from another table. You can do it as follows:


load data local infile 'c:/b.csv' into table table1 lines terminated by '\n' (ID) set value=(SELECT value-to-be-set from table2 where table2.ID=table1.ID);

Refer This: http://kedar.nitty-witty.com/blog/load-delimited-data-csv-excel-into-mysql-server
0
 
LVL 21

Expert Comment

by:K V
ID: 36940763
@mikedgibson!
I'm fine with no-points but I assume solution I provided will work as well!!!
0

Featured Post

Moving data to the cloud? Find out if you’re ready

Before moving to the cloud, it is important to carefully define your db needs, plan for the migration & understand prod. environment. This wp explains how to define what you need from a cloud provider, plan for the migration & what putting a cloud solution into practice entails.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Recently I was talking with Tim Sharp, one of my colleagues from our Technical Account Manager team about MongoDB’s scalability. While doing some quick training with some of the Percona team, Tim brought something to my attention...
Lotus Notes has been used since a very long time as an e-mail client and is very popular because of it's unmatched security. In this article we are going to learn about  RRV Bucket corruption and understand various methods to Fix "RRV Bucket Corrupt…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question