Solved

Importing from .csv with table lookups

Posted on 2011-09-28
4
231 Views
Last Modified: 2012-08-13
I have a CSV file that I want to import data from into an existing MySQL database. However, during the import I need to look certain values in order to insert the records correctly into the database. For example, I need to lookup the CompanyID using the CompanyName which is found in both the DB and the CSV file. I have done stuff like this in the past for Microsoft SQL Server using SSIS but wasn't sure if there was a similar type of tool for MySQL.
0
Comment
Question by:mikedgibson
  • 2
4 Comments
 
LVL 42

Accepted Solution

by:
dqmq earned 500 total points
ID: 36717439
Import the CSV "as is" to a staging table.  Then populate the final table by joining the staging table to the lookup table(s) to retrieve the foreign keys.  

0
 
LVL 23

Expert Comment

by:nemws1
ID: 36717758
How are you doing the import now?  Are you using LOAD DATA LOCAL INFILE?

Usually in these cases, I'll write a short program to parse the CSV file and load it, and within this script I'll do any lookups I need to do (along with any other data munging/fixing that I might need to do, like removing extra spaces from TEXT fields or making sure numbers are formatted the way I want them).

However, you can certainly do as dqmq suggests and that might work better for you, especially if you have a very large number of rows that you need to import on a regular basis.
0
 
LVL 21

Expert Comment

by:theGhost_k8
ID: 36814383
I think this can be done along with LOAD DATA itself:

consider you have b.csv with ID values and you need to populate value from another table. You can do it as follows:


load data local infile 'c:/b.csv' into table table1 lines terminated by '\n' (ID) set value=(SELECT value-to-be-set from table2 where table2.ID=table1.ID);

Refer This: http://kedar.nitty-witty.com/blog/load-delimited-data-csv-excel-into-mysql-server
0
 
LVL 21

Expert Comment

by:theGhost_k8
ID: 36940763
@mikedgibson!
I'm fine with no-points but I assume solution I provided will work as well!!!
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains all about SQL Server Piecemeal Restore with examples in step by step manner.
Password hashing is better than message digests or encryption, and you should be using it instead of message digests or encryption.  Find out why and how in this article, which supplements the original article on PHP Client Registration, Login, Logo…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

920 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now