Solved

Importing from .csv with table lookups

Posted on 2011-09-28
4
226 Views
Last Modified: 2012-08-13
I have a CSV file that I want to import data from into an existing MySQL database. However, during the import I need to look certain values in order to insert the records correctly into the database. For example, I need to lookup the CompanyID using the CompanyName which is found in both the DB and the CSV file. I have done stuff like this in the past for Microsoft SQL Server using SSIS but wasn't sure if there was a similar type of tool for MySQL.
0
Comment
Question by:mikedgibson
  • 2
4 Comments
 
LVL 42

Accepted Solution

by:
dqmq earned 500 total points
ID: 36717439
Import the CSV "as is" to a staging table.  Then populate the final table by joining the staging table to the lookup table(s) to retrieve the foreign keys.  

0
 
LVL 23

Expert Comment

by:nemws1
ID: 36717758
How are you doing the import now?  Are you using LOAD DATA LOCAL INFILE?

Usually in these cases, I'll write a short program to parse the CSV file and load it, and within this script I'll do any lookups I need to do (along with any other data munging/fixing that I might need to do, like removing extra spaces from TEXT fields or making sure numbers are formatted the way I want them).

However, you can certainly do as dqmq suggests and that might work better for you, especially if you have a very large number of rows that you need to import on a regular basis.
0
 
LVL 21

Expert Comment

by:theGhost_k8
ID: 36814383
I think this can be done along with LOAD DATA itself:

consider you have b.csv with ID values and you need to populate value from another table. You can do it as follows:


load data local infile 'c:/b.csv' into table table1 lines terminated by '\n' (ID) set value=(SELECT value-to-be-set from table2 where table2.ID=table1.ID);

Refer This: http://kedar.nitty-witty.com/blog/load-delimited-data-csv-excel-into-mysql-server
0
 
LVL 21

Expert Comment

by:theGhost_k8
ID: 36940763
@mikedgibson!
I'm fine with no-points but I assume solution I provided will work as well!!!
0

Featured Post

Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Linking dictionary and columns together 2 50
Determine next b-weekly date 12 56
Need help with a query 6 55
report returning null 21 52
This article describes some very basic things about SQL Server filegroups.
I annotated my article on ransomware somewhat extensively, but I keep adding new references and wanted to put a link to the reference library.  Despite all the reference tools I have on hand, it was not easy to find a way to do this easily. I finall…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now