Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

SQL Import Data from Text File - Validate Each Client ID before insert

Posted on 2014-02-19
3
1,188 Views
Last Modified: 2014-02-19
Hello Experts,

I have a tab delimited text file that I'll be using as a basis for periodic data import into my SQL 2005 database.  One of the fields in the text file is ClientID.  

What I need to do is loop through each row of the text file and determine whether the ClientID exists in our Clients table.  

If it does exist, it should go ahead and run a couple of different Insert queries in order to add specified fields from the text file row as a new record in certain tables.

If it does NOT exist, I need it to write out the entire row from the text file to an ImportError table.

I have all the code written for opening the text file and querying from it using some nice OPENROWSET code and I also have the Update queries written for both conditions (when ClientID is valid and when it is not valid).

What I Need is:

Guidance on how to best handle the initial step - check to see if the ClientID is valid in the Clients table and then direct the stored proc to carry out the appropriate update queries

I was thinking of the following plan:

Setting up a temporary table with 2 columns (ClientID as INT, Status as BIT)
Looping through the text file to find all distinct ClientIDs (can be more than 1 row with same ClientID)
Write out the ClientID and if valid, set status to 1, if not valid, set status to 0
Then, once the temp table is populated, use that as a To Do list of sorts and loop through each record using some sort of For Each or cursor functionality in order to read the status bit value of each row and either do the appropriate inserts (if 1) or write to error table (if 0).

I look forward to your creative ideas and appreciate your assistance!
0
Comment
Question by:dpmoney
3 Comments
 
LVL 65

Expert Comment

by:Jim Horn
ID: 39871083
>What I need to do is loop through each row of the text file and determine whether the ClientID exists in our Clients table.  

Don't loop.  What you just described can all be handled in the T-SQL MERGE statement., other than it will execute only one statement for any given action instead of 'a couple'.

But if you tack on an OUTPUT clause at the end, it'll handle subsequent statements.
0
 
LVL 69

Accepted Solution

by:
Scott Pletcher earned 500 total points
ID: 39871087
You need an UPDATE query for the main table, with an INNER JOIN, and an INSERT query for the ImportError table, with an OUTER JOIN.  [ixnay on the oopinglay!]

UPDATE mt
SET
FROM dbo.main_table mt
INNER JOIN (
    SELECT ...
    FROM OPENROWSET(...)
) AS trans ON
    trans.ClientID = mt.ClientID


INSERT INTO dbo.importError
SELECT ...
FROM OPENROWSET(...) AS trans
LEFT OUTER JOIN dbo.main_table mt ON
    mt.clientID = trans.clientID
WHERE
    mt.ClientID IS NULL
0
 

Author Closing Comment

by:dpmoney
ID: 39871290
You are indeed a prodigy!  

I was able to take your example code and map it to my SQL without issue - works like a champ!  If I could give you bonus points for the pig Latin, I would.  

Thanks again!
0

Featured Post

Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
Never store passwords in plain text or just their hash: it seems a no-brainier, but there are still plenty of people doing that. I present the why and how on this subject, offering my own real life solution that you can implement right away, bringin…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

839 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question