Solved

MS SQL Stored Procedure to validate and import CSV data into table

Posted on 2013-06-11
6
1,860 Views
Last Modified: 2013-07-03
Hello @Experts...

I need to write a SQL Stored Procedure, that does the following ..

Using Server.MapPath (ASP) selects a file that has been uploaded to the server.
Validates that the files is a CSV format.
Validates that the columns are correct (Column Headings) and as expected - name / col2 / col3 etc
Imports the data into a table.

If the file is not a CSV format and error code should be returned.
If the column headers are not correct, an error code should be returned.

That's the basis of the SP, probably the most complicated part (and the part that I would appreciate assistance with please)

It does other stuff too, like archiving the historic data, but I think I can cope with the additional stuff.

I would appreciate some help with the above though please.

FYI, my application is a classic ASP VBScript app.

Thank you
0
Comment
Question by:garethtnash
  • 3
  • 3
6 Comments
 
LVL 13

Expert Comment

by:jonnidip
ID: 39237528
Are you sure you want to validate the CSV from the SP?
Wouldn't it be better to do it from your application?

Anyway, if the SP should do this, you may read the file into a variable and split it into rows.
Then you can check if the first row has the correct number of values (column names).
After that you may want to split the rest of the rows into single fields.

You can use this function to split a string (varchar) into a table-value:
CREATE FUNCTION [dbo].[Split](
    @sInputList VARCHAR(MAX) -- List of delimited items
  , @sDelimiter CHAR(1) = ',' -- delimiter that separates items
) RETURNS @List TABLE (item VARCHAR(MAX))

BEGIN
DECLARE @sItem VARCHAR(MAX)
WHILE CHARINDEX(@sDelimiter,@sInputList,0) <> 0
 BEGIN
 SELECT
  @sItem=RTRIM(LTRIM(SUBSTRING(@sInputList,1,CHARINDEX(@sDelimiter,@sInputList,0)-1))),
  @sInputList=RTRIM(LTRIM(SUBSTRING(@sInputList,CHARINDEX(@sDelimiter,@sInputList,0)+LEN(@sDelimiter),LEN(@sInputList))))
 
 IF LEN(@sItem) > 0
  INSERT INTO @List SELECT @sItem
 END

IF LEN(@sInputList) > 0
 INSERT INTO @List SELECT @sInputList -- Put the last item in
RETURN
END

Open in new window


I have deployed this function in my master database, so I can call it by writing:
select * from master.dbo.split('test,test,test', ',')

Open in new window

and the result is a table with a single column and (in this example) 3 rows.


Regards.
0
 

Author Comment

by:garethtnash
ID: 39237599
Thanks jonnidip,

Is there anyway of validating that the column headers are as expected, forinstance I may be requiring

Name
Size
RRP

but might get

Name
Size
R.R.P

in which case the SP would not fire..

Thanks
0
 

Author Comment

by:garethtnash
ID: 39237600
execute .. :)
0
Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 
LVL 13

Accepted Solution

by:
jonnidip earned 500 total points
ID: 39237696
Since the Split function returns a Table, you may check if all the values are in another table (virtual or physical), or you can write your values as a csv string into your SP code.
For example:
declare @columnList varchar(max) = 'Name,Size,R.R.P'

select * from master.dbo.split(@columnList, ',')
where item not in (select * from master.dbo.split('Name,Size,RRP', ','))

Open in new window

This will return:
item
-----------------
R.R.P

(1 row(s) affected)

Open in new window


You may then want to check if the count of rows is = 0, that means that all the fields you have passed are contained in the expected column list.


Regards.
0
 

Author Closing Comment

by:garethtnash
ID: 39296905
Fantastic Thank you
0
 
LVL 13

Expert Comment

by:jonnidip
ID: 39296913
You're welcome!
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article we will get to know that how can we recover deleted data if it happens accidently. We really can recover deleted rows if we know the time when data is deleted by using the transaction log.
JSON is being used more and more, besides XML, and you surely wanted to parse the data out into SQL instead of doing it in some Javascript. The below function in SQL Server can do the job for you, returning a quick table with the parsed data.
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
Via a live example, show how to setup several different housekeeping processes for a SQL Server.

910 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now