Solved

MS SQL Stored Procedure to validate and import CSV data into table

Posted on 2013-06-11
6
1,876 Views
Last Modified: 2013-07-03
Hello @Experts...

I need to write a SQL Stored Procedure, that does the following ..

Using Server.MapPath (ASP) selects a file that has been uploaded to the server.
Validates that the files is a CSV format.
Validates that the columns are correct (Column Headings) and as expected - name / col2 / col3 etc
Imports the data into a table.

If the file is not a CSV format and error code should be returned.
If the column headers are not correct, an error code should be returned.

That's the basis of the SP, probably the most complicated part (and the part that I would appreciate assistance with please)

It does other stuff too, like archiving the historic data, but I think I can cope with the additional stuff.

I would appreciate some help with the above though please.

FYI, my application is a classic ASP VBScript app.

Thank you
0
Comment
Question by:garethtnash
  • 3
  • 3
6 Comments
 
LVL 13

Expert Comment

by:jonnidip
ID: 39237528
Are you sure you want to validate the CSV from the SP?
Wouldn't it be better to do it from your application?

Anyway, if the SP should do this, you may read the file into a variable and split it into rows.
Then you can check if the first row has the correct number of values (column names).
After that you may want to split the rest of the rows into single fields.

You can use this function to split a string (varchar) into a table-value:
CREATE FUNCTION [dbo].[Split](
    @sInputList VARCHAR(MAX) -- List of delimited items
  , @sDelimiter CHAR(1) = ',' -- delimiter that separates items
) RETURNS @List TABLE (item VARCHAR(MAX))

BEGIN
DECLARE @sItem VARCHAR(MAX)
WHILE CHARINDEX(@sDelimiter,@sInputList,0) <> 0
 BEGIN
 SELECT
  @sItem=RTRIM(LTRIM(SUBSTRING(@sInputList,1,CHARINDEX(@sDelimiter,@sInputList,0)-1))),
  @sInputList=RTRIM(LTRIM(SUBSTRING(@sInputList,CHARINDEX(@sDelimiter,@sInputList,0)+LEN(@sDelimiter),LEN(@sInputList))))
 
 IF LEN(@sItem) > 0
  INSERT INTO @List SELECT @sItem
 END

IF LEN(@sInputList) > 0
 INSERT INTO @List SELECT @sInputList -- Put the last item in
RETURN
END

Open in new window


I have deployed this function in my master database, so I can call it by writing:
select * from master.dbo.split('test,test,test', ',')

Open in new window

and the result is a table with a single column and (in this example) 3 rows.


Regards.
0
 

Author Comment

by:garethtnash
ID: 39237599
Thanks jonnidip,

Is there anyway of validating that the column headers are as expected, forinstance I may be requiring

Name
Size
RRP

but might get

Name
Size
R.R.P

in which case the SP would not fire..

Thanks
0
 

Author Comment

by:garethtnash
ID: 39237600
execute .. :)
0
Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

 
LVL 13

Accepted Solution

by:
jonnidip earned 500 total points
ID: 39237696
Since the Split function returns a Table, you may check if all the values are in another table (virtual or physical), or you can write your values as a csv string into your SP code.
For example:
declare @columnList varchar(max) = 'Name,Size,R.R.P'

select * from master.dbo.split(@columnList, ',')
where item not in (select * from master.dbo.split('Name,Size,RRP', ','))

Open in new window

This will return:
item
-----------------
R.R.P

(1 row(s) affected)

Open in new window


You may then want to check if the count of rows is = 0, that means that all the fields you have passed are contained in the expected column list.


Regards.
0
 

Author Closing Comment

by:garethtnash
ID: 39296905
Fantastic Thank you
0
 
LVL 13

Expert Comment

by:jonnidip
ID: 39296913
You're welcome!
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

If you have heard of RFC822 date formats, they can be quite a challenge in SQL Server. RFC822 is an Internet standard format for email message headers, including all dates within those headers. The RFC822 protocols are available in detail at:   ht…
This article shows gives you an overview on SQL Server 2016 row level security. You will also get to know the usages of row-level-security and how it works
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Viewers will learn how to use the INSERT statement to insert data into their tables. It will also introduce the NULL statement, to show them what happens when no value is giving for any given column.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question