Solved

MS SQL Stored Procedure to validate and import CSV data into table

Posted on 2013-06-11
6
1,899 Views
Last Modified: 2013-07-03
Hello @Experts...

I need to write a SQL Stored Procedure, that does the following ..

Using Server.MapPath (ASP) selects a file that has been uploaded to the server.
Validates that the files is a CSV format.
Validates that the columns are correct (Column Headings) and as expected - name / col2 / col3 etc
Imports the data into a table.

If the file is not a CSV format and error code should be returned.
If the column headers are not correct, an error code should be returned.

That's the basis of the SP, probably the most complicated part (and the part that I would appreciate assistance with please)

It does other stuff too, like archiving the historic data, but I think I can cope with the additional stuff.

I would appreciate some help with the above though please.

FYI, my application is a classic ASP VBScript app.

Thank you
0
Comment
Question by:garethtnash
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 
LVL 13

Expert Comment

by:jonnidip
ID: 39237528
Are you sure you want to validate the CSV from the SP?
Wouldn't it be better to do it from your application?

Anyway, if the SP should do this, you may read the file into a variable and split it into rows.
Then you can check if the first row has the correct number of values (column names).
After that you may want to split the rest of the rows into single fields.

You can use this function to split a string (varchar) into a table-value:
CREATE FUNCTION [dbo].[Split](
    @sInputList VARCHAR(MAX) -- List of delimited items
  , @sDelimiter CHAR(1) = ',' -- delimiter that separates items
) RETURNS @List TABLE (item VARCHAR(MAX))

BEGIN
DECLARE @sItem VARCHAR(MAX)
WHILE CHARINDEX(@sDelimiter,@sInputList,0) <> 0
 BEGIN
 SELECT
  @sItem=RTRIM(LTRIM(SUBSTRING(@sInputList,1,CHARINDEX(@sDelimiter,@sInputList,0)-1))),
  @sInputList=RTRIM(LTRIM(SUBSTRING(@sInputList,CHARINDEX(@sDelimiter,@sInputList,0)+LEN(@sDelimiter),LEN(@sInputList))))
 
 IF LEN(@sItem) > 0
  INSERT INTO @List SELECT @sItem
 END

IF LEN(@sInputList) > 0
 INSERT INTO @List SELECT @sInputList -- Put the last item in
RETURN
END

Open in new window


I have deployed this function in my master database, so I can call it by writing:
select * from master.dbo.split('test,test,test', ',')

Open in new window

and the result is a table with a single column and (in this example) 3 rows.


Regards.
0
 

Author Comment

by:garethtnash
ID: 39237599
Thanks jonnidip,

Is there anyway of validating that the column headers are as expected, forinstance I may be requiring

Name
Size
RRP

but might get

Name
Size
R.R.P

in which case the SP would not fire..

Thanks
0
 

Author Comment

by:garethtnash
ID: 39237600
execute .. :)
0
Free eBook: Backup on AWS

Everything you need to know about backup and disaster recovery with AWS, for FREE!

 
LVL 13

Accepted Solution

by:
jonnidip earned 500 total points
ID: 39237696
Since the Split function returns a Table, you may check if all the values are in another table (virtual or physical), or you can write your values as a csv string into your SP code.
For example:
declare @columnList varchar(max) = 'Name,Size,R.R.P'

select * from master.dbo.split(@columnList, ',')
where item not in (select * from master.dbo.split('Name,Size,RRP', ','))

Open in new window

This will return:
item
-----------------
R.R.P

(1 row(s) affected)

Open in new window


You may then want to check if the count of rows is = 0, that means that all the fields you have passed are contained in the expected column list.


Regards.
0
 

Author Closing Comment

by:garethtnash
ID: 39296905
Fantastic Thank you
0
 
LVL 13

Expert Comment

by:jonnidip
ID: 39296913
You're welcome!
0

Featured Post

Is Your DevOps Pipeline Leaking?

Is your CI/CD pipeline a hodge-podge of randomly connected tools? You’ve likely got a tool to fix one problem & then a different tool to fix another, resulting in a cluster of tools with overlapping functionality. Learn how to optimize your pipeline with Gartner's recommendations

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How to leverage one TLS certificate to encrypt Microsoft SQL traffic and Remote Desktop Services, versus creating multiple tickets for the same server.
For both online and offline retail, the cross-channel business is the most recent pattern in the B2C trade space.
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed

729 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question