Solved

Importing flat file dates in ssis

Posted on 2011-02-16
9
1,518 Views
Last Modified: 2013-11-10
Urgent reply needed.  ...   Importing date columns from a flat file in this format '11/12/2005' however sometimes the date field is blank. The data is coming from the flat file into a table as char(10) then copied over to a table on SQL 2005. (this process worked with DTS) I am now using SSIS 2005 and I am getting an error in ssis and failing.  I've tried to change the oledb destination data type to string to match the char but getting invalid value for cast specification - then tried to change the oledb destination column to datetime instead of string and error again with value could not be converted.  Now, it is failing on some rows because I set the column to ignore error/truncation and some values were passed - so I am thinking that blanks might be the cause? Added an error output to a textfile destination on the specific column and received this in the error column ? -10716076891148 .  I've spent days on this and I have a great headache .. file is needed urgent.  thnx much. forever grateful to whoever can resolve .
0
Comment
Question by:bar0822
9 Comments
 
LVL 39

Assisted Solution

by:lcohan
lcohan earned 167 total points
ID: 34911079
<<sometimes the date field is blank. The data is coming from the flat file into a table as char(10) then copied over to a table on SQL 2005.>>

Does the table where you inport date field first allow NULLs? Is this your DTS problem or when you copy data from that table to a SQL 2005 table? check to make sure that all fields have same data type and CAST or CONVERT your date to that data type and FORMAT! it is important to preserve the dd/mm/yyyy in all of them and use appropriate date format for that.
0
 

Author Comment

by:bar0822
ID: 34911569
yes the table allows nulls. Looks like all the fields that I can see in the flat file are mm/dd/yyyy and there are empty strings.  Whatever I do I am getting errors.
0
 
LVL 30

Expert Comment

by:Reza Rad
ID: 34915024
did you used any conversion trasnform after flat file source? or did you used any other trasnform?
if yes, try to configure error output on transforms and redirect error output to another flat file destination.
if no, try to configure error output on flat file source and redirect error output to another flat file destination.

then you can run the package and find out rows which caused error and troubleshoot them.

0
 

Author Comment

by:bar0822
ID: 34916883
I redirected the rows to another flat file destination from the flat file source and all the rows were directed to the flat file source including the blanks and the dates which were mm/dd/yyyy and the error code -10716076891148
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 22

Assisted Solution

by:8080_Diver
8080_Diver earned 333 total points
ID: 34918646
You might want to try using ISDATE(yourcolumn, NULL) to preclude the error on all spaces.  To solve the conversion problem, I would use a CASE statement that checks for NULL and, if it is not NULL, then use a CONVERT(DateTime, yourinputcolumn, 101) to properly convert the MM/DD/YYYY date format to a SQL Server DateTime datatype.
0
 

Author Comment

by:bar0822
ID: 34919446
ok. I am just learning SSIS - where do i use the isdate(date column, null)
and case statement?
0
 
LVL 22

Accepted Solution

by:
8080_Diver earned 333 total points
ID: 34919775
Do you have a SQL statement that is accessing the falt file data?

I generally pull flat file data into a staging table that has allof the fields defined as VarChar({somethingappropriate}) and then I use a SQL statement like SQL_1, below, to do the actual insertion of the data into the destination/target table.

In your case (and after a quick Google ;-), you need the INSERT statement to look sort of like SQL_2, below.
-- SQL_1
INSERT INTO targettable
(
 {list of columns} 
)
SELECT {list of columns}
FROM stagingtable;

-- SQL_2
INSERT INTO targettable
(
 {list of columns} 
)
SELECT {list of columns}
       ,CASE WHEN ISDATE(yourquestionablecolumn)= 1
             THEN CONVERT(DateTime, yourquestionablecolumn, 101)
             ELSE NULL
        END 
       ,{list of more columns}
FROM stagingtable;

Open in new window

0
 

Assisted Solution

by:bar0822
bar0822 earned 0 total points
ID: 34920290
I am putting the data from flat file directly into a temp table with (char) datafields but then trying to copy the data from this table into the SQL Table but using an oledb connection destination .. I am not fooling SQL.  I will try your solution above, so grateful for your help.  Will let u know how it goes.

0
 

Author Closing Comment

by:bar0822
ID: 34959172
the solution worked without any problems.  I was able to import from flat file to sql table using convert from a sql statement  - i was having trouble with the data conversion task- ran faster also.
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

Use this article to create a batch file to backup a Microsoft SQL Server database to a Windows folder.  The folder can be on the local hard drive or on a network share.  This batch file will query the SQL server to get the current date & time and wi…
Slowly Changing Dimension Transformation component in data task flow is very useful for us to manage and control how data changes in SSIS.
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function
Using examples as well as descriptions, and references to Books Online, show the documentation available for datatypes, explain the available data types and show how data can be passed into and out of variables.

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now