SSIS Pull strange, Corrupted source file???
Posted on 2010-01-12
Just wondering if anyone else has hit this problem, and can either tell me what is wrong with SSIS, or if this is a sign of a corrupted source file.
We are pulling over 30 files all fine except for one. All the fields in the problem file are mapped correctly in the source, and in the destination... But one field pulls in null if the Selection asks for a pull of all the records (about 850,000).. Yet with just changing the selection from "Select * from table" to Select * from table where log_date >= yesterday I pull about 2,000 records with the problem field displaying the ad number it should be.
Background on this table, we are converting from data pulls through Visual Studion statements that we used until we found the wonders of SSIS.. This is still working since we are working the kinks out of our new pull (kinks like this one).. The same file went from a 1 1/2 hour pull when we did the full pull once a month, to this week takeing 2 days to pull, but the problem field did pull fine..
Now if I do a SQL cmdShell command pull of the file, it seem to pull fine.
But, with 2 strange unexplainable problems on the same file I am leaning toward the source file being corrupted, maybe on this problem field. I was just hoping some of you SSIS, SQL gurus might be able to give me some advice on other ways to test this theroy out..
PS.. the problem table is not in SQL.. I am connecting to it through ODBC.. (ie the cmdShell pull)