SSIS Pull strange, Corrupted source file???

Just wondering if anyone else has hit this problem, and can either tell me what is wrong with SSIS, or if this is a sign of a corrupted source file.

We are pulling over 30 files all fine except for one. All the fields in the problem file are mapped correctly in the source, and in the destination... But one field pulls in null if the Selection asks for a pull of all the records (about 850,000).. Yet with just changing the selection from "Select * from table"  to Select * from table where log_date >= yesterday  I pull about 2,000 records with the problem field displaying the ad number it should be.

Background on this table, we are converting from data pulls through Visual Studion statements that we used until we found the wonders of SSIS.. This is still working since we are working the kinks out of our new pull (kinks like this one)..  The same file went from a 1 1/2 hour pull when we did the full pull once a month, to this week takeing 2 days to pull, but the problem field did pull fine..

Now if I do a SQL cmdShell  command pull of the file, it seem to pull fine.

But, with 2 strange unexplainable problems on the same file I am leaning toward the source file being corrupted, maybe on this problem field. I was just hoping some of you SSIS, SQL gurus might be able to give me some advice on other ways to test this theroy out..

PS.. the problem table is not in SQL.. I am connecting to it through ODBC.. (ie the cmdShell pull)
LVL 1
moosetrackerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

rajvjaCommented:
Check the datatype and size of the problem field. Try changing to sufficient type and size.
I got the same problem and resolved using this.
0
Reza RadConsultant, TrainerCommented:
could you upload your package and the corrupted file here?
0
PedroCGDCommented:
What is the error message?
Regards,
pedro
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

moosetrackerAuthor Commented:
No Error message. Runs successfully, no matter what. It is just that if I use the following statement in the
Select * from artransaction where Profit_no = 1 and log_date > 1/1/2010  ..   All is ok
Select * from artransaction where Profit_no = 1 and log_date > 1/1/2009  and log_date <= 12/31/2009  .. All is ok
Select * from artransaction where Profit_no = 1 and log_date > 1/1/2008  and log_date <= 12/31/2008

But
Select * from artransaction where Profit_no = 1  ...   I get all the records, only for one field (Ad_no) the field is only null values..  Yet for the 2 filtered Select statements while some fields are null, about 90% is filled with a number.

I can not upload the table.. It's on a server I have limited access to, plus that would be mega huge.
I would be willing to upload the package, but the Attach file does not see the extension of the package as one it accepts, so how would I do that?



0
moosetrackerAuthor Commented:
I have narrowed down the problem is in the year 2006

Select * from artransaction where Profit_no = 1 and log_date >= 1/1/2006  and log_date <= 12/31/2006

Has the problem of Nulling out all the data in the Ad_no field..  Now I will try to narrow down to the month the problem is in.  
0
moosetrackerAuthor Commented:
I've narrowed the pull down to
Select * from artransaction where Profit_no = 1 and log_date >= 3/1/2006  and log_date <= 3/31/2006

as having something in it that makes all the records pulled Null out the Ad_no field..

This was the first month we went live with this system...  Hmm..
Thanks guys.. I am really leaning toward corrupted data at this point..
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
PedroCGDCommented:
:-)
always welcome!
Pedro
0
grzegorzsCommented:
Place a grid view data viewer after source and see what leaves the source. It maybe a problem with datatypes or conversion. Maybe there is value in source file which is misinterpreted because it contains a special character (I had similar problem: I had comma decimal seperator in numbers and the comma was also a column seperator...bad luck...)

best regards
Grzegorz
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.