for some reason, I can't seem to figure out why I'm missing some lines when I import a csv file that I exported using sqlplus. The ssis package gives no warnings or errors as well.
The reason I know this is because when I loaded the csv file into a separate analysis application it gets the correct totals for numeric columns and row counts.
But for some reason, SSIS doesn't seem to capture all of the lines...
there were 6313052 rows in the csv file and ssis imports 6308607...
I've tried different code pages too (1250, 1252, and UTF8) but they didn't seem to have an affect
I checked out this link: Why all the records are not being copied from CSV to SQL table in a SSIS package
and numbers I've checked on numbers 2, 3, and 4.
Although, for number 1, I'm using a for each loop container descirbed in this site:http://help.pragmaticworks.com/dtsxchange/scr/FAQ%20-%20How%20to%20loop%20through%20files%20in%20a%20specified%20folder,%20load%20one%20by%20one%20and%20move%20to%20archive%20folder%20using%20SSIS.htm
to loop through files in a folder and import them.
I've also thought about missing delimiters, but I exported the files myself using sqlplus like this:
trim(to_char(t1.D_DTM, 'yyyy-mm-dd hh24:mm:ss'))||','||
I'm not sure how I could miss delimiters if I export a file like that...
Also, I tested importing these files into MySql using LOAD DATA INFILE and that seems to work fine and import all of the data...
I've also looked for commas in my varchar columns but was unable to find any...