DTS - loading files
Posted on 2006-06-08
I am loading a bunch of csv files daily via dts. Each file has two attributes - Symbol and Weight.
The table that these are loaded in has four attributes - BasketID, Symbol, Weight, Update
1) Symbol and Weight are a one-to-one in the data transformation. meaning, Source.Symbol = Destination.Symbol. and the same for Weight.
2) BasketID is inserted via a global variable which is based upon the name of the file being inserted. If ADRA_ETF_Composition.csv file is loaded, ADRA_basket is loaded into the BasketID field.
3) Update is simply a getdate() default.
In theory, everything works just fine. My logging lists every file that is processed, and each file is moved to an archival directory when finished.
I have two tables in place. The Composition table and the Composition2 table. Compostion2 is the one I am loading via DTS.
The other table is old and will be dropped/decommissioned once I have confirmed this new table is accurate -- or, being loaded just fine.
I have just now noticed that Symbol and Weight are being carried through every file that is being loaded.
For example, ADRA_ETF_Composition.csv is the first file loaded, there are 50 Symbols and weights for BasketID ADRA_basket.
The very same 50 symbols and weights are there for basketID ADRD_basket...and ADRE_basket, and for every basket in the table.
It appears the only thing I am doing per file is the basketID and the Update (which is simply a getdate default).
Can anybody suggest for me how it is that I am locking myself into the Symbol and Weight values of the first file loaded, and then carrying them through for every subsequent file loaded?