SSIS Lookup Error- Help needed
Hi EE,
Back again with the lookup issue in SSIS,
SSIS lookup failing with the following errors
I know its related to underlying data but i am just baffled on how to sort this out.
I have attached the sql sheet with the source data and the dimension i am trying to lookup.
However i am unable to upload .dtx file with the project. But its a simple package containing the source data and then a lookup to the dimension data.
Please help.
Regards,
Ali
Data.xlsx
Back again with the lookup issue in SSIS,
SSIS lookup failing with the following errors
[ClientRefLkup [296]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "ClientRefLkup" failed because error code 0xC020901E occurred, and the error row disposition on "ClientRefLkup.Outputs[Lookup Match Output]" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "ClientRefLkup" (296) failed with error code 0xC0209029 while processing input "Lookup Input" (310). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.
[Booking Source [2]] Error: The attempt to add a row to the Data Flow task buffer failed with error code 0xC0047020.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on Booking Source returned error code 0xC02020C4. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
I know its related to underlying data but i am just baffled on how to sort this out.
I have attached the sql sheet with the source data and the dimension i am trying to lookup.
However i am unable to upload .dtx file with the project. But its a simple package containing the source data and then a lookup to the dimension data.
Please help.
Regards,
Ali
Data.xlsx
Last Comment
Without looking at the actual LookUp transformation and the associated keys, it is slightly difficult to comment on this one. But, here's something I noticed in your "Data.xlsx" file.
My understanding is that you are performing a lookup for SourceData.ClientOptionalR
If that is true, it appears that you do not have all the possible values in your dimension. For example:
- The value "Unknown" is required for all "ClientRefType"
- For "ClientRefType" = Clientref2, the following values do not exist in the dimension table, but are present in your inbound data: Client Ref 4, Client Ref 1, Client Ref 2
Similarly, you would need to check for all other "ClientRefType".
If this is an expected pattern, I would explore handing this as a late-arriving dimension in your ETL.
Also, in the look-up transform, you would need to choose what needs to happen when a look-up fails (i.e. value is not found in the look-up dimension table). Please choose the appropriate option. Typically, you would want to set this to "Redirect rows to no match output" to allow you to handle late-arriving dimensions appropriately rather than having the rows go to the Error Output (default behaviour).
My understanding is that you are performing a lookup for SourceData.ClientOptionalR
If that is true, it appears that you do not have all the possible values in your dimension. For example:
- The value "Unknown" is required for all "ClientRefType"
- For "ClientRefType" = Clientref2, the following values do not exist in the dimension table, but are present in your inbound data: Client Ref 4, Client Ref 1, Client Ref 2
Similarly, you would need to check for all other "ClientRefType".
If this is an expected pattern, I would explore handing this as a late-arriving dimension in your ETL.
Also, in the look-up transform, you would need to choose what needs to happen when a look-up fails (i.e. value is not found in the look-up dimension table). Please choose the appropriate option. Typically, you would want to set this to "Redirect rows to no match output" to allow you to handle late-arriving dimensions appropriately rather than having the rows go to the Error Output (default behaviour).
ASKER
Hi Nakul,
Thanks a lot for your reply,
Yes you are right i have clientref1 , 2 and so on. I have attached the image here
This explains the lookup.
one thing is quite bizarre, i have converted the stored procedure into a user defined function and selected the Distinct ClientOptionalRef (only this column) from the function and it worked. I have also tested it by putting Unknown against every clientref but still fails.
well another question, I am using SK0 and Unknown for the table and then using Union all in my lookup as you can see to bring back the Unknown value. should it not work?
regards,
regards
Thanks a lot for your reply,
Yes you are right i have clientref1 , 2 and so on. I have attached the image here
This explains the lookup. one thing is quite bizarre, i have converted the stored procedure into a user defined function and selected the Distinct ClientOptionalRef (only this column) from the function and it worked. I have also tested it by putting Unknown against every clientref but still fails.
well another question, I am using SK0 and Unknown for the table and then using Union all in my lookup as you can see to bring back the Unknown value. should it not work?
regards,
regards
ASKER CERTIFIED SOLUTION
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
ASKER
Hi Nakul,
thanks again for your kind help. Yes its the case sensitivity issue which i was not aware of.
So do i need to save my dimensions data in Upper / lower case and do the same with the source data?
regards.
Ali
thanks again for your kind help. Yes its the case sensitivity issue which i was not aware of.
So do i need to save my dimensions data in Upper / lower case and do the same with the source data?
regards.
Ali
SOLUTION
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
Microsoft SQL Server
Microsoft SQL Server is a suite of relational database management system (RDBMS) products providing multi-user database access functionality.SQL Server is available in multiple versions, typically identified by release year, and versions are subdivided into editions to distinguish between product functionality. Component services include integration (SSIS), reporting (SSRS), analysis (SSAS), data quality, master data, T-SQL and performance tuning.
171K
Questions
--
Followers
--
Top Experts
Get a personalized solution from industry experts
TRUSTED BY
ASKER
Open in new window