asked on

SSIS Lookup Error- Help needed

Hi EE,

Back again with the lookup issue in SSIS,
SSIS lookup failing with the following errors

[ClientRefLkup [296]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "ClientRefLkup" failed because error code 0xC020901E occurred, and the error row disposition on "ClientRefLkup.Outputs[Lookup Match Output]" specifies failure on error. An error occurred on the specified object of the specified component.  There may be error messages posted before this with more information about the failure.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "ClientRefLkup" (296) failed with error code 0xC0209029 while processing input "Lookup Input" (310). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure.

[Booking Source [2]] Error: The attempt to add a row to the Data Flow task buffer failed with error code 0xC0047020.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on Booking Source returned error code 0xC02020C4.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be error messages posted before this with more information about the failure.

Open in new window

I know its related to underlying data but i am just baffled on how to sort this out.
I have attached the sql sheet with the source data and the dimension i am trying to lookup.
However i am unable to upload .dtx file with the project. But its a simple package containing the source data and then a lookup to the dimension data.
Please help.

Regards,

Ali
Data.xlsx

Ali Shah

ASKER

Sorry but here's the dimension table structre

CREATE TABLE [dbo].[DimClientReferences](
	[ClientRefSK] [INT] IDENTITY(1,1) NOT NULL,
	[ClientRef] [VARCHAR](200) NULL,
	[ClientRefType] [VARCHAR](50) NULL,
 CONSTRAINT [PK_DimClientReferences] PRIMARY KEY CLUSTERED 
(
	[ClientRefSK] ASC)

Open in new window

Nakul Vachhrajani

Without looking at the actual LookUp transformation and the associated keys, it is slightly difficult to comment on this one. But, here's something I noticed in your "Data.xlsx" file.

My understanding is that you are performing a lookup for SourceData.ClientOptionalRef, SourceData.Clientref1, SourceData.Clientref2 and so on. The lookup is being made on the dimension DimClient for a particular "ClientRefType" which corresponds to the column in the SourceData that you are working on.

If that is true, it appears that you do not have all the possible values in your dimension. For example:
- The value "Unknown" is required for all "ClientRefType"
- For "ClientRefType" = Clientref2, the following values do not exist in the dimension table, but are present in your inbound data: Client Ref 4, Client Ref 1, Client Ref 2

Similarly, you would need to check for all other "ClientRefType".

If this is an expected pattern, I would explore handing this as a late-arriving dimension in your ETL.
Also, in the look-up transform, you would need to choose what needs to happen when a look-up fails (i.e. value is not found in the look-up dimension table). Please choose the appropriate option. Typically, you would want to set this to "Redirect rows to no match output" to allow you to handle late-arriving dimensions appropriately rather than having the rows go to the Error Output (default behaviour).

Ali Shah

ASKER

Hi Nakul,
Thanks a lot for your reply,
Yes you are right i have clientref1 , 2 and so on. I have attached the image here

This explains the lookup.
one thing is quite bizarre, i have converted the stored procedure into a user defined function and selected the Distinct ClientOptionalRef (only this column) from the function and it worked. I have also tested it by putting Unknown against every clientref but still fails.

well another question, I am using SK0 and Unknown for the table and then using Union all in my lookup as you can see to bring back the Unknown value. should it not work?

regards,

regards

ASKER CERTIFIED SOLUTION

Nakul Vachhrajani

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Ali Shah

ASKER

Hi Nakul,

thanks again for your kind help. Yes its the case sensitivity issue which i was not aware of.
So do i need to save my dimensions data in Upper / lower case and do the same with the source data?

regards.

Ali

SOLUTION

Nakul Vachhrajani

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial