MS CRM 2011 Import Data - Lookup reference could not be resolved

Hi,

The message usually means you're trying to import an entity that has a relationship to another entity, but the relationship can't be created because the record doesn't exist.

If you have not yet, please follow below post and let us know-
http://blogs.msdn.com/b/dynamicscrmonline/archive/2011/06/24/crm-2011-data-import-wizard-importing-related-accounts-and-contacts.aspx

Are you committing the records (saving) before importing the next table? it sounds like the account records are not completely stored so the contracts records can't find them and in the time taken to re-run they appear

Good day,

Thank you for the information.

1. I knew about the excellent website that you recommended and used the recommendation of the zipfile for contracts and contract lines. It worked well until CRM gave an error message of not able to extract the information from the zipfile.
2. Since this happened we import first contracts and then contract lines individually.
3. The link between contracts and accounts are account name and it is the same, thus it is existing.
4. We do not make any changes to the data when it had failures with the message of "The lookup reference could not be resolved".
5. We just re submit and then it works.
6. We submit the accounts; contracts and contract lines in sequence immediately after each other.
7. is there some setting that we need to change or what else can I try to overcome this requirement of re submitting failures and then it is successfull
Thanking you in advance for your assistance.

I'm applying knowledge here of Oracle data load techniques which may or may not apply so someone may be able to improve on this:-

Between the load of bulk data and the availability of the records within that data there is a series of housekeeping tasks - indexing, log files, cascading reference updates and so on - which is referred to as 'the commit'.  To facilitate a rapid turn around these are generally done in 'blocks' of data which is managed at the database level; for example: the first 1000 records might be read and validated in memory then 'committed' as a set. Obviously there is a small window during which the record has been read but the data is not yet 'committed'. I am guessing that some of your parent records are not completed when the child records are validated. The simplest solution is to stagger the loads so the commit has time to complete. In my case we sorted the records in the dataset so the children were in the same sequence as the parent and reduced the commit size to approx. 250 records (the actual amount is a number of database pages rather than a number of records)  - this worked fine for smaller numbers but was inefficient when doing several millions so for the bigger data set we disabled referential integrity checks until the data was loaded and made the commit size as large as possible.

I don't know if such manipulation is possible within CRM interface or practical - if the CRM system is still in use whilst the load takes place.

Hi,

We submit the accounts; contracts and contract lines in sequence immediately after each other.

Do you check in Data Import records if the data is successfully imported/Import job is completed successfully for the first file and move ahead for the second one?

Hi Regmigrant,

1. I use the import facility (Import Data) as in the ribbon of MS CRM 2011.
2. We submit the three tables in sequence.
3. I have attached an example of imports.
3.1 The import highlighted as "1" was the first import and all the accounts were
       successful        but no contracts and contract lines.
3.2  We do not change any data in the tables and submit the import facility again for the
       contracts and contract lines. (labelled as 2 on attachment) All Contracts were  
       successful imported. No contract lines were successful.
3.3  We do not change any contract line data and submit the contract lines again. It is
       successful (Labelled as 3 on (attachment).

How can I apply a commit as explained by regmigrant.

The imports are submitted in sequence  - thus the first one finishes before the next one will start.

Thank you for the help.
Import-Statistics.png

Hi,

Do Remember... Database level changes like commit and rollback are strictly unsuported by Microsoft in case of MS Dynamics. Anything to modify must be done through either CRM UI or MS Dynamics CRM SDK.

Have you tried some pause while importing all 3 files??

Based on Rikin's comment above I would stay away from the database level commits, page sizes etc unless you have a DBA that knows precisely what they are doing. However from your description it sounds more and more like the problem I described.

Presumably this is not a one-off exercise to pre-load the data but will happen regularly and you are trying to automate or script it.

I don't know the detail of Import Data for Dynamics but when you say 'in sequence' do you issue a single Import command that specifies three files? I would recommend issuing each one as a single step and confirming the load before issuing the next.

Even if you currently issue 'import file1' then immediately 'import file2' its likely that the import code creates a thread and returns a command prompt before the load/commit cycle completes because the import sequence is 'asynchronous'

See this link for executing Data Import
http://msdn.microsoft.com/en-us/library/gg328031(v=crm.6).aspx

it clearly states:
"You must let each job complete successfully before you can submit a new message request for the next asynchronous job."

An although you are loading a complete set of data I believe you are creating three separate asynchronous jobs which is leading to your error

Hi,

This is what I have been trying to say... Data Import is being handled by CrmAsyncService and it will have a lag in some seconds up to 2 minutes.

Good day Rikin and Regmigrant,

1. We changed our import by ensuring that each import job completed before the next one
    is submitted.
    1.1  Submit import for accounts and wait until completed.
    1.2  Submit import for contracts and wait until completed.
    1.3  Submit import for contract lines.

The imports completed successful!!! No failures with the description of "The lookup reference could not be resolved".  

2. Is there something that I can do in order to be able to:
    2.1 Submit the imports(accounts; contracts and contract lines) with the import
          utility in the correct order and
    2.2 The accounts import will then first finish before the next import starts and so for
           contracts.

Thank you very much for the effort and assistance so far. It is much appreciated.
Pierre.

Thank you very much for the help.
Kind Regards
Pierre.

I concur - a split would be appropriate

Please tell me how to split - it was my intention to split and unfortunately by accident added it to two comments that belong to the same person. My sincere apologies. What must I do to reverse and submit it equally split.

You want to request an alert to moderators and put the request in there

Thank you very much.
Kind Regards.
Pierre.