Solved

SQLXML bulkload - parent-child relationship and autogeneration of IDs.  IDs not inherited.

Posted on 2006-11-13
20
1,316 Views
Last Modified: 2013-11-19

I am trying to get a bulk-load statement to work using SQLXML.

I can insert the rows into the parent table fine, and the DB generates a value for the PK coulmn (using the Default).

I cannot get the child rows to be inserted because the FK column can't be null.  MS say this is by design (great) and provide no solution.

My tables have a constraint (FK) between them.

My source XML contains no ID.



However, an alternative I have done is to edit the XMl at runtime and insert a new node to make an ID for each parent.
The problem still exists, and it crashing saying that it cannot find a source for the child's FK column.

 
What I really want is for it to throw the parent data at the DB and have an ID auto-generated (using the DEFAULT) for the parent ID, and then have this propgate down to be the FK for the children.  However SQLXML is unable to do this (microsoft document this), but it is (apparently) able to propogate the FK down if the parent's ID is provided in the source data.  I'm trying this approach but it's failing.


Please advise

SCHEMA:
<?xml version="1.0" encoding="utf-8" ?>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sql="urn:schemas-microsoft-com:mapping-schema">
      <xsd:annotation>
            <xsd:appinfo>
                  <sql:relationship name="FK_tbl_SailingSchedule_RoutingDetails_tbl_SailingSchedule"
                        parent="tbl_SailingSchedule"
                        parent-key="SailingSchedule_id"
                        child="tbl_SailingSchedule_RoutingDetails"
                        child-key="RoutingDetails_SailingId" />

            </xsd:appinfo>
      </xsd:annotation>

      <xsd:element name="ScheduleDetails" sql:relation="tbl_SailingSchedule" sql:key-fields="SailingSchedule_id" >
            <xsd:complexType>
                  <xsd:sequence>
                        <xsd:element name="SailingSchedule_id"        type="xsd:string" />
                        <xsd:element name="VesselVoyageID"        type="xsd:string" />
                        <xsd:element name="VesselName"        type="xsd:string" />
                        <xsd:element name="IMONumber"        type="xsd:string" />
                        <xsd:element name="Voyage"        type="xsd:string" />
                        <xsd:element name="CarrierSCAC"        type="xsd:string" />
                        <xsd:element name="SCAC"        type="xsd:string" />
                        <xsd:element name="AmsFlag"        type="xsd:string" />
                        <xsd:element name="CFSOrigin"        type="xsd:string" />
                        <xsd:element name="CFSDestination"        type="xsd:string" />
                         
                         <xsd:element name="RoutingDetails"
                              sql:relationship="FK_tbl_SailingSchedule_RoutingDetails_tbl_SailingSchedule"
                              sql:relation="tbl_SailingSchedule_RoutingDetails" sql:key-fields="RoutingDetails_id"
                         >
                         
                         
                              <xsd:complexType>
                                    <xsd:sequence maxOccurs="unbounded">                                    
                                          
                                          <xsd:element name="StageQualifier"        type="xsd:string" />
                                          <xsd:element name="Transportmode"        type="xsd:string" />
                                          <xsd:element name="TransportName"        type="xsd:string" />
                                          <xsd:element name="IMONumber"        type="xsd:string" />
                                          <xsd:element name="Origin"        type="xsd:string" />
                                          <xsd:element name="ETD"        type="xsd:string" />
                                          <xsd:element name="Destination"        type="xsd:string" />
                                          <xsd:element name="ETA"        type="xsd:string" />
                                    </xsd:sequence>
                              </xsd:complexType>
                        </xsd:element>
                         
                  </xsd:sequence>
            </xsd:complexType>
      </xsd:element>
 
</xsd:schema>




====================================================================

TABLES:


CREATE TABLE [tbl_SailingSchedule] (
     [SailingSchedule_id]  uniqueidentifier ROWGUIDCOL  NOT NULL CONSTRAINT [DF_tbl_SailingSchedule_SailingSchedule_id] DEFAULT (newid()),
     [VesselVoyageID] [int] NULL ,
     [VesselName] [nvarchar] (255) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
     [IMONumber] [int] NULL ,
     [Voyage] [nvarchar] (10) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
     [CarrierSCAC] [char] (4) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
     [SCAC] [char] (4) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
     [AmsFlag] [char] (1) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
     [CFSOrigin] [char] (5) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
     [CFSDestination] [char] (5) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
      PRIMARY KEY  CLUSTERED
     (
          [SailingSchedule_id]
     )  ON [PRIMARY]
) ON [PRIMARY]
GO



CREATE TABLE [tbl_SailingSchedule_RoutingDetails] (
     [RoutingDetails_id]  uniqueidentifier ROWGUIDCOL  NOT NULL CONSTRAINT [DF_tbl_SailingSchedule_RoutingDetials_RoutingDetials_id] DEFAULT (newid()),
     [RoutingDetails_SailingId] [uniqueidentifier] NOT NULL ,
     [StageQualifier] [int] NULL ,
     [Transportmode] [int] NULL ,
     [TransportName] [nvarchar] (255) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
     [IMONumber] [int] NULL ,
     [Origin] [char] (5) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
     [Destination] [char] (5) COLLATE SQL_Latin1_General_Pref_CP1_CI_AS NULL ,
     [ETD] [datetime] NULL ,
     [ETA] [datetime] NULL ,
      PRIMARY KEY  CLUSTERED
     (
          [RoutingDetails_id]
     )  ON [PRIMARY] ,
     CONSTRAINT [FK_tbl_SailingSchedule_RoutingDetails_tbl_SailingSchedule] FOREIGN KEY
     (
          [RoutingDetails_SailingId]
     ) REFERENCES [tbl_SailingSchedule] (
          [SailingSchedule_id]
     ) ON DELETE CASCADE  ON UPDATE CASCADE
) ON [PRIMARY]
GO



====================================================================





SOURCE DATA:



<Schedule  >
  <!--  <ScheduleEnvelope>
        <SenderID>NACA</SenderID>
        <ReceiverID>KNPROD</ReceiverID>
        <Password>TradingPartners</Password>
        <Type>SCHEDULE</Type>
        <Version>1.0.7</Version>
        <EnvelopeID>NACA-10708</EnvelopeID>
    </ScheduleEnvelope>
    -->
    <ScheduleDetails>
        <VesselVoyageID>11499412</VesselVoyageID>
        <VesselName>KOOKABURRA 1</VesselName>
        <IMONumber>9319571</IMONumber>
        <Voyage>611</Voyage>
        <CarrierSCAC>USLB</CarrierSCAC>
        <SCAC>NAQA</SCAC>
        <AmsFlag>N</AmsFlag>
        <CFSOrigin>USATL</CFSOrigin>
        <CFSDestination>AUADL</CFSDestination>
        <RoutingDetails>
            <StageQualifier>1</StageQualifier>
            <Transportmode>2</Transportmode>
            <TransportName>RAIL</TransportName>
            <IMONumber>
            </IMONumber>
            <Origin>USATL</Origin>
            <ETD>2006-10-23</ETD>
            <Destination>USLAX</Destination>
            <ETA>2006-10-31</ETA>
        </RoutingDetails>
        <RoutingDetails>
            <StageQualifier>3</StageQualifier>
            <Transportmode>4</Transportmode>
            <TransportName>KOOKABURRA 1</TransportName>
            <IMONumber>9319571</IMONumber>
            <Origin>USLAX</Origin>
            <ETD>2006-11-02</ETD>
            <Destination>AUMEL</Destination>
            <ETA>2006-11-22</ETA>
        </RoutingDetails>
        <RoutingDetails>
            <StageQualifier>5</StageQualifier>
            <Transportmode>5</Transportmode>
            <TransportName>TBN</TransportName>
            <IMONumber>
            </IMONumber>
            <Origin>AUMEL</Origin>
            <ETD>2006-11-22</ETD>
            <Destination>AUADL</Destination>
            <ETA>2006-11-25</ETA>
        </RoutingDetails>
        <RoutingDetails>
            <StageQualifier>6</StageQualifier>
            <Transportmode>1</Transportmode>
            <TransportName>TRUCK</TransportName>
            <IMONumber>
            </IMONumber>
            <Origin>AUADL</Origin>
            <ETD>2006-11-25</ETD>
            <Destination>AUADL</Destination>
            <ETA>2006-11-25</ETA>
        </RoutingDetails>
    </ScheduleDetails>
    <ScheduleDetails>
        <VesselVoyageID>11499786</VesselVoyageID>
        <VesselName>MAERSK DALE</VesselName>
        <IMONumber>9232577</IMONumber>
        <Voyage>608</Voyage>
        <CarrierSCAC>HLCU</CarrierSCAC>
        <SCAC>NAQA</SCAC>
        <AmsFlag>N</AmsFlag>
        <CFSOrigin>USATL</CFSOrigin>
        <CFSDestination>NZAKL</CFSDestination>
        <RoutingDetails>
            <StageQualifier>1</StageQualifier>
            <Transportmode>1</Transportmode>
            <TransportName>TRUCK</TransportName>
            <IMONumber>
            </IMONumber>
            <Origin>USATL</Origin>
            <ETD>2006-10-23</ETD>
            <Destination>USSAV</Destination>
            <ETA>2006-10-28</ETA>
        </RoutingDetails>
        <RoutingDetails>
            <StageQualifier>2</StageQualifier>
            <Transportmode>4</Transportmode>
            <TransportName>MAERSK DALE</TransportName>
            <IMONumber>9232577</IMONumber>
            <Origin>USSAV</Origin>
            <ETD>2006-10-28</ETD>
            <Destination>NZAKL</Destination>
            <ETA>2006-11-17</ETA>
        </RoutingDetails>
        <RoutingDetails>
            <StageQualifier>6</StageQualifier>
            <Transportmode>1</Transportmode>
            <TransportName>TRUCK</TransportName>
            <IMONumber>
            </IMONumber>
            <Origin>NZAKL</Origin>
            <ETD>2006-11-17</ETD>
            <Destination>NZAKL</Destination>
            <ETA>2006-11-17</ETA>
        </RoutingDetails>
    </ScheduleDetails>
</Schedule>



 

====================================================================

CODE
           


            static private void InsertData()
            {
                  //JAS_Generated_GUID
                  System.Xml.XmlDocument soDoc = new System.Xml.XmlDocument();
                  soDoc.Load(@"..\..\SMALL_SAMPLE_SAIL_SCHED2WithData.XML");
                  System.Xml.XmlNodeList soDetailsNodes = soDoc.DocumentElement.SelectNodes("ScheduleDetails");
                  foreach (System.Xml.XmlNode soNode in soDetailsNodes)
                  {
                        System.Xml.XmlNode soNewNode = soDoc.CreateElement("SailingSchedule_id");
                        soNewNode.InnerText= System.Guid.NewGuid().ToString();
                        soNode.AppendChild(soNewNode);      
                  }
                  soDoc.Save(@"..\..\SMALL_SAMPLE_SAIL_SCHED2WithData_WITHID.XML");
                  


                  try
                  {
                        SQLXMLBULKLOADLib.SQLXMLBulkLoad3Class objBL = new SQLXMLBULKLOADLib.SQLXMLBulkLoad3Class();
                        objBL.ConnectionString = "Provider=sqloledb;server=sql.jaslon.local;database=MLW3;integrated security=SSPI";
                        objBL.ErrorLogFile = "error.xml";
                        objBL.KeepIdentity = true;
                        objBL.Execute (@"..\..\SailingsheduleToJasDb_Mapping.xml",  @"..\..\SMALL_SAMPLE_SAIL_SCHED2WithData.XML");
                        
                  }
                  catch(Exception e)
                  {
                        Console.WriteLine(e.ToString());
                  }



            }





===================================================================================

ERROR:

System.Runtime.InteropServices.COMException (0x80004005): No data was provided for column 'RoutingDetails_SailingId' on table 'tbl_SailingSchedule_RoutingDetails', and this column cannot contain NULL values.
   at SQLXMLBULKLOADLib.SQLXMLBulkLoad3Class.Execute(String bstrSchemaFile, Obje
ct vDataFile)
   at SailingSchedule.Class1.InsertData() in d:\daniel documents and work\visual
 studio projects\nacalogistics\sailingschedule\class1.cs:line 66

0
Comment
Question by:jasww
  • 11
  • 9
20 Comments
 
LVL 16

Expert Comment

by:muzzy2003
ID: 17938963
Your SailingSchedule element doesn't contain a SailingSchedule_id. This is what would be used to populate the erroring field. SQLXMLBulkLoad can't retrieve the default value generated by the database!
0
 

Author Comment

by:jasww
ID: 17939085
Yes, I know that.  This is why I have put in place code to generate a SailingSchedule_id element for each SailingSchedule node.
Although I can update the source XML to have an ID, it still does not propogate down to the child.
Please explain how to make it work.
0
 
LVL 16

Expert Comment

by:muzzy2003
ID: 17939129
So with this element in your XML, you are still getting exactly the same error message?
0
Guide to Performance: Optimization & Monitoring

Nowadays, monitoring is a mixture of tools, systems, and codes—making it a very complex process. And with this complexity, comes variables for failure. Get DZone’s new Guide to Performance to learn how to proactively find these variables and solve them before a disruption occurs.

 
LVL 16

Expert Comment

by:muzzy2003
ID: 17939148
(Sorry, BTW, just re-read your question properly!)
0
 

Author Comment

by:jasww
ID: 17939158
Yes.

I just noticed a mistake in the above code pasted, the Execute() line should load the new, edited XML:

 objBL.Execute (@"..\..\SailingsheduleToJasDb_Mapping.xml",  @"..\..\SMALL_SAMPLE_SAIL_SCHED2WithData_WITHID.XML"); //correct
 objBL.Execute (@"..\..\SailingsheduleToJasDb_Mapping.xml",  @"..\..\SMALL_SAMPLE_SAIL_SCHED2WithData.XML");  // wrong


But the error message is the same.


0
 
LVL 16

Accepted Solution

by:
muzzy2003 earned 250 total points
ID: 17939206
You're appending the new node, which puts it at the end of the list of child elements. Your schema specifies it as the first element in a sequence. Try using the PrependChild method rather than the AppendChild method and try again.
0
 

Author Comment

by:jasww
ID: 17939227
Or, could I put this node at the end of the schema?


i.e.


<?xml version="1.0" encoding="utf-8" ?>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sql="urn:schemas-microsoft-com:mapping-schema">
     <xsd:annotation>
          <xsd:appinfo>
               <sql:relationship name="FK_tbl_SailingSchedule_RoutingDetails_tbl_SailingSchedule"
                    parent="tbl_SailingSchedule"
                    parent-key="SailingSchedule_id"
                    child="tbl_SailingSchedule_RoutingDetails"
                    child-key="RoutingDetails_SailingId" />

          </xsd:appinfo>
     </xsd:annotation>

     <xsd:element name="ScheduleDetails" sql:relation="tbl_SailingSchedule" sql:key-fields="SailingSchedule_id" >
          <xsd:complexType>
               <xsd:sequence>
                     <!--   **** ID ELEMENT REMOVED *** -->
                    <xsd:element name="VesselVoyageID"        type="xsd:string" />
                    <xsd:element name="VesselName"        type="xsd:string" />
                    <xsd:element name="IMONumber"        type="xsd:string" />
                    <xsd:element name="Voyage"        type="xsd:string" />
                    <xsd:element name="CarrierSCAC"        type="xsd:string" />
                    <xsd:element name="SCAC"        type="xsd:string" />
                    <xsd:element name="AmsFlag"        type="xsd:string" />
                    <xsd:element name="CFSOrigin"        type="xsd:string" />
                    <xsd:element name="CFSDestination"        type="xsd:string" />
                     
                     <xsd:element name="RoutingDetails"
                         sql:relationship="FK_tbl_SailingSchedule_RoutingDetails_tbl_SailingSchedule"
                         sql:relation="tbl_SailingSchedule_RoutingDetails" sql:key-fields="RoutingDetails_id"
                     >
                     
                     
                         <xsd:complexType>
                              <xsd:sequence maxOccurs="unbounded">                              
                                   
                                   <xsd:element name="StageQualifier"        type="xsd:string" />
                                   <xsd:element name="Transportmode"        type="xsd:string" />
                                   <xsd:element name="TransportName"        type="xsd:string" />
                                   <xsd:element name="IMONumber"        type="xsd:string" />
                                   <xsd:element name="Origin"        type="xsd:string" />
                                   <xsd:element name="ETD"        type="xsd:string" />
                                   <xsd:element name="Destination"        type="xsd:string" />
                                   <xsd:element name="ETA"        type="xsd:string" />
                              </xsd:sequence>
                         </xsd:complexType>
                    </xsd:element>

                     <!-- *** ID ELEMENT INSERTED AT BOTTOM **** -->
                    <xsd:element name="SailingSchedule_id"        type="xsd:string" />
                     
               </xsd:sequence>
          </xsd:complexType>
     </xsd:element>
 
</xsd:schema>

0
 

Author Comment

by:jasww
ID: 17939247
The re-ordered schema made no difference.

Let me try the PrependChild() method.... but I don't think it will make any difference.  It's not that (parent) table tbl_SailingSchedule can't find its PK, it's that (child) table tbl_SailingSchedule_RoutingDetails can't find its FK.

0
 
LVL 16

Expert Comment

by:muzzy2003
ID: 17939266
The parent table won't error if it doesn't find its PK, though, will it? You've provided a default. Let me know if PrependChild helps at all, and I'll keep thinking.
0
 
LVL 16

Expert Comment

by:muzzy2003
ID: 17939281
Capitalisation - your C# code is adding an element named SailingSchedule_id, when it should be SailingSchedule_Id. This plus PrependChild should do it with a bit of luck.
0
 
LVL 16

Expert Comment

by:muzzy2003
ID: 17939282
Ignore last - eyes swimming, they seem to all be consistent on second look. Sorry.
0
 
LVL 16

Expert Comment

by:muzzy2003
ID: 17939296
Does it help to remove this attribute from the child element?

sql:key-fields="RoutingDetails_id"
0
 

Author Comment

by:jasww
ID: 17939351
The error message is now:


System.Runtime.InteropServices.COMException (0x80004005): Invalid character value for cast specification.
   at SQLXMLBULKLOADLib.SQLXMLBulkLoad3Class.Execute(String bstrSchemaFile, Obje
ct vDataFile)
   at SailingSchedule.Class1.InsertData() in d:\daniel documents and work\visual studio projects\nacalogistics\sailingschedule\class1.cs:line 66





Could this mean that it's now trying to insert the data into the child row but somehow the ID I generate is duff?  

The edited XML looks like:



 <ScheduleDetails>
    <SailingSchedule_id>ad754bd6-c30d-4f3e-9e4c-c53272159518</SailingSchedule_id>
    <VesselVoyageID>11499412</VesselVoyageID>
    <VesselName>KOOKABURRA 1</VesselName>
    <IMONumber>9319571</IMONumber>
    <Voyage>611</Voyage>
    <CarrierSCAC>USLB</CarrierSCAC>
    <SCAC>NAQA</SCAC>
    <AmsFlag>N</AmsFlag>
    <CFSOrigin>USATL</CFSOrigin>
    <CFSDestination>AUADL</CFSDestination>
    <RoutingDetails>
      <StageQualifier>1</StageQualifier>
      <Transportmode>2</Transportmode>
      <TransportName>RAIL</TransportName>
      <IMONumber>
      </IMONumber>
      <Origin>USATL</Origin>
      <ETD>2006-10-23</ETD>
      <Destination>USLAX</Destination>
      <ETA>2006-10-31</ETA>
    </RoutingDetails>
    <RoutingDetails>
      <StageQualifier>3</StageQualifier>
      <Transportmode>4</Transportmode>
      <TransportName>KOOKABURRA 1</TransportName>
      <IMONumber>9319571</IMONumber>
      <Origin>USLAX</Origin>
      <ETD>2006-11-02</ETD>
      <Destination>AUMEL</Destination>
      <ETA>2006-11-22</ETA>
    </RoutingDetails>
    <RoutingDetails>
      <StageQualifier>5</StageQualifier>
      <Transportmode>5</Transportmode>
      <TransportName>TBN</TransportName>
      <IMONumber>
      </IMONumber>
      <Origin>AUMEL</Origin>
      <ETD>2006-11-22</ETD>
      <Destination>AUADL</Destination>
      <ETA>2006-11-25</ETA>
    </RoutingDetails>
    <RoutingDetails>
      <StageQualifier>6</StageQualifier>
      <Transportmode>1</Transportmode>
      <TransportName>TRUCK</TransportName>
      <IMONumber>
      </IMONumber>
      <Origin>AUADL</Origin>
      <ETD>2006-11-25</ETD>
      <Destination>AUADL</Destination>
      <ETA>2006-11-25</ETA>
    </RoutingDetails>
  </ScheduleDetails>

... snip ...
0
 

Author Comment

by:jasww
ID: 17939370
The error log file from the bulk load library looks like this:


  <?xml version="1.0" ?>
- <Error>
- <Record>
  <HResult>0x80004005</HResult>
  <SQLState>22018</SQLState>
  <NativeError />
  <Source>Microsoft OLE DB Provider for SQL Server</Source>
- <Description>
- <![CDATA[ Invalid character value for cast specification.
  ]]>
  </Description>
  </Record>
  </Error>
0
 
LVL 16

Expert Comment

by:muzzy2003
ID: 17939413
Some sort of error converting the string to a GUID would be my guess. The syntax looks OK to me. Perhaps try converting the GUID to upper case like in the MSDN examples involving GUIDs? (SHOULDN'T make a difference, but this is M$ we're talking about).
0
 
LVL 16

Expert Comment

by:muzzy2003
ID: 17939430
IMONumber is an int column - if there is no value in your XML, you must omit the element, otherwise SQL is trying to cast an empty string to an int! Check all other non-string columns as well.

It may be nothing to do with the GUIDs at all now.
0
 

Author Comment

by:jasww
ID: 17939448
I have already coverted all the collumns to be very lenient - nullable large nvarchars.  Still same problem.

Let me try upper case guids.
0
 

Author Comment

by:jasww
ID: 17939474
Cracked it.

Need upper case AND {braces} around my guid.

Must be:
{D868513C-99B1-467A-A26F-593715C061EE}



Then it works fine.



Thanks for your help.
0
 
LVL 16

Expert Comment

by:muzzy2003
ID: 17939541
Yee hah! The joys of bulk load, eh?

Glad we got there in the end.
0
 

Author Comment

by:jasww
ID: 17939602
Cheers.


Just out of interest, how on earth does it work?  I see from a profiler trace that a statemen that starts with the text "INSERT BULK tbl_SailingSchedule...." but nowhere ever do I see a filename or data values being passed.  Weird.


Still, job done.

0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Formatting issues in XSL FO 3 50
Convert XML to excel12book 5 33
How to redirect all users but me properly to another page with htaccess 2 153
tutorial for ebay api 3 80
Most of the sites are being standardized with W3C Web Standards. W3C provides lot of web standard services to the web. They have the web specification, process and documentation for all the web standards. You can apply HTML, CSS and Accessibility st…
The Confluence of Individual Knowledge and the Collective Intelligence At this writing (summer 2013) the term API (http://dictionary.reference.com/browse/API?s=t) has made its way into the popular lexicon of the English language.  A few years ago, …
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…
The viewer will learn how to count occurrences of each item in an array.

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question