jasww
asked on
SQLXML bulkload - parent-child relationship and autogeneration of IDs. IDs not inherited.
I am trying to get a bulk-load statement to work using SQLXML.
I can insert the rows into the parent table fine, and the DB generates a value for the PK coulmn (using the Default).
I cannot get the child rows to be inserted because the FK column can't be null. MS say this is by design (great) and provide no solution.
My tables have a constraint (FK) between them.
My source XML contains no ID.
However, an alternative I have done is to edit the XMl at runtime and insert a new node to make an ID for each parent.
The problem still exists, and it crashing saying that it cannot find a source for the child's FK column.
What I really want is for it to throw the parent data at the DB and have an ID auto-generated (using the DEFAULT) for the parent ID, and then have this propgate down to be the FK for the children. However SQLXML is unable to do this (microsoft document this), but it is (apparently) able to propogate the FK down if the parent's ID is provided in the source data. I'm trying this approach but it's failing.
Please advise
SCHEMA:
<?xml version="1.0" encoding="utf-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sql="urn:schemas-mic
<xsd:annotation>
<xsd:appinfo>
<sql:relationship name="FK_tbl_SailingSchedu
parent="tbl_SailingSchedul
parent-key="SailingSchedul
child="tbl_SailingSchedule
child-key="RoutingDetails_
</xsd:appinfo>
</xsd:annotation>
<xsd:element name="ScheduleDetails" sql:relation="tbl_SailingS
<xsd:complexType>
<xsd:sequence>
<xsd:element name="SailingSchedule_id" type="xsd:string" />
<xsd:element name="VesselVoyageID" type="xsd:string" />
<xsd:element name="VesselName" type="xsd:string" />
<xsd:element name="IMONumber" type="xsd:string" />
<xsd:element name="Voyage" type="xsd:string" />
<xsd:element name="CarrierSCAC" type="xsd:string" />
<xsd:element name="SCAC" type="xsd:string" />
<xsd:element name="AmsFlag" type="xsd:string" />
<xsd:element name="CFSOrigin" type="xsd:string" />
<xsd:element name="CFSDestination" type="xsd:string" />
<xsd:element name="RoutingDetails"
sql:relationship="FK_tbl_S
sql:relation="tbl_SailingS
>
<xsd:complexType>
<xsd:sequence maxOccurs="unbounded">
<xsd:element name="StageQualifier" type="xsd:string" />
<xsd:element name="Transportmode" type="xsd:string" />
<xsd:element name="TransportName" type="xsd:string" />
<xsd:element name="IMONumber" type="xsd:string" />
<xsd:element name="Origin" type="xsd:string" />
<xsd:element name="ETD" type="xsd:string" />
<xsd:element name="Destination" type="xsd:string" />
<xsd:element name="ETA" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
==========================
TABLES:
CREATE TABLE [tbl_SailingSchedule] (
[SailingSchedule_id] uniqueidentifier ROWGUIDCOL NOT NULL CONSTRAINT [DF_tbl_SailingSchedule_Sa
[VesselVoyageID] [int] NULL ,
[VesselName] [nvarchar] (255) COLLATE SQL_Latin1_General_Pref_CP
[IMONumber] [int] NULL ,
[Voyage] [nvarchar] (10) COLLATE SQL_Latin1_General_Pref_CP
[CarrierSCAC] [char] (4) COLLATE SQL_Latin1_General_Pref_CP
[SCAC] [char] (4) COLLATE SQL_Latin1_General_Pref_CP
[AmsFlag] [char] (1) COLLATE SQL_Latin1_General_Pref_CP
[CFSOrigin] [char] (5) COLLATE SQL_Latin1_General_Pref_CP
[CFSDestination] [char] (5) COLLATE SQL_Latin1_General_Pref_CP
PRIMARY KEY CLUSTERED
(
[SailingSchedule_id]
) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [tbl_SailingSchedule_Routi
[RoutingDetails_id] uniqueidentifier ROWGUIDCOL NOT NULL CONSTRAINT [DF_tbl_SailingSchedule_Ro
[RoutingDetails_SailingId]
[StageQualifier] [int] NULL ,
[Transportmode] [int] NULL ,
[TransportName] [nvarchar] (255) COLLATE SQL_Latin1_General_Pref_CP
[IMONumber] [int] NULL ,
[Origin] [char] (5) COLLATE SQL_Latin1_General_Pref_CP
[Destination] [char] (5) COLLATE SQL_Latin1_General_Pref_CP
[ETD] [datetime] NULL ,
[ETA] [datetime] NULL ,
PRIMARY KEY CLUSTERED
(
[RoutingDetails_id]
) ON [PRIMARY] ,
CONSTRAINT [FK_tbl_SailingSchedule_Ro
(
[RoutingDetails_SailingId]
) REFERENCES [tbl_SailingSchedule] (
[SailingSchedule_id]
) ON DELETE CASCADE ON UPDATE CASCADE
) ON [PRIMARY]
GO
==========================
SOURCE DATA:
<Schedule >
<!-- <ScheduleEnvelope>
<SenderID>NACA</SenderID>
<ReceiverID>KNPROD</Receiv
<Password>TradingPartners<
<Type>SCHEDULE</Type>
<Version>1.0.7</Version>
<EnvelopeID>NACA-10708</En
</ScheduleEnvelope>
-->
<ScheduleDetails>
<VesselVoyageID>11499412</
<VesselName>KOOKABURRA 1</VesselName>
<IMONumber>9319571</IMONum
<Voyage>611</Voyage>
<CarrierSCAC>USLB</Carrier
<SCAC>NAQA</SCAC>
<AmsFlag>N</AmsFlag>
<CFSOrigin>USATL</CFSOrigi
<CFSDestination>AUADL</CFS
<RoutingDetails>
<StageQualifier>1</StageQu
<Transportmode>2</Transpor
<TransportName>RAIL</Trans
<IMONumber>
</IMONumber>
<Origin>USATL</Origin>
<ETD>2006-10-23</ETD>
<Destination>USLAX</Destin
<ETA>2006-10-31</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>3</StageQu
<Transportmode>4</Transpor
<TransportName>KOOKABURRA 1</TransportName>
<IMONumber>9319571</IMONum
<Origin>USLAX</Origin>
<ETD>2006-11-02</ETD>
<Destination>AUMEL</Destin
<ETA>2006-11-22</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>5</StageQu
<Transportmode>5</Transpor
<TransportName>TBN</Transp
<IMONumber>
</IMONumber>
<Origin>AUMEL</Origin>
<ETD>2006-11-22</ETD>
<Destination>AUADL</Destin
<ETA>2006-11-25</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>6</StageQu
<Transportmode>1</Transpor
<TransportName>TRUCK</Tran
<IMONumber>
</IMONumber>
<Origin>AUADL</Origin>
<ETD>2006-11-25</ETD>
<Destination>AUADL</Destin
<ETA>2006-11-25</ETA>
</RoutingDetails>
</ScheduleDetails>
<ScheduleDetails>
<VesselVoyageID>11499786</
<VesselName>MAERSK DALE</VesselName>
<IMONumber>9232577</IMONum
<Voyage>608</Voyage>
<CarrierSCAC>HLCU</Carrier
<SCAC>NAQA</SCAC>
<AmsFlag>N</AmsFlag>
<CFSOrigin>USATL</CFSOrigi
<CFSDestination>NZAKL</CFS
<RoutingDetails>
<StageQualifier>1</StageQu
<Transportmode>1</Transpor
<TransportName>TRUCK</Tran
<IMONumber>
</IMONumber>
<Origin>USATL</Origin>
<ETD>2006-10-23</ETD>
<Destination>USSAV</Destin
<ETA>2006-10-28</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>2</StageQu
<Transportmode>4</Transpor
<TransportName>MAERSK DALE</TransportName>
<IMONumber>9232577</IMONum
<Origin>USSAV</Origin>
<ETD>2006-10-28</ETD>
<Destination>NZAKL</Destin
<ETA>2006-11-17</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>6</StageQu
<Transportmode>1</Transpor
<TransportName>TRUCK</Tran
<IMONumber>
</IMONumber>
<Origin>NZAKL</Origin>
<ETD>2006-11-17</ETD>
<Destination>NZAKL</Destin
<ETA>2006-11-17</ETA>
</RoutingDetails>
</ScheduleDetails>
</Schedule>
==========================
CODE
static private void InsertData()
{
//JAS_Generated_GUID
System.Xml.XmlDocument soDoc = new System.Xml.XmlDocument();
soDoc.Load(@"..\..\SMALL_S
System.Xml.XmlNodeList soDetailsNodes = soDoc.DocumentElement.Sele
foreach (System.Xml.XmlNode soNode in soDetailsNodes)
{
System.Xml.XmlNode soNewNode = soDoc.CreateElement("Saili
soNewNode.InnerText= System.Guid.NewGuid().ToSt
soNode.AppendChild(soNewNo
}
soDoc.Save(@"..\..\SMALL_S
try
{
SQLXMLBULKLOADLib.SQLXMLBu
objBL.ConnectionString = "Provider=sqloledb;server=
objBL.ErrorLogFile = "error.xml";
objBL.KeepIdentity = true;
objBL.Execute (@"..\..\SailingsheduleToJ
}
catch(Exception e)
{
Console.WriteLine(e.ToStri
}
}
==========================
ERROR:
System.Runtime.InteropServ
at SQLXMLBULKLOADLib.SQLXMLBu
ct vDataFile)
at SailingSchedule.Class1.Ins
studio projects\nacalogistics\sai
Your SailingSchedule element doesn't contain a SailingSchedule_id. This is what would be used to populate the erroring field. SQLXMLBulkLoad can't retrieve the default value generated by the database!
ASKER
Yes, I know that. This is why I have put in place code to generate a SailingSchedule_id element for each SailingSchedule node.
Although I can update the source XML to have an ID, it still does not propogate down to the child.
Please explain how to make it work.
Although I can update the source XML to have an ID, it still does not propogate down to the child.
Please explain how to make it work.
So with this element in your XML, you are still getting exactly the same error message?
(Sorry, BTW, just re-read your question properly!)
ASKER
Yes.
I just noticed a mistake in the above code pasted, the Execute() line should load the new, edited XML:
objBL.Execute (@"..\..\SailingsheduleToJ asDb_Mappi ng.xml", @"..\..\SMALL_SAMPLE_SAIL_ SCHED2With Data_WITHI D.XML"); //correct
objBL.Execute (@"..\..\SailingsheduleToJ asDb_Mappi ng.xml", @"..\..\SMALL_SAMPLE_SAIL_ SCHED2With Data.XML") ; // wrong
But the error message is the same.
I just noticed a mistake in the above code pasted, the Execute() line should load the new, edited XML:
objBL.Execute (@"..\..\SailingsheduleToJ
objBL.Execute (@"..\..\SailingsheduleToJ
But the error message is the same.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Or, could I put this node at the end of the schema?
i.e.
<?xml version="1.0" encoding="utf-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sql="urn:schemas-mic rosoft-com :mapping-s chema">
<xsd:annotation>
<xsd:appinfo>
<sql:relationship name="FK_tbl_SailingSchedu le_Routing Details_tb l_SailingS chedule"
parent="tbl_SailingSchedul e"
parent-key="SailingSchedul e_id"
child="tbl_SailingSchedule _RoutingDe tails"
child-key="RoutingDetails_ SailingId" />
</xsd:appinfo>
</xsd:annotation>
<xsd:element name="ScheduleDetails" sql:relation="tbl_SailingS chedule" sql:key-fields="SailingSch edule_id" >
<xsd:complexType>
<xsd:sequence>
<!-- **** ID ELEMENT REMOVED *** -->
<xsd:element name="VesselVoyageID" type="xsd:string" />
<xsd:element name="VesselName" type="xsd:string" />
<xsd:element name="IMONumber" type="xsd:string" />
<xsd:element name="Voyage" type="xsd:string" />
<xsd:element name="CarrierSCAC" type="xsd:string" />
<xsd:element name="SCAC" type="xsd:string" />
<xsd:element name="AmsFlag" type="xsd:string" />
<xsd:element name="CFSOrigin" type="xsd:string" />
<xsd:element name="CFSDestination" type="xsd:string" />
<xsd:element name="RoutingDetails"
sql:relationship="FK_tbl_S ailingSche dule_Routi ngDetails_ tbl_Sailin gSchedule"
sql:relation="tbl_SailingS chedule_Ro utingDetai ls" sql:key-fields="RoutingDet ails_id"
>
<xsd:complexType>
<xsd:sequence maxOccurs="unbounded">
<xsd:element name="StageQualifier" type="xsd:string" />
<xsd:element name="Transportmode" type="xsd:string" />
<xsd:element name="TransportName" type="xsd:string" />
<xsd:element name="IMONumber" type="xsd:string" />
<xsd:element name="Origin" type="xsd:string" />
<xsd:element name="ETD" type="xsd:string" />
<xsd:element name="Destination" type="xsd:string" />
<xsd:element name="ETA" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<!-- *** ID ELEMENT INSERTED AT BOTTOM **** -->
<xsd:element name="SailingSchedule_id" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
i.e.
<?xml version="1.0" encoding="utf-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sql="urn:schemas-mic
<xsd:annotation>
<xsd:appinfo>
<sql:relationship name="FK_tbl_SailingSchedu
parent="tbl_SailingSchedul
parent-key="SailingSchedul
child="tbl_SailingSchedule
child-key="RoutingDetails_
</xsd:appinfo>
</xsd:annotation>
<xsd:element name="ScheduleDetails" sql:relation="tbl_SailingS
<xsd:complexType>
<xsd:sequence>
<!-- **** ID ELEMENT REMOVED *** -->
<xsd:element name="VesselVoyageID" type="xsd:string" />
<xsd:element name="VesselName" type="xsd:string" />
<xsd:element name="IMONumber" type="xsd:string" />
<xsd:element name="Voyage" type="xsd:string" />
<xsd:element name="CarrierSCAC" type="xsd:string" />
<xsd:element name="SCAC" type="xsd:string" />
<xsd:element name="AmsFlag" type="xsd:string" />
<xsd:element name="CFSOrigin" type="xsd:string" />
<xsd:element name="CFSDestination" type="xsd:string" />
<xsd:element name="RoutingDetails"
sql:relationship="FK_tbl_S
sql:relation="tbl_SailingS
>
<xsd:complexType>
<xsd:sequence maxOccurs="unbounded">
<xsd:element name="StageQualifier" type="xsd:string" />
<xsd:element name="Transportmode" type="xsd:string" />
<xsd:element name="TransportName" type="xsd:string" />
<xsd:element name="IMONumber" type="xsd:string" />
<xsd:element name="Origin" type="xsd:string" />
<xsd:element name="ETD" type="xsd:string" />
<xsd:element name="Destination" type="xsd:string" />
<xsd:element name="ETA" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<!-- *** ID ELEMENT INSERTED AT BOTTOM **** -->
<xsd:element name="SailingSchedule_id" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
ASKER
The re-ordered schema made no difference.
Let me try the PrependChild() method.... but I don't think it will make any difference. It's not that (parent) table tbl_SailingSchedule can't find its PK, it's that (child) table tbl_SailingSchedule_Routin gDetails can't find its FK.
Let me try the PrependChild() method.... but I don't think it will make any difference. It's not that (parent) table tbl_SailingSchedule can't find its PK, it's that (child) table tbl_SailingSchedule_Routin
The parent table won't error if it doesn't find its PK, though, will it? You've provided a default. Let me know if PrependChild helps at all, and I'll keep thinking.
Capitalisation - your C# code is adding an element named SailingSchedule_id, when it should be SailingSchedule_Id. This plus PrependChild should do it with a bit of luck.
Ignore last - eyes swimming, they seem to all be consistent on second look. Sorry.
Does it help to remove this attribute from the child element?
sql:key-fields="RoutingDet ails_id"
sql:key-fields="RoutingDet
ASKER
The error message is now:
System.Runtime.InteropServ ices.COMEx ception (0x80004005): Invalid character value for cast specification.
at SQLXMLBULKLOADLib.SQLXMLBu lkLoad3Cla ss.Execute (String bstrSchemaFile, Obje
ct vDataFile)
at SailingSchedule.Class1.Ins ertData() in d:\daniel documents and work\visual studio projects\nacalogistics\sai lingschedu le\class1. cs:line 66
Could this mean that it's now trying to insert the data into the child row but somehow the ID I generate is duff?
The edited XML looks like:
<ScheduleDetails>
<SailingSchedule_id>ad754b d6-c30d-4f 3e-9e4c-c5 3272159518 </SailingS chedule_id >
<VesselVoyageID>11499412</ VesselVoya geID>
<VesselName>KOOKABURRA 1</VesselName>
<IMONumber>9319571</IMONum ber>
<Voyage>611</Voyage>
<CarrierSCAC>USLB</Carrier SCAC>
<SCAC>NAQA</SCAC>
<AmsFlag>N</AmsFlag>
<CFSOrigin>USATL</CFSOrigi n>
<CFSDestination>AUADL</CFS Destinatio n>
<RoutingDetails>
<StageQualifier>1</StageQu alifier>
<Transportmode>2</Transpor tmode>
<TransportName>RAIL</Trans portName>
<IMONumber>
</IMONumber>
<Origin>USATL</Origin>
<ETD>2006-10-23</ETD>
<Destination>USLAX</Destin ation>
<ETA>2006-10-31</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>3</StageQu alifier>
<Transportmode>4</Transpor tmode>
<TransportName>KOOKABURRA 1</TransportName>
<IMONumber>9319571</IMONum ber>
<Origin>USLAX</Origin>
<ETD>2006-11-02</ETD>
<Destination>AUMEL</Destin ation>
<ETA>2006-11-22</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>5</StageQu alifier>
<Transportmode>5</Transpor tmode>
<TransportName>TBN</Transp ortName>
<IMONumber>
</IMONumber>
<Origin>AUMEL</Origin>
<ETD>2006-11-22</ETD>
<Destination>AUADL</Destin ation>
<ETA>2006-11-25</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>6</StageQu alifier>
<Transportmode>1</Transpor tmode>
<TransportName>TRUCK</Tran sportName>
<IMONumber>
</IMONumber>
<Origin>AUADL</Origin>
<ETD>2006-11-25</ETD>
<Destination>AUADL</Destin ation>
<ETA>2006-11-25</ETA>
</RoutingDetails>
</ScheduleDetails>
... snip ...
System.Runtime.InteropServ
at SQLXMLBULKLOADLib.SQLXMLBu
ct vDataFile)
at SailingSchedule.Class1.Ins
Could this mean that it's now trying to insert the data into the child row but somehow the ID I generate is duff?
The edited XML looks like:
<ScheduleDetails>
<SailingSchedule_id>ad754b
<VesselVoyageID>11499412</
<VesselName>KOOKABURRA 1</VesselName>
<IMONumber>9319571</IMONum
<Voyage>611</Voyage>
<CarrierSCAC>USLB</Carrier
<SCAC>NAQA</SCAC>
<AmsFlag>N</AmsFlag>
<CFSOrigin>USATL</CFSOrigi
<CFSDestination>AUADL</CFS
<RoutingDetails>
<StageQualifier>1</StageQu
<Transportmode>2</Transpor
<TransportName>RAIL</Trans
<IMONumber>
</IMONumber>
<Origin>USATL</Origin>
<ETD>2006-10-23</ETD>
<Destination>USLAX</Destin
<ETA>2006-10-31</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>3</StageQu
<Transportmode>4</Transpor
<TransportName>KOOKABURRA 1</TransportName>
<IMONumber>9319571</IMONum
<Origin>USLAX</Origin>
<ETD>2006-11-02</ETD>
<Destination>AUMEL</Destin
<ETA>2006-11-22</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>5</StageQu
<Transportmode>5</Transpor
<TransportName>TBN</Transp
<IMONumber>
</IMONumber>
<Origin>AUMEL</Origin>
<ETD>2006-11-22</ETD>
<Destination>AUADL</Destin
<ETA>2006-11-25</ETA>
</RoutingDetails>
<RoutingDetails>
<StageQualifier>6</StageQu
<Transportmode>1</Transpor
<TransportName>TRUCK</Tran
<IMONumber>
</IMONumber>
<Origin>AUADL</Origin>
<ETD>2006-11-25</ETD>
<Destination>AUADL</Destin
<ETA>2006-11-25</ETA>
</RoutingDetails>
</ScheduleDetails>
... snip ...
ASKER
The error log file from the bulk load library looks like this:
<?xml version="1.0" ?>
- <Error>
- <Record>
<HResult>0x80004005</HResu lt>
<SQLState>22018</SQLState>
<NativeError />
<Source>Microsoft OLE DB Provider for SQL Server</Source>
- <Description>
- <![CDATA[ Invalid character value for cast specification.
]]>
</Description>
</Record>
</Error>
<?xml version="1.0" ?>
- <Error>
- <Record>
<HResult>0x80004005</HResu
<SQLState>22018</SQLState>
<NativeError />
<Source>Microsoft OLE DB Provider for SQL Server</Source>
- <Description>
- <![CDATA[ Invalid character value for cast specification.
]]>
</Description>
</Record>
</Error>
Some sort of error converting the string to a GUID would be my guess. The syntax looks OK to me. Perhaps try converting the GUID to upper case like in the MSDN examples involving GUIDs? (SHOULDN'T make a difference, but this is M$ we're talking about).
IMONumber is an int column - if there is no value in your XML, you must omit the element, otherwise SQL is trying to cast an empty string to an int! Check all other non-string columns as well.
It may be nothing to do with the GUIDs at all now.
It may be nothing to do with the GUIDs at all now.
ASKER
I have already coverted all the collumns to be very lenient - nullable large nvarchars. Still same problem.
Let me try upper case guids.
Let me try upper case guids.
ASKER
Cracked it.
Need upper case AND {braces} around my guid.
Must be:
{D868513C-99B1-467A-A26F-5 93715C061E E}
Then it works fine.
Thanks for your help.
Need upper case AND {braces} around my guid.
Must be:
{D868513C-99B1-467A-A26F-5
Then it works fine.
Thanks for your help.
Yee hah! The joys of bulk load, eh?
Glad we got there in the end.
Glad we got there in the end.
ASKER
Cheers.
Just out of interest, how on earth does it work? I see from a profiler trace that a statemen that starts with the text "INSERT BULK tbl_SailingSchedule...." but nowhere ever do I see a filename or data values being passed. Weird.
Still, job done.
Just out of interest, how on earth does it work? I see from a profiler trace that a statemen that starts with the text "INSERT BULK tbl_SailingSchedule...." but nowhere ever do I see a filename or data values being passed. Weird.
Still, job done.