Solved

Merging 2 XML without duplicates

Posted on 2009-05-05
13
1,076 Views
Last Modified: 2013-12-17
I have 2 source xml files (test1.xml and test2.xml) when I used the following program it generated the resultant xml file (test3.xml); but I see that values got repeated. Can someone adviice how to avoid duplicates.

Test1.XML
<GSMThreshold>
  <BasicMetrics>
    <MetricCategory>GSM Pilot<MetricName>GSM Dominant Pilot EcIo<Rec1><Threshold>-11</Threshold><Color>16711935</Color></Rec1><Rec2><Threshold>-9</Threshold><Color>16711935</Color></Rec2></MetricName></MetricCategory>
  </BasicMetrics>

Test2.XML

<?xml version="1.0" encoding="utf-8"?>
<GSMThreshold>
  <BasicMetrics>
    <MetricCategory>GSM Pilot<MetricName>GSM Dominant Pilot EcIo<Rec1><Threshold>-11</Threshold><Color>16776960</Color></Rec1><Rec2><Threshold>-9</Threshold><Color>255</Color></Rec2><Rec3><Threshold>-6</Threshold><Color>16711680</Color></Rec3></MetricName></MetricCategory>
  </BasicMetrics>
</GSMThreshold>

Test3.xml
<?xml version="1.0" standalone="yes"?>
<GSMThreshold>
  <BasicMetrics>
    <MetricCategory>
      <MetricName>
        <Rec1>
          <Threshold>-11</Threshold>
          <Color>16711935</Color>
        </Rec1>
        <Rec1>
          <Threshold>-11</Threshold>
          <Color>16776960</Color>
        </Rec1>
        <Rec2>
          <Threshold>-9</Threshold>
          <Color>16711935</Color>
        </Rec2>
        <Rec2>
          <Threshold>-9</Threshold>
          <Color>255</Color>
        </Rec2>
        <Rec3>
          <Threshold>-6</Threshold>
          <Color>16711680</Color>
        </Rec3>
      </MetricName>
    </MetricCategory>
  </BasicMetrics>
</GSMThreshold>


private void button1_Click(object sender, EventArgs e)

        {

            try

            {

                XmlTextReader xmlreader1 = new XmlTextReader("C:\\temp\\Test1.xml");

                XmlTextReader xmlreader2 = new XmlTextReader("C:\\temp\\Test2.xml");
 

                DataSet ds = new DataSet();

                ds.ReadXml(xmlreader1);

                DataSet ds2 = new DataSet();

                ds2.ReadXml(xmlreader2);

                ds.Merge(ds2);

                ds.WriteXml("C:\\temp\\Test3.xml");

                 }

            catch (System.Exception ex)

            {

                Console.Write(ex.Message);

            }

            Console.Read();

        }

Open in new window

0
Comment
Question by:shanvidhya
  • 6
  • 5
  • 2
13 Comments
 
LVL 12

Expert Comment

by:williamcampbell
ID: 24307246
Manually I think ... Something like below
               DataTable tempTable = ds.Copy();  

	            tempTable.Merge(ds2, false); 
 

	            foreach (DataRow dr in ds.Rows)  

	                foreach (DataRow dr2 in ds2.Rows)  

	                {  

	                    if (dr["id"].Equals(dr2["id"]))  

	                    {  

	                        DataRow findRow = null;  

	                        findRow = tempTable.Rows.Find(dr["id"]);  

	                        if (findRow != null)  

	                            tempTable.Rows.Remove(findRow);  

	                        break;  

	                    }  

	                }  
 

              // tempTable has result

Open in new window

0
 

Author Comment

by:shanvidhya
ID: 24307412
William,

I beleive possibly there could be following problems in your suggestion:

DataTable tempTable = ds.Copy();  --> May not allow copying dataset to datatable
                  tempTable.Merge(ds2, false);  --> First parameter suppose to be datatable
 
                  foreach (DataRow dr in ds.Rows)  --> instead of dataset.rows it should be datatable.rows

Please advice
0
 
LVL 23

Expert Comment

by:Tiggerito
ID: 24312883
0
 
LVL 12

Expert Comment

by:williamcampbell
ID: 24318092
My Code was a guideline about how to do it you will have to make some minor change to get it to work.

Do you want me to write the code for you?
0
 

Author Comment

by:shanvidhya
ID: 24356533
Tiggerito, the following code which I wrote works fine; but seems to be too long; is there any way to optimize this code. Also, sometimes SAVE thows error saying that file is been used by a different process. Any ideas?

private void MergeXML()
        {
            XmlDocument oSourceXML = new XmlDocument();
            XmlDocument oUserXML = new XmlDocument();
            XmlNode oU1Node = null;
            XmlNode oCategory = null;
            XmlNode oT = null;
            XmlNode ot1 = null;
            bool bFound = false;
            bool bGroupFound = false;
            XmlNode oMetricName = null;
            XmlNode oColor = null; //Recs
            XmlNode oThreshold = null; //Recs
            XmlNode oRec = null; //Recs
            XmlNode oPlotType = null;
            XmlNode oRangeMin = null;
            XmlNode oRangeMax = null;
            int iLoop = 0;

            //Console.WriteLine(Username(1))

            if (File.Exists (UserSettingsPath + "\\" + UserThreshold))
            {
                oSourceXML.Load(Application.StartupPath + "\\" + SourceThreshold);
                oUserXML.Load(UserSettingsPath + "\\" + UserThreshold);
            }
            else
            {
                return;
            }

            foreach (XmlNode oS1Node in oSourceXML.DocumentElement.ChildNodes)
            {

                foreach (XmlNode oU1NodeWithinLoop in oUserXML.DocumentElement.ChildNodes)
                {
                    oU1Node = oU1NodeWithinLoop;

                    foreach (XmlNode oTWithinLoop in oS1Node.ChildNodes) //Source Groups
                    {
                        oT = oTWithinLoop;

                        bGroupFound = false;

                        foreach (XmlNode ot1WithinLoop in oU1NodeWithinLoop.ChildNodes) //User Groups
                        {
                            ot1 = ot1WithinLoop;

                            foreach (XmlNode oDef in oTWithinLoop.ChildNodes) //Source Metrics
                            {
                                bFound = false;

                                foreach (XmlNode oMod in ot1WithinLoop.ChildNodes) //User Metrics
                                {

                                    if ((oTWithinLoop.ChildNodes[0].Value == ot1WithinLoop.ChildNodes[0].Value) & (oDef.ChildNodes.Count > 0))
                                    {
                                        bGroupFound = true;
                                        if (oMod.ChildNodes.Count > 0)
                                        {
                                            if (oDef.ChildNodes[0].Value == oMod.ChildNodes[0].Value)
                                            {
                                                bFound = true;
                                                goto NextMetric;
                                            }
                                        }

                                    }
                                    else
                                    {
                                        goto NextMetric;
                                    }

                                } //End User Metrics

                                if (bGroupFound && (!bFound))
                                {
                                    iLoop = 0;
                                    foreach (XmlNode oTemp in oDef.ChildNodes)
                                    {

                                        if (oTemp.ChildNodes.Count == 0)
                                        {

                                            oMetricName = oUserXML.CreateElement("MetricName");
                                            oMetricName.InnerText = Convert.ToString((oTemp.InnerText));
                                            ot1WithinLoop.AppendChild(oMetricName);
                                        }
                                        else if ((oTemp.Name.IndexOf("PlotType", 0) + 1) > 0)
                                        {
                                            oPlotType = oUserXML.CreateElement("PlotType");
                                            oPlotType.InnerText = oTemp.InnerText;
                                            oMetricName.AppendChild(oPlotType);
                                        }
                                        else if ((oTemp.Name.IndexOf("RangeMax", 0) + 1) > 0)
                                        {
                                            oRangeMax = oUserXML.CreateElement("RangeMax");
                                            oRangeMax.InnerText = oTemp.InnerText;
                                            oMetricName.AppendChild(oRangeMax);
                                        }
                                        else if ((oTemp.Name.IndexOf("RangeMin", 0) + 1) > 0)
                                        {
                                            oRangeMin = oUserXML.CreateElement("RangeMin");
                                            oRangeMin.InnerText = oTemp.InnerText;
                                            oMetricName.AppendChild(oRangeMin);
                                        }
                                        else if ((oTemp.Name.IndexOf("Rec", 0) + 1) > 0)
                                        {

                                            iLoop = iLoop + 1;

                                            oRec = oUserXML.CreateElement("Rec" + iLoop);
                                            oMetricName.AppendChild(oRec);

                                            foreach (XmlNode oTemp1 in oTemp.ChildNodes)
                                            {

                                                if (oTemp1.Name == "Threshold")
                                                {
                                                    oThreshold = oUserXML.CreateElement("Threshold");
                                                    oThreshold.InnerText = oTemp1.InnerText;
                                                    oRec.AppendChild(oThreshold);
                                                }
                                                else
                                                {
                                                    oColor = oUserXML.CreateElement("Color");
                                                    oColor.InnerText = oTemp1.InnerText;
                                                    oRec.AppendChild(oColor);
                                                }
                                            }
                                        }
                                    }

                                }
                            NextMetric: ;
                            } //End Source Metrics

                        } //End User Groups

                        if (!bGroupFound)
                        {
                            Console.WriteLine("Add Group if not found");

                            foreach (XmlNode oTemp in oTWithinLoop.ChildNodes)
                            {

                                if (oTemp.ChildNodes.Count == 0)
                                {

                                    oCategory = oUserXML.CreateElement("MetricCategory");
                                    oCategory.InnerText = oTWithinLoop.ChildNodes[0].InnerText;
                                    oU1NodeWithinLoop.AppendChild(oCategory);
                                }
                                else
                                {

                                    foreach (XmlNode oTemp1 in oTemp.ChildNodes)
                                    {

                                        if (oTemp1.ChildNodes.Count == 0)
                                        {
                                            oMetricName = oUserXML.CreateElement("MetricName");
                                            oMetricName.InnerText = oTemp1.InnerText;
                                            oCategory.AppendChild(oMetricName);
                                        }
                                        else if ((oTemp1.Name.IndexOf("PlotType", 0) + 1) > 0)
                                        {
                                            oPlotType = oUserXML.CreateElement("PlotType");
                                            oPlotType.InnerText = oTemp1.InnerText;
                                            oMetricName.AppendChild(oPlotType);
                                        }
                                        else if ((oTemp1.Name.IndexOf("RangeMax", 0) + 1) > 0)
                                        {
                                            oRangeMax = oUserXML.CreateElement("RangeMax");
                                            oRangeMax.InnerText = oTemp1.InnerText;
                                            oMetricName.AppendChild(oRangeMax);
                                        }
                                        else if ((oTemp1.Name.IndexOf("RangeMin", 0) + 1) > 0)
                                        {
                                            oRangeMin = oUserXML.CreateElement("RangeMin");
                                            oRangeMin.InnerText = oTemp1.InnerText;
                                            oMetricName.AppendChild(oRangeMin);
                                        }
                                        else if ((oTemp1.Name.IndexOf("Rec", 0) + 1) > 0)
                                        {
                                            iLoop = iLoop + 1;

                                            oRec = oUserXML.CreateElement("Rec" + iLoop);
                                            oMetricName.AppendChild(oRec);

                                            foreach (XmlNode oTemp2 in oTemp1.ChildNodes)
                                            {

                                                if (oTemp2.Name == "Threshold")
                                                {
                                                    oThreshold = oUserXML.CreateElement("Threshold");
                                                    oThreshold.InnerText = oTemp2.InnerText;
                                                    oRec.AppendChild(oThreshold);
                                                }
                                                else
                                                {
                                                    oColor = oUserXML.CreateElement("Color");
                                                    oColor.InnerText = oTemp2.InnerText;
                                                    oRec.AppendChild(oColor);
                                                }

                                            }
                                        }
                                    }
                                }
                            }

                        }

                    } //End Source Groups

                }

            }

            oUserXML.Save(UserSettingsPath + "\\" + UserThreshold);

            oUserXML = null;

        }
0
 
LVL 23

Expert Comment

by:Tiggerito
ID: 24362938
Your code implies theres a lot more to the XML than you previously stated.

Could you expaline the rules/structure related to elements like PlotType, RangeMax etc.

A complete  XML example would help.

I've also noticed your example xml countans mixed text and elements inside the MetricCategory element. is this correct? It seems you are ignoring it? This mixed content text/xml is normally considered bad practice.

What are the rules on merging. I understand that the Rec? elements would merge. How do you determine that other elements are the same and therefore mergable?

What are the rules when you merge. B overwrites A?

Your code is very hard to follow, has lots of looping and goto's! Some comments would help.
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 

Author Comment

by:shanvidhya
ID: 24376501
Tiggerito,

Please find the attached sxreenshots of the Source XML, User XML and the final Merged XML with descriptions. Please let me know if you need more info.

Thanks,
Shan
MergeXML.doc
0
 
LVL 23

Expert Comment

by:Tiggerito
ID: 24383087
So the intial #text nodes are the way elements are detected as elements to merge.

You should realy change this to an attribute. As I stated earlier, mixing text and elements is bad practice.

If you change it to an attribute called 'name' you can directly use my generic solution offered here:

http://www.experts-exchange.com/Programming/Languages/C_Sharp/Q_23999589.html

If you want to stick with the text nodes then you could re-work line 40 onwards to find mergable elements in your own way.
0
 

Author Comment

by:shanvidhya
ID: 24395483
Tiggerito,

I am not sure about your suggestion. I am a learner in the xml; so I am missing many things. I am attaching the XML file renamed as **.doc; please download and rename the extension as xml. If would be a big help if you can please change the type of each nodes in this xml as per your suggestion; I can change the program accordingly or try to use the link you suggested.  

Thanks,

GSMThreshold.doc
0
 
LVL 23

Expert Comment

by:Tiggerito
ID: 24396466
This is how I suggest you format your XML.

Some people use "id" as the attribute in place of "name" but my previous code example is based on "name".

At this point it could be worth upping the points!
<?xml version="1.0" encoding="utf-8"?>

<GSMThreshold>

	<BasicMetrics>

		<MetricCategory name="GSM Pilot">

			<MetricName name="GSM Dominant Pilot EcIo">

				<PlotType>Range</PlotType>

				<RangeMax>0</RangeMax>

				<RangeMin>-120</RangeMin>

				<Rec name="1">

					<Threshold>-11</Threshold>

					<Color>13749760</Color>

				</Rec>

				<Rec name="2">

					<Threshold>-9</Threshold>

					<Color>5275647</Color>

				</Rec>

				<Rec name="3">

					<Threshold>-3</Threshold>

					<Color>12695295</Color>

				</Rec>

			</MetricName>

			<MetricName name="GSM Pilot Scan Ec">

				<PlotType>Range</PlotType>

				<RangeMax>0</RangeMax>

				<RangeMin>-120</RangeMin>

				<Rec name="1">

					<Threshold>-80</Threshold>

					<Color>13828244</Color>

				</Rec>

				<Rec name="2">

					<Threshold>-75</Threshold>

					<Color>3937500</Color>

				</Rec>

				<Rec name="3">

					<Threshold>-60</Threshold>

					<Color>36095</Color>

				</Rec>

			</MetricName>

		</MetricCategory>

		<MetricCategory name="GSM Power">

			<MetricName name="GSM Best Neighbor Rx Level">

				<PlotType>Range</PlotType>

				<RangeMax>0</RangeMax>

				<RangeMin>-120</RangeMin>

				<Rec name="1">

					<Threshold>-120</Threshold>

					<Color>-2147483646</Color>

				</Rec>

				<Rec name="2">

					<Threshold>-105</Threshold>

					<Color>8421376</Color>

				</Rec>

				<Rec name="3">

					<Threshold>-100</Threshold>

					<Color>16761087</Color>

				</Rec>

				<Rec name="4">

					<Threshold>-95</Threshold>

					<Color>255</Color>

				</Rec>

				<Rec name="5">

					<Threshold>-90</Threshold>

					<Color>8438015</Color>

				</Rec>

				<Rec name="6">

					<Threshold>-85</Threshold>

					<Color>65535</Color>

				</Rec>

				<Rec name="7">

					<Threshold>-80</Threshold>

					<Color>65280</Color>

				</Rec>

				<Rec name="8">

					<Threshold>-75</Threshold>

					<Color>16711680</Color>

				</Rec>

				<Rec name="9">

					<Threshold>-70</Threshold>

					<Color>16776960</Color>

				</Rec>

			</MetricName>

			<MetricName name="GSM Serving Cell Rx Level Full">

				<PlotType>Range</PlotType>

				<RangeMax>0</RangeMax>

				<RangeMin>-120</RangeMin>

				<Rec name="1">

					<Threshold>-120</Threshold>

					<Color>16711935</Color>

				</Rec>

				<Rec name="2">

					<Threshold>-105</Threshold>

					<Color>8421631</Color>

				</Rec>

				<Rec name="3">

					<Threshold>-100</Threshold>

					<Color>16761087</Color>

				</Rec>

				<Rec name="4">

					<Threshold>-95</Threshold>

					<Color>255</Color>

				</Rec>

				<Rec name="5">

					<Threshold>-90</Threshold>

					<Color>8438015</Color>

				</Rec>

				<Rec name="6">

					<Threshold>-85</Threshold>

					<Color>65535</Color>

				</Rec>

				<Rec name="7">

					<Threshold>-80</Threshold>

					<Color>65280</Color>

				</Rec>

				<Rec name="8">

					<Threshold>-75</Threshold>

					<Color>16711680</Color>

				</Rec>

				<Rec name="9">

					<Threshold>-70</Threshold>

					<Color>16776960</Color>

				</Rec>

			</MetricName>

			<MetricName name="GSM Serving Cell Rx Level Sub">

				<PlotType>Range</PlotType>

				<RangeMax>0</RangeMax>

				<RangeMin>-125</RangeMin>

				<Rec name="1">

					<Threshold>-120</Threshold>

					<Color>16711935</Color>

				</Rec>

				<Rec name="2">

					<Threshold>-105</Threshold>

					<Color>255</Color>

				</Rec>

				<Rec name="3">

					<Threshold>-100</Threshold>

					<Color>16761087</Color>

				</Rec>

				<Rec name="4">

					<Threshold>-95</Threshold>

					<Color>255</Color>

				</Rec>

				<Rec name="5">

					<Threshold>-90</Threshold>

					<Color>8438015</Color>

				</Rec>

				<Rec name="6">

					<Threshold>-85</Threshold>

					<Color>65535</Color>

				</Rec>

				<Rec name="7">

					<Threshold>-80</Threshold>

					<Color>65280</Color>

				</Rec>

				<Rec name="8">

					<Threshold>-75</Threshold>

					<Color>16711680</Color>

				</Rec>

				<Rec name="9">

					<Threshold>-70</Threshold>

					<Color>16776960</Color>

				</Rec>

			</MetricName>

		</MetricCategory>

	</BasicMetrics>

</GSMThreshold>

Open in new window

0
 

Author Comment

by:shanvidhya
ID: 24397090
Tiggerito,

I tried your code with the following function and xml you provided; but it didnt generate the merge xml properly.

Test1 is the modified xml you provided
Test2 is the test1 xml with few nodes deleted

the resultant xml test3 was same as test2.xml


private void btnProcess_Click(object sender, EventArgs e)
        {

            XmlDocument oSourceXML = new XmlDocument();
            XmlDocument oUserXML = new XmlDocument();

            oSourceXML.Load("C:\\temp1.xml");
            oUserXML.Load("C:\\temp2.xml");

            Merge(oSourceXML, oUserXML);

            oUserXML.Save("C:\\temp3.xml");
            oUserXML = null;
            oSourceXML= null;

        }

once this problem is fixed I will double the points.
0
 
LVL 23

Accepted Solution

by:
Tiggerito earned 250 total points
ID: 24397314
The code merges into oSourceXML and your saving oUserXML

setting local variables to null like that is not required in .Net
0
 

Author Closing Comment

by:shanvidhya
ID: 31578128
Thanks Tiggerito for your help. I will make changes to my approach in handling XML as per your suggestion.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
Access reports are powerful and flexible. Learn how to create a query and then a grouped report using the wizard. Modify the report design after the wizard is done to make it look better. There will be another video to explain how to put the final p…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now