Solved

Merging 2 XML without duplicates

Posted on 2009-05-05
13
1,091 Views
Last Modified: 2013-12-17
I have 2 source xml files (test1.xml and test2.xml) when I used the following program it generated the resultant xml file (test3.xml); but I see that values got repeated. Can someone adviice how to avoid duplicates.

Test1.XML
<GSMThreshold>
  <BasicMetrics>
    <MetricCategory>GSM Pilot<MetricName>GSM Dominant Pilot EcIo<Rec1><Threshold>-11</Threshold><Color>16711935</Color></Rec1><Rec2><Threshold>-9</Threshold><Color>16711935</Color></Rec2></MetricName></MetricCategory>
  </BasicMetrics>

Test2.XML

<?xml version="1.0" encoding="utf-8"?>
<GSMThreshold>
  <BasicMetrics>
    <MetricCategory>GSM Pilot<MetricName>GSM Dominant Pilot EcIo<Rec1><Threshold>-11</Threshold><Color>16776960</Color></Rec1><Rec2><Threshold>-9</Threshold><Color>255</Color></Rec2><Rec3><Threshold>-6</Threshold><Color>16711680</Color></Rec3></MetricName></MetricCategory>
  </BasicMetrics>
</GSMThreshold>

Test3.xml
<?xml version="1.0" standalone="yes"?>
<GSMThreshold>
  <BasicMetrics>
    <MetricCategory>
      <MetricName>
        <Rec1>
          <Threshold>-11</Threshold>
          <Color>16711935</Color>
        </Rec1>
        <Rec1>
          <Threshold>-11</Threshold>
          <Color>16776960</Color>
        </Rec1>
        <Rec2>
          <Threshold>-9</Threshold>
          <Color>16711935</Color>
        </Rec2>
        <Rec2>
          <Threshold>-9</Threshold>
          <Color>255</Color>
        </Rec2>
        <Rec3>
          <Threshold>-6</Threshold>
          <Color>16711680</Color>
        </Rec3>
      </MetricName>
    </MetricCategory>
  </BasicMetrics>
</GSMThreshold>


private void button1_Click(object sender, EventArgs e)
        {
            try
            {
                XmlTextReader xmlreader1 = new XmlTextReader("C:\\temp\\Test1.xml");
                XmlTextReader xmlreader2 = new XmlTextReader("C:\\temp\\Test2.xml");
 
                DataSet ds = new DataSet();
                ds.ReadXml(xmlreader1);
                DataSet ds2 = new DataSet();
                ds2.ReadXml(xmlreader2);
                ds.Merge(ds2);
                ds.WriteXml("C:\\temp\\Test3.xml");
                 }
            catch (System.Exception ex)
            {
                Console.Write(ex.Message);
            }
            Console.Read();
        }

Open in new window

0
Comment
Question by:shanvidhya
  • 6
  • 5
  • 2
13 Comments
 
LVL 12

Expert Comment

by:williamcampbell
ID: 24307246
Manually I think ... Something like below
               DataTable tempTable = ds.Copy();  
	            tempTable.Merge(ds2, false); 
 
	            foreach (DataRow dr in ds.Rows)  
	                foreach (DataRow dr2 in ds2.Rows)  
	                {  
	                    if (dr["id"].Equals(dr2["id"]))  
	                    {  
	                        DataRow findRow = null;  
	                        findRow = tempTable.Rows.Find(dr["id"]);  
	                        if (findRow != null)  
	                            tempTable.Rows.Remove(findRow);  
	                        break;  
	                    }  
	                }  
 
              // tempTable has result

Open in new window

0
 

Author Comment

by:shanvidhya
ID: 24307412
William,

I beleive possibly there could be following problems in your suggestion:

DataTable tempTable = ds.Copy();  --> May not allow copying dataset to datatable
                  tempTable.Merge(ds2, false);  --> First parameter suppose to be datatable
 
                  foreach (DataRow dr in ds.Rows)  --> instead of dataset.rows it should be datatable.rows

Please advice
0
 
LVL 23

Expert Comment

by:Tony McCreath
ID: 24312883
0
Resolve Critical IT Incidents Fast

If your data, services or processes become compromised, your organization can suffer damage in just minutes and how fast you communicate during a major IT incident is everything. Learn how to immediately identify incidents & best practices to resolve them quickly and effectively.

 
LVL 12

Expert Comment

by:williamcampbell
ID: 24318092
My Code was a guideline about how to do it you will have to make some minor change to get it to work.

Do you want me to write the code for you?
0
 

Author Comment

by:shanvidhya
ID: 24356533
Tiggerito, the following code which I wrote works fine; but seems to be too long; is there any way to optimize this code. Also, sometimes SAVE thows error saying that file is been used by a different process. Any ideas?

private void MergeXML()
        {
            XmlDocument oSourceXML = new XmlDocument();
            XmlDocument oUserXML = new XmlDocument();
            XmlNode oU1Node = null;
            XmlNode oCategory = null;
            XmlNode oT = null;
            XmlNode ot1 = null;
            bool bFound = false;
            bool bGroupFound = false;
            XmlNode oMetricName = null;
            XmlNode oColor = null; //Recs
            XmlNode oThreshold = null; //Recs
            XmlNode oRec = null; //Recs
            XmlNode oPlotType = null;
            XmlNode oRangeMin = null;
            XmlNode oRangeMax = null;
            int iLoop = 0;

            //Console.WriteLine(Username(1))

            if (File.Exists (UserSettingsPath + "\\" + UserThreshold))
            {
                oSourceXML.Load(Application.StartupPath + "\\" + SourceThreshold);
                oUserXML.Load(UserSettingsPath + "\\" + UserThreshold);
            }
            else
            {
                return;
            }

            foreach (XmlNode oS1Node in oSourceXML.DocumentElement.ChildNodes)
            {

                foreach (XmlNode oU1NodeWithinLoop in oUserXML.DocumentElement.ChildNodes)
                {
                    oU1Node = oU1NodeWithinLoop;

                    foreach (XmlNode oTWithinLoop in oS1Node.ChildNodes) //Source Groups
                    {
                        oT = oTWithinLoop;

                        bGroupFound = false;

                        foreach (XmlNode ot1WithinLoop in oU1NodeWithinLoop.ChildNodes) //User Groups
                        {
                            ot1 = ot1WithinLoop;

                            foreach (XmlNode oDef in oTWithinLoop.ChildNodes) //Source Metrics
                            {
                                bFound = false;

                                foreach (XmlNode oMod in ot1WithinLoop.ChildNodes) //User Metrics
                                {

                                    if ((oTWithinLoop.ChildNodes[0].Value == ot1WithinLoop.ChildNodes[0].Value) & (oDef.ChildNodes.Count > 0))
                                    {
                                        bGroupFound = true;
                                        if (oMod.ChildNodes.Count > 0)
                                        {
                                            if (oDef.ChildNodes[0].Value == oMod.ChildNodes[0].Value)
                                            {
                                                bFound = true;
                                                goto NextMetric;
                                            }
                                        }

                                    }
                                    else
                                    {
                                        goto NextMetric;
                                    }

                                } //End User Metrics

                                if (bGroupFound && (!bFound))
                                {
                                    iLoop = 0;
                                    foreach (XmlNode oTemp in oDef.ChildNodes)
                                    {

                                        if (oTemp.ChildNodes.Count == 0)
                                        {

                                            oMetricName = oUserXML.CreateElement("MetricName");
                                            oMetricName.InnerText = Convert.ToString((oTemp.InnerText));
                                            ot1WithinLoop.AppendChild(oMetricName);
                                        }
                                        else if ((oTemp.Name.IndexOf("PlotType", 0) + 1) > 0)
                                        {
                                            oPlotType = oUserXML.CreateElement("PlotType");
                                            oPlotType.InnerText = oTemp.InnerText;
                                            oMetricName.AppendChild(oPlotType);
                                        }
                                        else if ((oTemp.Name.IndexOf("RangeMax", 0) + 1) > 0)
                                        {
                                            oRangeMax = oUserXML.CreateElement("RangeMax");
                                            oRangeMax.InnerText = oTemp.InnerText;
                                            oMetricName.AppendChild(oRangeMax);
                                        }
                                        else if ((oTemp.Name.IndexOf("RangeMin", 0) + 1) > 0)
                                        {
                                            oRangeMin = oUserXML.CreateElement("RangeMin");
                                            oRangeMin.InnerText = oTemp.InnerText;
                                            oMetricName.AppendChild(oRangeMin);
                                        }
                                        else if ((oTemp.Name.IndexOf("Rec", 0) + 1) > 0)
                                        {

                                            iLoop = iLoop + 1;

                                            oRec = oUserXML.CreateElement("Rec" + iLoop);
                                            oMetricName.AppendChild(oRec);

                                            foreach (XmlNode oTemp1 in oTemp.ChildNodes)
                                            {

                                                if (oTemp1.Name == "Threshold")
                                                {
                                                    oThreshold = oUserXML.CreateElement("Threshold");
                                                    oThreshold.InnerText = oTemp1.InnerText;
                                                    oRec.AppendChild(oThreshold);
                                                }
                                                else
                                                {
                                                    oColor = oUserXML.CreateElement("Color");
                                                    oColor.InnerText = oTemp1.InnerText;
                                                    oRec.AppendChild(oColor);
                                                }
                                            }
                                        }
                                    }

                                }
                            NextMetric: ;
                            } //End Source Metrics

                        } //End User Groups

                        if (!bGroupFound)
                        {
                            Console.WriteLine("Add Group if not found");

                            foreach (XmlNode oTemp in oTWithinLoop.ChildNodes)
                            {

                                if (oTemp.ChildNodes.Count == 0)
                                {

                                    oCategory = oUserXML.CreateElement("MetricCategory");
                                    oCategory.InnerText = oTWithinLoop.ChildNodes[0].InnerText;
                                    oU1NodeWithinLoop.AppendChild(oCategory);
                                }
                                else
                                {

                                    foreach (XmlNode oTemp1 in oTemp.ChildNodes)
                                    {

                                        if (oTemp1.ChildNodes.Count == 0)
                                        {
                                            oMetricName = oUserXML.CreateElement("MetricName");
                                            oMetricName.InnerText = oTemp1.InnerText;
                                            oCategory.AppendChild(oMetricName);
                                        }
                                        else if ((oTemp1.Name.IndexOf("PlotType", 0) + 1) > 0)
                                        {
                                            oPlotType = oUserXML.CreateElement("PlotType");
                                            oPlotType.InnerText = oTemp1.InnerText;
                                            oMetricName.AppendChild(oPlotType);
                                        }
                                        else if ((oTemp1.Name.IndexOf("RangeMax", 0) + 1) > 0)
                                        {
                                            oRangeMax = oUserXML.CreateElement("RangeMax");
                                            oRangeMax.InnerText = oTemp1.InnerText;
                                            oMetricName.AppendChild(oRangeMax);
                                        }
                                        else if ((oTemp1.Name.IndexOf("RangeMin", 0) + 1) > 0)
                                        {
                                            oRangeMin = oUserXML.CreateElement("RangeMin");
                                            oRangeMin.InnerText = oTemp1.InnerText;
                                            oMetricName.AppendChild(oRangeMin);
                                        }
                                        else if ((oTemp1.Name.IndexOf("Rec", 0) + 1) > 0)
                                        {
                                            iLoop = iLoop + 1;

                                            oRec = oUserXML.CreateElement("Rec" + iLoop);
                                            oMetricName.AppendChild(oRec);

                                            foreach (XmlNode oTemp2 in oTemp1.ChildNodes)
                                            {

                                                if (oTemp2.Name == "Threshold")
                                                {
                                                    oThreshold = oUserXML.CreateElement("Threshold");
                                                    oThreshold.InnerText = oTemp2.InnerText;
                                                    oRec.AppendChild(oThreshold);
                                                }
                                                else
                                                {
                                                    oColor = oUserXML.CreateElement("Color");
                                                    oColor.InnerText = oTemp2.InnerText;
                                                    oRec.AppendChild(oColor);
                                                }

                                            }
                                        }
                                    }
                                }
                            }

                        }

                    } //End Source Groups

                }

            }

            oUserXML.Save(UserSettingsPath + "\\" + UserThreshold);

            oUserXML = null;

        }
0
 
LVL 23

Expert Comment

by:Tony McCreath
ID: 24362938
Your code implies theres a lot more to the XML than you previously stated.

Could you expaline the rules/structure related to elements like PlotType, RangeMax etc.

A complete  XML example would help.

I've also noticed your example xml countans mixed text and elements inside the MetricCategory element. is this correct? It seems you are ignoring it? This mixed content text/xml is normally considered bad practice.

What are the rules on merging. I understand that the Rec? elements would merge. How do you determine that other elements are the same and therefore mergable?

What are the rules when you merge. B overwrites A?

Your code is very hard to follow, has lots of looping and goto's! Some comments would help.
0
 

Author Comment

by:shanvidhya
ID: 24376501
Tiggerito,

Please find the attached sxreenshots of the Source XML, User XML and the final Merged XML with descriptions. Please let me know if you need more info.

Thanks,
Shan
MergeXML.doc
0
 
LVL 23

Expert Comment

by:Tony McCreath
ID: 24383087
So the intial #text nodes are the way elements are detected as elements to merge.

You should realy change this to an attribute. As I stated earlier, mixing text and elements is bad practice.

If you change it to an attribute called 'name' you can directly use my generic solution offered here:

http://www.experts-exchange.com/Programming/Languages/C_Sharp/Q_23999589.html

If you want to stick with the text nodes then you could re-work line 40 onwards to find mergable elements in your own way.
0
 

Author Comment

by:shanvidhya
ID: 24395483
Tiggerito,

I am not sure about your suggestion. I am a learner in the xml; so I am missing many things. I am attaching the XML file renamed as **.doc; please download and rename the extension as xml. If would be a big help if you can please change the type of each nodes in this xml as per your suggestion; I can change the program accordingly or try to use the link you suggested.  

Thanks,

GSMThreshold.doc
0
 
LVL 23

Expert Comment

by:Tony McCreath
ID: 24396466
This is how I suggest you format your XML.

Some people use "id" as the attribute in place of "name" but my previous code example is based on "name".

At this point it could be worth upping the points!
<?xml version="1.0" encoding="utf-8"?>
<GSMThreshold>
	<BasicMetrics>
		<MetricCategory name="GSM Pilot">
			<MetricName name="GSM Dominant Pilot EcIo">
				<PlotType>Range</PlotType>
				<RangeMax>0</RangeMax>
				<RangeMin>-120</RangeMin>
				<Rec name="1">
					<Threshold>-11</Threshold>
					<Color>13749760</Color>
				</Rec>
				<Rec name="2">
					<Threshold>-9</Threshold>
					<Color>5275647</Color>
				</Rec>
				<Rec name="3">
					<Threshold>-3</Threshold>
					<Color>12695295</Color>
				</Rec>
			</MetricName>
			<MetricName name="GSM Pilot Scan Ec">
				<PlotType>Range</PlotType>
				<RangeMax>0</RangeMax>
				<RangeMin>-120</RangeMin>
				<Rec name="1">
					<Threshold>-80</Threshold>
					<Color>13828244</Color>
				</Rec>
				<Rec name="2">
					<Threshold>-75</Threshold>
					<Color>3937500</Color>
				</Rec>
				<Rec name="3">
					<Threshold>-60</Threshold>
					<Color>36095</Color>
				</Rec>
			</MetricName>
		</MetricCategory>
		<MetricCategory name="GSM Power">
			<MetricName name="GSM Best Neighbor Rx Level">
				<PlotType>Range</PlotType>
				<RangeMax>0</RangeMax>
				<RangeMin>-120</RangeMin>
				<Rec name="1">
					<Threshold>-120</Threshold>
					<Color>-2147483646</Color>
				</Rec>
				<Rec name="2">
					<Threshold>-105</Threshold>
					<Color>8421376</Color>
				</Rec>
				<Rec name="3">
					<Threshold>-100</Threshold>
					<Color>16761087</Color>
				</Rec>
				<Rec name="4">
					<Threshold>-95</Threshold>
					<Color>255</Color>
				</Rec>
				<Rec name="5">
					<Threshold>-90</Threshold>
					<Color>8438015</Color>
				</Rec>
				<Rec name="6">
					<Threshold>-85</Threshold>
					<Color>65535</Color>
				</Rec>
				<Rec name="7">
					<Threshold>-80</Threshold>
					<Color>65280</Color>
				</Rec>
				<Rec name="8">
					<Threshold>-75</Threshold>
					<Color>16711680</Color>
				</Rec>
				<Rec name="9">
					<Threshold>-70</Threshold>
					<Color>16776960</Color>
				</Rec>
			</MetricName>
			<MetricName name="GSM Serving Cell Rx Level Full">
				<PlotType>Range</PlotType>
				<RangeMax>0</RangeMax>
				<RangeMin>-120</RangeMin>
				<Rec name="1">
					<Threshold>-120</Threshold>
					<Color>16711935</Color>
				</Rec>
				<Rec name="2">
					<Threshold>-105</Threshold>
					<Color>8421631</Color>
				</Rec>
				<Rec name="3">
					<Threshold>-100</Threshold>
					<Color>16761087</Color>
				</Rec>
				<Rec name="4">
					<Threshold>-95</Threshold>
					<Color>255</Color>
				</Rec>
				<Rec name="5">
					<Threshold>-90</Threshold>
					<Color>8438015</Color>
				</Rec>
				<Rec name="6">
					<Threshold>-85</Threshold>
					<Color>65535</Color>
				</Rec>
				<Rec name="7">
					<Threshold>-80</Threshold>
					<Color>65280</Color>
				</Rec>
				<Rec name="8">
					<Threshold>-75</Threshold>
					<Color>16711680</Color>
				</Rec>
				<Rec name="9">
					<Threshold>-70</Threshold>
					<Color>16776960</Color>
				</Rec>
			</MetricName>
			<MetricName name="GSM Serving Cell Rx Level Sub">
				<PlotType>Range</PlotType>
				<RangeMax>0</RangeMax>
				<RangeMin>-125</RangeMin>
				<Rec name="1">
					<Threshold>-120</Threshold>
					<Color>16711935</Color>
				</Rec>
				<Rec name="2">
					<Threshold>-105</Threshold>
					<Color>255</Color>
				</Rec>
				<Rec name="3">
					<Threshold>-100</Threshold>
					<Color>16761087</Color>
				</Rec>
				<Rec name="4">
					<Threshold>-95</Threshold>
					<Color>255</Color>
				</Rec>
				<Rec name="5">
					<Threshold>-90</Threshold>
					<Color>8438015</Color>
				</Rec>
				<Rec name="6">
					<Threshold>-85</Threshold>
					<Color>65535</Color>
				</Rec>
				<Rec name="7">
					<Threshold>-80</Threshold>
					<Color>65280</Color>
				</Rec>
				<Rec name="8">
					<Threshold>-75</Threshold>
					<Color>16711680</Color>
				</Rec>
				<Rec name="9">
					<Threshold>-70</Threshold>
					<Color>16776960</Color>
				</Rec>
			</MetricName>
		</MetricCategory>
	</BasicMetrics>
</GSMThreshold>

Open in new window

0
 

Author Comment

by:shanvidhya
ID: 24397090
Tiggerito,

I tried your code with the following function and xml you provided; but it didnt generate the merge xml properly.

Test1 is the modified xml you provided
Test2 is the test1 xml with few nodes deleted

the resultant xml test3 was same as test2.xml


private void btnProcess_Click(object sender, EventArgs e)
        {

            XmlDocument oSourceXML = new XmlDocument();
            XmlDocument oUserXML = new XmlDocument();

            oSourceXML.Load("C:\\temp1.xml");
            oUserXML.Load("C:\\temp2.xml");

            Merge(oSourceXML, oUserXML);

            oUserXML.Save("C:\\temp3.xml");
            oUserXML = null;
            oSourceXML= null;

        }

once this problem is fixed I will double the points.
0
 
LVL 23

Accepted Solution

by:
Tony McCreath earned 250 total points
ID: 24397314
The code merges into oSourceXML and your saving oUserXML

setting local variables to null like that is not required in .Net
0
 

Author Closing Comment

by:shanvidhya
ID: 31578128
Thanks Tiggerito for your help. I will make changes to my approach in handling XML as per your suggestion.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Wouldn’t it be nice if you could test whether an element is contained in an array by using a Contains method just like the one available on List objects? Wouldn’t it be good if you could write code like this? (CODE) In .NET 3.5, this is possible…
Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …
This video shows how to use Hyena, from SystemTools Software, to bulk import 100 user accounts from an external text file. View in 1080p for best video quality.

808 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question