asked on

SAX Parsing of flat file

Hi Experts,

I have a .txt file as attached below. What i am trying to do here is to convert this CSV file into a deep structure XML. The example .txt file (Input.txt) which is attached has 4 records (Header (H), ITEM (I), ItemDetail(ID), Address (A) ), these 4 records repeat in the same sequence and there can be repeating Recordsets (H,I,ID,A) and a Trailer at the end.

The object here is to convert the .txt to a deep structure and i am doing this using 2 different steps, the first stage is to convert the CSV to an XML using SAP PI integration tool with its feature of File content Conversion (FCC), Now i need to apply the java mapping (also attached below) to convert this XML to a deep structure.

My issue here is i haven't written this code and got it from SAP forums, i would like to test this code to create the Deep structure (also attached is the deep structure snapshot). All i would need is to have a main method to debug this code and i am having trouble writing the main method to execute this class.

Note: I am not able to attach the library files that need to execute this code as the attachments don't allow all formats. If any one can suggest any way to upload them i can do it or i can send them to their email.

I am attaching the following,
-Text file (Input.txt)
-XML file after SAP PI FCC (MainDocument.xml)
-Code for the class to convert the MainDocument.xml to a Deep structure (SAX_File Parsing.doc)
-Deep structure snapshot (OutXML.jpg)
Input.txt
MainDocument.xml
DeepStructureSAXMapping.docx
DeepStructureSAXMapping.java
OutXML.jpg

kalyangkm

ASKER

Hi

Can anyone please suggest some solution for my question.

mccarl

Firstly, I will adress the issue at hand...

All i would need is to have a main method to debug this code

In order to make this easier to test, I would modify your "transform" method slightly to separate the actual SAX transformation from the reliance on the SAP classes. See the below code...

package com.sempra.sap.nv;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;

import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;

import com.sap.aii.mapping.api.AbstractTransformation;
import com.sap.aii.mapping.api.StreamTransformationException;
import com.sap.aii.mapping.api.TransformationInput;
import com.sap.aii.mapping.api.TransformationOutput;


public class DeepStructureSAXMapping extends AbstractTransformation {
	
    // main method for testing
    public static void main(String[] args) {
	    doTransform(new FileInputStream("input.xml"), new FileOutputStream("output.xml");
    }
	
	
    public void transform(TransformationInput in, TransformationOutput out)
    throws StreamTransformationException{
        getTrace().addInfo("JAVA Mapping Called");
        InputStream instream = in.getInputPayload().getInputStream();
        getTrace().addInfo("Input XML Payload is successfuly Read");
        OutputStream outstream = out.getOutputPayload().getOutputStream();
        
        doTransform(instream, outstream);
    }
	
    public void doTransform(InputStream instream, OutputStream outstream) {
        Handling handler = new Handling();
 
        SAXParserFactory factory = SAXParserFactory.newInstance();
        String result = null;
 
        try {
            SAXParser saxpr = factory.newSAXParser();
            saxpr.parse(instream, handler);
        } catch (Exception e) {
            getTrace().addWarning("$$$ Exception occured $$$" + e.getMessage());
            e.printStackTrace();
            getTrace().addInfo(e.getMessage(), e);
        }
 
        result = handler.getResult(); //Getting the final Output Structure
        try {
            outstream.write(result.getBytes());
           
            getTrace().addInfo("Output XML Payload is successfuly Written");
        } catch (IOException e) {
            getTrace().addWarning(
                    "$$$ Exception occured while writing the o/p payload $$$");
            e.printStackTrace();
            getTrace().addInfo(e.getMessage(), e);
        }
		
    }
	
	  /* Second class Which extends DefaultHandler */
	 
    class Handling extends DefaultHandler {
 
        StringBuffer fresult = new StringBuffer();
 
        private String ITEM_Node = "closed";
        private String ItemDetail_Node = "closed";
 
        public void startDocument() throws SAXException {
            fresult.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
            fresult.append("<ns0:ReceiverFile_MT xmlns:ns0=\"Test\">");
        }
 
        public void endDocument() throws SAXException {
 
            fresult.append("</ns0:ReceiverFile_MT>");
        }
 
        public void startElement(String namespaceURI, String localName,
                String qName, Attributes atts) throws SAXException {
            String lName = localName;
            if (lName.equals(""))
                lName = qName;
            if (lName.equals("HEADER")) {
                fresult.append("<HEADER>");
            }
 
            if (lName.equals("CREATION_DATE")) {
                fresult.append("<CREATION_DATE>");
            }
 
            if (lName.equals("CREATION_TIME")) {
                fresult.append("<CREATION_TIME>");
            }
 
            if (lName.equals("ITEM")) {
                if (ItemDetail_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ItemDetail>");
                    ItemDetail_Node = "closed";
                }
                if (ITEM_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ITEM>");
                    ITEM_Node = "closed";
                }
                fresult.append("<ITEM>");
                ITEM_Node = "open";
            }
            if (lName.equals("OrderNumber")) {
                fresult.append("<OrderNumber>");
            }
 
            if (lName.equals("ItemDetail")) {
                if (ItemDetail_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ItemDetail>");
                    ItemDetail_Node = "closed";
                }
                fresult.append("<ItemDetail>");
                ItemDetail_Node = "open";
            }
 
            if (lName.equals("OrderQualtity")) {
                fresult.append("<OrderQualtity>");
            }
 
            if (lName.equals("TotalCost")) {
                fresult.append("<TotalCost>");
            }
 
            if (lName.equals("Address")) {
                fresult.append("<Address>");
            }
 
            if (lName.equals("HouseNumber")) {
                fresult.append("<HouseNumber>");
            }
 
            if (lName.equals("Street")) {
                fresult.append("<Street>");
            }
 
            if (lName.equals("PostCode")) {
                fresult.append("<PostCode>");
            }
 
            if (lName.equals("Trailer")) {
 
                if (ItemDetail_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ItemDetail>");
                    ItemDetail_Node = "closed";
                }
                if (ITEM_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ITEM>");
                    ITEM_Node = "closed";
                }
                fresult.append("<Trailer>");
            }
 
            if (lName.equals("NoofRecords")) {
                fresult.append("<NoofRecords>");
            } // Says No of ITEM Records
 
        }
 
        public void endElement(String namespaceURI, String localName,
                String qName) throws SAXException {
            String lName = localName;
            if (lName.equals(""))
                lName = qName;
 
            if (lName.equals("HEADER")) {
                fresult.append("</HEADER>");
            }
 
            if (lName.equals("CREATION_DATE")) {
                fresult.append("</CREATION_DATE>");
            }
 
            if (lName.equals("CREATION_TIME")) {
                fresult.append("</CREATION_TIME>");
            }
 
            if (lName.equals("OrderNumber")) {
                fresult.append("</OrderNumber>");
            }
            if (lName.equals("OrderQualtity")) {
                fresult.append("</OrderQualtity>");
            }
 
            if (lName.equals("TotalCost")) {
                fresult.append("</TotalCost>");
            }
 
            if (lName.equals("Address")) {
                fresult.append("</Address>");
            }
 
            if (lName.equals("HouseNumber")) {
                fresult.append("</HouseNumber>");
            }
 
            if (lName.equals("Street")) {
                fresult.append("</Street>");
            }
 
            if (lName.equals("PostCode")) {
                fresult.append("</PostCode>");
            }
 
            if (lName.equals("Trailer")) {
                fresult.append("</Trailer>");
            }
 
            if (lName.equals("NoofRecords")) {
                fresult.append("</NoofRecords>");
            }
        }
 
        public void characters(char[] ch, int start, int length)
                throws SAXException {
            String val = new String(ch, start, length);
            fresult.append(val);
        }
 
        public String getResult() {
            return fresult.toString();
        }
    }
   
}

Open in new window

However, having said that, have you considered using something like XSLT to convert between the two different XML structures? That is pretty much what XSLT is best at.

kalyangkm

ASKER

Hi McCarl,

Thanks for the response. First let me look into the java mapping. I have used your transform method and correspondingly made changes as necessary but its complaining in the following line withe the error "Cannot Make Static Reference to Non-Static Method"

static void main(String[] args) {
doTransform(new FileInputStream("/Users/Kalyan/Desktop/File Parsing/Deep Structure/MainDocument.xml"), new FileOutputStream("/Users/Kalyan/Desktop/File Parsing/Deep Structure/MainDocument1.xml"));
}

Here is the updated Code

package com.sempra.sap.nv;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;

import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;

import com.sap.aii.mapping.api.AbstractTransformation;
import com.sap.aii.mapping.api.StreamTransformationException;
import com.sap.aii.mapping.api.TransformationInput;
import com.sap.aii.mapping.api.TransformationOutput;


public class DeepStructureSAXMapping extends AbstractTransformation {
	
	// main method for testing
    public static void main(String[] args) {
	    doTransform(new FileInputStream("/Users/Kalyan/Desktop/File Parsing/Deep Structure/MainDocument.xml"), new FileOutputStream("/Users/Kalyan/Desktop/File Parsing/Deep Structure/MainDocument1.xml"));
    }
	
	public void transform(TransformationInput in, TransformationOutput out)
	throws StreamTransformationException{
        getTrace().addInfo("JAVA Mapping Called");
        InputStream instream = in.getInputPayload().getInputStream();
        getTrace().addInfo("Input XML Payload is successfuly Read");
        OutputStream outstream = out.getOutputPayload().getOutputStream();
        
        doTransform(instream, outstream);
    }   
	
	public void doTransform(InputStream instream, OutputStream outstream) {
        Handling handler = new Handling();
 
        SAXParserFactory factory = SAXParserFactory.newInstance();
        String result = null;
 
        try {
            SAXParser saxpr = factory.newSAXParser();
            saxpr.parse(instream, handler);
        } catch (Exception e) {
            getTrace().addWarning("$$$ Exception occured $$$" + e.getMessage());
            e.printStackTrace();
            getTrace().addInfo(e.getMessage(), e);
        }
 
        result = handler.getResult(); //Getting the final Output Structure
        try {
            //out.getOutputPayload().getOutputStream().write(result.getBytes());
        	outstream.write(result.getBytes());
            getTrace().addInfo("Output XML Payload is successfuly Written");
        } catch (IOException e) {
            getTrace().addWarning(
                    "$$$ Exception occured while writing the o/p payload $$$");
            e.printStackTrace();
            getTrace().addInfo(e.getMessage(), e);
        }
		
	}
	
	  /* Second class Which extends DefaultHandler */
	 
    class Handling extends DefaultHandler {
 
        StringBuffer fresult = new StringBuffer();
 
        private String ITEM_Node = "closed";
        private String ItemDetail_Node = "closed";
 
        public void startDocument() throws SAXException {
            fresult.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
            fresult.append("<ns0:ReceiverFile_MT xmlns:ns0=\"Test\">");
        }
 
        public void endDocument() throws SAXException {
 
            fresult.append("</ns0:ReceiverFile_MT>");
        }
 
        public void startElement(String namespaceURI, String localName,
                String qName, Attributes atts) throws SAXException {
            String lName = localName;
            if (lName.equals(""))
                lName = qName;
            if (lName.equals("HEADER")) {
                fresult.append("<HEADER>");
            }
 
            if (lName.equals("CREATION_DATE")) {
                fresult.append("<CREATION_DATE>");
            }
 
            if (lName.equals("CREATION_TIME")) {
                fresult.append("<CREATION_TIME>");
            }
 
            if (lName.equals("ITEM")) {
                if (ItemDetail_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ItemDetail>");
                    ItemDetail_Node = "closed";
                }
                if (ITEM_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ITEM>");
                    ITEM_Node = "closed";
                }
                fresult.append("<ITEM>");
                ITEM_Node = "open";
            }
            if (lName.equals("OrderNumber")) {
                fresult.append("<OrderNumber>");
            }
 
            if (lName.equals("ItemDetail")) {
                if (ItemDetail_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ItemDetail>");
                    ItemDetail_Node = "closed";
                }
                fresult.append("<ItemDetail>");
                ItemDetail_Node = "open";
            }
 
            if (lName.equals("OrderQualtity")) {
                fresult.append("<OrderQualtity>");
            }
 
            if (lName.equals("TotalCost")) {
                fresult.append("<TotalCost>");
            }
 
            if (lName.equals("Address")) {
                fresult.append("<Address>");
            }
 
            if (lName.equals("HouseNumber")) {
                fresult.append("<HouseNumber>");
            }
 
            if (lName.equals("Street")) {
                fresult.append("<Street>");
            }
 
            if (lName.equals("PostCode")) {
                fresult.append("<PostCode>");
            }
 
            if (lName.equals("Trailer")) {
 
                if (ItemDetail_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ItemDetail>");
                    ItemDetail_Node = "closed";
                }
                if (ITEM_Node.equalsIgnoreCase("open")) {
                    fresult.append("</ITEM>");
                    ITEM_Node = "closed";
                }
                fresult.append("<Trailer>");
            }
 
            if (lName.equals("NoofRecords")) {
                fresult.append("<NoofRecords>");
            } // Says No of ITEM Records
 
        }
 
        public void endElement(String namespaceURI, String localName,
                String qName) throws SAXException {
            String lName = localName;
            if (lName.equals(""))
                lName = qName;
 
            if (lName.equals("HEADER")) {
                fresult.append("</HEADER>");
            }
 
            if (lName.equals("CREATION_DATE")) {
                fresult.append("</CREATION_DATE>");
            }
 
            if (lName.equals("CREATION_TIME")) {
                fresult.append("</CREATION_TIME>");
            }
 
            if (lName.equals("OrderNumber")) {
                fresult.append("</OrderNumber>");
            }
            if (lName.equals("OrderQualtity")) {
                fresult.append("</OrderQualtity>");
            }
 
            if (lName.equals("TotalCost")) {
                fresult.append("</TotalCost>");
            }
 
            if (lName.equals("Address")) {
                fresult.append("</Address>");
            }
 
            if (lName.equals("HouseNumber")) {
                fresult.append("</HouseNumber>");
            }
 
            if (lName.equals("Street")) {
                fresult.append("</Street>");
            }
 
            if (lName.equals("PostCode")) {
                fresult.append("</PostCode>");
            }
 
            if (lName.equals("Trailer")) {
                fresult.append("</Trailer>");
            }
 
            if (lName.equals("NoofRecords")) {
                fresult.append("</NoofRecords>");
            }
        }
 
        public void characters(char[] ch, int start, int length)
                throws SAXException {
            String val = new String(ch, start, length);
            fresult.append(val);
        }
 
        public String getResult() {
            return fresult.toString();
        }
    }
   
}

Open in new window

ASKER CERTIFIED SOLUTION

mccarl

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

kalyangkm

ASKER

Thanks McCarrl, It worked. If i try XSLT i will open another question.

mccarl

No worries, glad to help! :)

kalyangkm

ASKER

Just one question, I was using XML spy and the trial version got expired. Is there any other XML tool which is not as expensive as xml spy to purchase. Any suggestion would be appreciated.

mccarl

No sorry, don't know of much. I too haven't been impressed by anything open source in this area so I've just made do.

I'd suggest asking this as a new question to get more opinions from others.