[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Adding a wrapper element based on size of children elements

Posted on 2009-12-21
12
Medium Priority
?
444 Views
Last Modified: 2013-11-18
Hi,

I have an xsl transform that adds a wrapper element around every n elements in a list. The problem I am now facing is that if the children of the list elements are too large the wrapper becomes too large. Ideally I would like to add a wrapper element around an arbitrary number of elements in the list, but where the total size of the wrapper does not exceed a certain number of megabytes (about 1-3 (parameter driven)). Is there anyway I can do this? I have included my current xslt map so for your perusal.

Take care,

Ivar

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:lsx="xalan://com.lodestarcorp.core.xml.xslt" xmlns:xalan="http://xml.apache.org/xalan" exclude-result-prefixes="lsx xalan">

	<xsl:output method="xml" version="1.0" encoding="ISO-8859-1" indent="yes"/>
	<xsl:param name="LSMapUtils"/>
	<xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>

	<xsl:template match="REQUEST">


		<!-- Go to the database to retrieve the number of meter reads maximum per file for this Service -->

	<!--
		<xsl:variable name="Query1" select="lsx:compiledQuery($LSMapUtils,'select ss.numval from ##Q.MTXSERVSETTINGS SS, ##Q.MTXSERVCODES SC, ##Q.MTXSETTINGTYPES ST where SS.UIDMTXSERVCODES = SC.UIDMTXSERVCODES and SS.UIDMTXSETTINGTYPES = ST.UIDMTXSETTINGTYPES and SC.SERVICECODE =? and ST.SETTINGTYPE =?')"/>
		<xsl:value-of select="lsx:bindString($Query1, ./@SERVICE)"/>
		<xsl:value-of select="lsx:bindString($Query1,'MTX_BMR_MAX_METERS_PER_RESPONSE_FILE' )"/>
		<xsl:variable name="maxAccounts" select="ceiling(lsx:execute($Query1, true()))"/>
	Get the REQUEST node attributes which will be applied to the REQUEST nodes-->
	<xsl:variable name="maxAccounts" select="500"/>
		<xsl:variable name="this-attributes" select="@*[name() != 'REQFILE']"/>
		<xsl:variable name="requestFile" select="@REQFILE"/>
		<xsl:variable name="totalSplitCount" select="ceiling(count(ACCOUNT) div $maxAccounts)"/>
		<xsl:element name="ROOT">
		<xsl:choose>
			<xsl:when test="$maxAccounts = 1">
						<xsl:for-each select="ACCOUNT">
				<xsl:element name="REQUEST">
					<xsl:copy-of select="$this-attributes"/>
					<xsl:attribute name="SPLITINDEX"><xsl:value-of select="position()"/></xsl:attribute>
					<xsl:attribute name="TOTALSPLITCOUNT"><xsl:value-of select="$totalSplitCount"/></xsl:attribute>
					<xsl:copy-of select="."/>
				</xsl:element>
			</xsl:for-each>
			</xsl:when>
			<xsl:otherwise>
						<xsl:for-each select="ACCOUNT[position() mod $maxAccounts = 1]">
				<xsl:element name="REQUEST">
					<xsl:copy-of select="$this-attributes"/>
					<xsl:attribute name="REQFILE"><xsl:value-of select="$requestFile"/>_<xsl:value-of select="position()"/></xsl:attribute>
					<xsl:attribute name="SPLITINDEX"><xsl:value-of select="position()"/></xsl:attribute>
					<xsl:attribute name="TOTALSPLITCOUNT"><xsl:value-of select="$totalSplitCount"/></xsl:attribute>
					<xsl:copy-of select="self::ACCOUNT | following-sibling::ACCOUNT[position() &lt; $maxAccounts]"/>
				</xsl:element>
			</xsl:for-each>
			</xsl:otherwise>
		</xsl:choose>
		
	

		</xsl:element>
			
			
			
	</xsl:template>
</xsl:stylesheet>

Open in new window

0
Comment
Question by:isvensen
  • 8
  • 3
11 Comments
 
LVL 60

Expert Comment

by:Geert Bormans
ID: 26107424
Hi Ivar,

I am looking into this, but it is not very clear to me
It seems you allready have what you need ( a parameter driven wrapper)
Or do you want to make this adjustable to the size?
If the later is what you want... can you give an indication then what the trigger would be...
an example XML would also be welcome

cheers

Geert
0
 

Author Comment

by:isvensen
ID: 26108719
Hi Gert,

Ok, what I want is a trigger that inserts a wrapper element (REQUEST) once the size of the elements to be wrapped reaches a limit). The size is obviously dependent on the number of accounts, the number of premises and the number of icps, and the number of meters etc. So it's not easy to predict this. This is where the conundrum is. If I was doing this in a standard procedural way, I would dome something like
while not eof{
Create wrapper start
while (tot_size < max_size)
{
Add child_elements;
tot_size += child_elements.size;
}
}
or something akin to that.

I've attached an XML, this isn't very big, but the size parameter will be parametrised and read from the db, so  if you are doing this you can just set it low enough to see the split.


Sample.xml
0
 
LVL 60

Expert Comment

by:Geert Bormans
ID: 26108862
If I hear database...
how much of the database export is controlled by you?
You could possibly have a marker introduced (as an attribute one way or another)
to hard code the optimal split in the XML allready
Does that sound feasible?

I will look into your XML and think about it tomorrow morning, bedtime now, cheers
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 

Author Comment

by:isvensen
ID: 26108924
Unfortunately, nothing in what we receive is actually controlled by us, so we need to calculate this. There may be some redesign options, but the client does not want to pay for that. If we can't find an xslt solution we will probably stick with the number of accounts as previous, and then they have to ensure the data fits, and catch any exceptions.
0
 
LVL 60

Expert Comment

by:Geert Bormans
ID: 26121740
Hey Ivar, I fail to see the trigger here.
If I understand well... you have a bunch of ACCOUNT elements
and you wrap  them in groups of 500
 <xsl:variable name="maxAccounts" select="500"/>
now you want them to be wrapped in smaller chunks if the byte content of a chunk is too big?
You made everything in an account "string", I assume most of the fields have predictable sizes
likely the READING attribute REGISTER is too big

Do you want me to add up the byte content (of which fields then) to make the cut?
0
 
LVL 60

Expert Comment

by:Geert Bormans
ID: 26121938
well, thinking about it, if you really need to groc up byte content and count it, I believe you should make this a two-step.
You could do this as follows (I always have two options :-)
option 1: have two stylesheets in a row. The first one two add a marker on where to split, the second to actually split
option 2: have the two steps in one stylesheet, this can be done if you have an extension function which provides a nodeset . That is highly processor dependant... am I correct that you are using Xalan? I will show you how to do that in Xalan then. I need to think about the memory problem then, can you give a rough estimate of what the maximum da
0
 
LVL 60

Expert Comment

by:Geert Bormans
ID: 26121942
... pushed the button too soon, sorry

well, thinking about it, if you really need to groc up byte content and count it, I believe you should make this a two-step.
You could do this as follows (I always have two options :-)
option 1: have two stylesheets in a row. The first one two add a marker on where to split, the second to actually split
option 2: have the two steps in one stylesheet, this can be done if you have an extension function which provides a nodeset . That is highly processor dependant... am I correct that you are using Xalan? I will show you how to do that in Xalan then. I need to think about the memory problem then, can you give a rough estimate of what the maximum dataset will be in Mbytes? Maybe that will be too big to simply have an internal step in between in memory

Let me know what you want to do then
0
 

Author Comment

by:isvensen
ID: 26135266
The maximum dataset in Mb is probably around the 50 Mb mark. This is why we split the file into smaller chunks, as our software can only handle <5Mb files, which is again why we try to optimise so the chunks are less than 5Mb (preferably less than 2 Mb for performance). So yes you are right in that what you need is to add up the byte content and then split based on that. Using two style sheets would be fine.

Take care and Merry Christmas,

Ivar
0
 
LVL 60

Expert Comment

by:Geert Bormans
ID: 26136885
Merry Christmass for you too.

Here is a stylesheet that counts all characters in the attribute and element names and content
this is a rough estimate (well, not so rough) for the byte content of an ACCOUNT element
at the top you give the maximum size, and this stylesheet will put an attribute "
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:strip-space elements="*"/>
    <xsl:output indent="yes"/>
    <xsl:param name="max-size">1000</xsl:param>
    <xsl:template match="REQUEST">
        <xsl:copy>
            <xsl:copy-of select="@*"/>
            <xsl:apply-templates select="ACCOUNT[1]">
                <xsl:with-param name="cur-size" select="0"/>
            </xsl:apply-templates>
        </xsl:copy>
    </xsl:template>
    <xsl:template match="ACCOUNT">
        <xsl:param name="cur-size"/> <!-- passed in from previous node -->
        <!-- a concatenation of all account element names -->
        <xsl:variable name="elem-names">
            <xsl:for-each select="self::* | .//*">
                <xsl:text>[</xsl:text>
                <xsl:value-of select="name()"/>
                <xsl:text>]</xsl:text>
                <xsl:text>[/</xsl:text>
                <xsl:value-of select="name()"/>
                <xsl:text>]</xsl:text>
            </xsl:for-each>
        </xsl:variable>
        <!-- a concatenation of all account element content -->
        <xsl:variable name="elem-content"> 
            <xsl:for-each select=".//text()">
                <xsl:value-of select="."/>
            </xsl:for-each>
        </xsl:variable>
        <!-- a concatenation of all account attributes names -->
        <xsl:variable name="attr-names">
            <xsl:for-each select="self::*/@* | .//*/@*">
                <xsl:text>-</xsl:text>
                <xsl:value-of select="name()"/>
            </xsl:for-each>
        </xsl:variable>
        <!-- a concatenation of all account attribute values -->
        <xsl:variable name="attr-content">
            <xsl:for-each select="self::*/@* | .//*/@*">
                <xsl:text>{</xsl:text>
                <xsl:value-of select="."/>
                <xsl:text>}</xsl:text>
            </xsl:for-each>
        </xsl:variable>
        <!-- the sum of all string-lengths in an account !! estimate only -->
        <xsl:variable name="this-account-size" select="string-length($elem-names) + string-length($elem-content) + string-length($attr-names) + string-length($attr-content)"></xsl:variable>
        <!-- new-sum to pass to next node -->
        <xsl:variable name="new-size">
            <xsl:choose>
                <xsl:when test="$cur-size + $this-account-size &gt; $max-size">
                    <xsl:value-of select="0"/>
                </xsl:when>
                <xsl:otherwise>
                    <xsl:value-of select="$cur-size + $this-account-size"/>
                </xsl:otherwise>
            </xsl:choose>
        </xsl:variable>
        <xsl:copy>
            <xsl:if test="$cur-size + $this-account-size &gt; $max-size or not(preceding-sibling::ACCOUNT)">
                <xsl:attribute name="break-here">
                    <xsl:text>true</xsl:text>
                </xsl:attribute>
            </xsl:if>
            <xsl:copy-of select="@*"/>
            <xsl:copy-of select="node()"/>
        </xsl:copy>
        <xsl:apply-templates select="following-sibling::ACCOUNT[1]">
            <xsl:with-param name="cur-size" select="$new-size"/>
        </xsl:apply-templates>
    </xsl:template>

</xsl:stylesheet>

Open in new window

0
 
LVL 60

Expert Comment

by:Geert Bormans
ID: 26136916
Here is your second stylesheet
it wraps on ACCOUNT having the break-here attribute
this could be optimised, since I do too much walking up and down the preceding and following axes,
but first check wheither this behaves well in your architecture
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:strip-space elements="*"/>
    <xsl:output indent="yes"/>
    <xsl:template match="REQUEST">
        <xsl:copy>
            <xsl:copy-of select="@*"/>
            <xsl:apply-templates select="ACCOUNT[@break-here]">
            </xsl:apply-templates>
        </xsl:copy>
    </xsl:template>
    <xsl:template match="ACCOUNT[@break-here]">
        <xsl:variable name="this-id" select="generate-id()"/>
        <wrapper>
            <xsl:copy-of select="self::* | following-sibling::ACCOUNT[not(@break-here)][generate-id(preceding-sibling::ACCOUNT[@break-here][1]) = $this-id]" />
        </wrapper>
    </xsl:template>

</xsl:stylesheet>

Open in new window

0
 
LVL 60

Accepted Solution

by:
Geert Bormans earned 2000 total points
ID: 26136945
please forget my previous stylesheet, that is highly inefficient,
this one beats it in elegance and speed
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:strip-space elements="*"/>
    <xsl:output indent="yes"/>
    <xsl:template match="REQUEST">
        <xsl:copy>
            <xsl:copy-of select="@*"/>
            <xsl:apply-templates select="ACCOUNT[@break-here][1]">
            </xsl:apply-templates>
        </xsl:copy>
    </xsl:template>
    <xsl:template match="ACCOUNT[@break-here]">
        <wrapper>
            <xsl:copy-of select="." />
            <xsl:apply-templates select="following-sibling::ACCOUNT[1][not(@break-here)]" />
        </wrapper>
        <xsl:apply-templates select="following-sibling::ACCOUNT[@break-here][1]"></xsl:apply-templates>
    </xsl:template>
    <xsl:template match="ACCOUNT[not(@break-here)]">
        <xsl:copy-of select="." />
        <xsl:apply-templates select="following-sibling::ACCOUNT[1][not(@break-here)]" />
    </xsl:template>
</xsl:stylesheet>

Open in new window

0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The Confluence of Individual Knowledge and the Collective Intelligence At this writing (summer 2013) the term API (http://dictionary.reference.com/browse/API?s=t) has made its way into the popular lexicon of the English language.  A few years ago, …
Have you tried to learn about Unicode, UTF-8, and multibyte text encoding and all the articles are just too "academic" or too technical? This article aims to make the whole topic easy for just about anyone to understand.
The viewer will receive an overview of the basics of CSS showing inline styles. In the head tags set up your style tags: (CODE) Reference the nav tag and set your properties.: (CODE) Set the reference for the UL element and styles for it to ensu…
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).
Suggested Courses

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question