• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 449
  • Last Modified:

Adding a wrapper element based on size of children elements

Hi,

I have an xsl transform that adds a wrapper element around every n elements in a list. The problem I am now facing is that if the children of the list elements are too large the wrapper becomes too large. Ideally I would like to add a wrapper element around an arbitrary number of elements in the list, but where the total size of the wrapper does not exceed a certain number of megabytes (about 1-3 (parameter driven)). Is there anyway I can do this? I have included my current xslt map so for your perusal.

Take care,

Ivar

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:lsx="xalan://com.lodestarcorp.core.xml.xslt" xmlns:xalan="http://xml.apache.org/xalan" exclude-result-prefixes="lsx xalan">

	<xsl:output method="xml" version="1.0" encoding="ISO-8859-1" indent="yes"/>
	<xsl:param name="LSMapUtils"/>
	<xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>

	<xsl:template match="REQUEST">


		<!-- Go to the database to retrieve the number of meter reads maximum per file for this Service -->

	<!--
		<xsl:variable name="Query1" select="lsx:compiledQuery($LSMapUtils,'select ss.numval from ##Q.MTXSERVSETTINGS SS, ##Q.MTXSERVCODES SC, ##Q.MTXSETTINGTYPES ST where SS.UIDMTXSERVCODES = SC.UIDMTXSERVCODES and SS.UIDMTXSETTINGTYPES = ST.UIDMTXSETTINGTYPES and SC.SERVICECODE =? and ST.SETTINGTYPE =?')"/>
		<xsl:value-of select="lsx:bindString($Query1, ./@SERVICE)"/>
		<xsl:value-of select="lsx:bindString($Query1,'MTX_BMR_MAX_METERS_PER_RESPONSE_FILE' )"/>
		<xsl:variable name="maxAccounts" select="ceiling(lsx:execute($Query1, true()))"/>
	Get the REQUEST node attributes which will be applied to the REQUEST nodes-->
	<xsl:variable name="maxAccounts" select="500"/>
		<xsl:variable name="this-attributes" select="@*[name() != 'REQFILE']"/>
		<xsl:variable name="requestFile" select="@REQFILE"/>
		<xsl:variable name="totalSplitCount" select="ceiling(count(ACCOUNT) div $maxAccounts)"/>
		<xsl:element name="ROOT">
		<xsl:choose>
			<xsl:when test="$maxAccounts = 1">
						<xsl:for-each select="ACCOUNT">
				<xsl:element name="REQUEST">
					<xsl:copy-of select="$this-attributes"/>
					<xsl:attribute name="SPLITINDEX"><xsl:value-of select="position()"/></xsl:attribute>
					<xsl:attribute name="TOTALSPLITCOUNT"><xsl:value-of select="$totalSplitCount"/></xsl:attribute>
					<xsl:copy-of select="."/>
				</xsl:element>
			</xsl:for-each>
			</xsl:when>
			<xsl:otherwise>
						<xsl:for-each select="ACCOUNT[position() mod $maxAccounts = 1]">
				<xsl:element name="REQUEST">
					<xsl:copy-of select="$this-attributes"/>
					<xsl:attribute name="REQFILE"><xsl:value-of select="$requestFile"/>_<xsl:value-of select="position()"/></xsl:attribute>
					<xsl:attribute name="SPLITINDEX"><xsl:value-of select="position()"/></xsl:attribute>
					<xsl:attribute name="TOTALSPLITCOUNT"><xsl:value-of select="$totalSplitCount"/></xsl:attribute>
					<xsl:copy-of select="self::ACCOUNT | following-sibling::ACCOUNT[position() &lt; $maxAccounts]"/>
				</xsl:element>
			</xsl:for-each>
			</xsl:otherwise>
		</xsl:choose>
		
	

		</xsl:element>
			
			
			
	</xsl:template>
</xsl:stylesheet>

Open in new window

0
isvensen
Asked:
isvensen
  • 8
  • 3
1 Solution
 
Geert BormansInformation ArchitectCommented:
Hi Ivar,

I am looking into this, but it is not very clear to me
It seems you allready have what you need ( a parameter driven wrapper)
Or do you want to make this adjustable to the size?
If the later is what you want... can you give an indication then what the trigger would be...
an example XML would also be welcome

cheers

Geert
0
 
isvensenAuthor Commented:
Hi Gert,

Ok, what I want is a trigger that inserts a wrapper element (REQUEST) once the size of the elements to be wrapped reaches a limit). The size is obviously dependent on the number of accounts, the number of premises and the number of icps, and the number of meters etc. So it's not easy to predict this. This is where the conundrum is. If I was doing this in a standard procedural way, I would dome something like
while not eof{
Create wrapper start
while (tot_size < max_size)
{
Add child_elements;
tot_size += child_elements.size;
}
}
or something akin to that.

I've attached an XML, this isn't very big, but the size parameter will be parametrised and read from the db, so  if you are doing this you can just set it low enough to see the split.


Sample.xml
0
 
Geert BormansInformation ArchitectCommented:
If I hear database...
how much of the database export is controlled by you?
You could possibly have a marker introduced (as an attribute one way or another)
to hard code the optimal split in the XML allready
Does that sound feasible?

I will look into your XML and think about it tomorrow morning, bedtime now, cheers
0
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
isvensenAuthor Commented:
Unfortunately, nothing in what we receive is actually controlled by us, so we need to calculate this. There may be some redesign options, but the client does not want to pay for that. If we can't find an xslt solution we will probably stick with the number of accounts as previous, and then they have to ensure the data fits, and catch any exceptions.
0
 
Geert BormansInformation ArchitectCommented:
Hey Ivar, I fail to see the trigger here.
If I understand well... you have a bunch of ACCOUNT elements
and you wrap  them in groups of 500
 <xsl:variable name="maxAccounts" select="500"/>
now you want them to be wrapped in smaller chunks if the byte content of a chunk is too big?
You made everything in an account "string", I assume most of the fields have predictable sizes
likely the READING attribute REGISTER is too big

Do you want me to add up the byte content (of which fields then) to make the cut?
0
 
Geert BormansInformation ArchitectCommented:
well, thinking about it, if you really need to groc up byte content and count it, I believe you should make this a two-step.
You could do this as follows (I always have two options :-)
option 1: have two stylesheets in a row. The first one two add a marker on where to split, the second to actually split
option 2: have the two steps in one stylesheet, this can be done if you have an extension function which provides a nodeset . That is highly processor dependant... am I correct that you are using Xalan? I will show you how to do that in Xalan then. I need to think about the memory problem then, can you give a rough estimate of what the maximum da
0
 
Geert BormansInformation ArchitectCommented:
... pushed the button too soon, sorry

well, thinking about it, if you really need to groc up byte content and count it, I believe you should make this a two-step.
You could do this as follows (I always have two options :-)
option 1: have two stylesheets in a row. The first one two add a marker on where to split, the second to actually split
option 2: have the two steps in one stylesheet, this can be done if you have an extension function which provides a nodeset . That is highly processor dependant... am I correct that you are using Xalan? I will show you how to do that in Xalan then. I need to think about the memory problem then, can you give a rough estimate of what the maximum dataset will be in Mbytes? Maybe that will be too big to simply have an internal step in between in memory

Let me know what you want to do then
0
 
isvensenAuthor Commented:
The maximum dataset in Mb is probably around the 50 Mb mark. This is why we split the file into smaller chunks, as our software can only handle <5Mb files, which is again why we try to optimise so the chunks are less than 5Mb (preferably less than 2 Mb for performance). So yes you are right in that what you need is to add up the byte content and then split based on that. Using two style sheets would be fine.

Take care and Merry Christmas,

Ivar
0
 
Geert BormansInformation ArchitectCommented:
Merry Christmass for you too.

Here is a stylesheet that counts all characters in the attribute and element names and content
this is a rough estimate (well, not so rough) for the byte content of an ACCOUNT element
at the top you give the maximum size, and this stylesheet will put an attribute "
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:strip-space elements="*"/>
    <xsl:output indent="yes"/>
    <xsl:param name="max-size">1000</xsl:param>
    <xsl:template match="REQUEST">
        <xsl:copy>
            <xsl:copy-of select="@*"/>
            <xsl:apply-templates select="ACCOUNT[1]">
                <xsl:with-param name="cur-size" select="0"/>
            </xsl:apply-templates>
        </xsl:copy>
    </xsl:template>
    <xsl:template match="ACCOUNT">
        <xsl:param name="cur-size"/> <!-- passed in from previous node -->
        <!-- a concatenation of all account element names -->
        <xsl:variable name="elem-names">
            <xsl:for-each select="self::* | .//*">
                <xsl:text>[</xsl:text>
                <xsl:value-of select="name()"/>
                <xsl:text>]</xsl:text>
                <xsl:text>[/</xsl:text>
                <xsl:value-of select="name()"/>
                <xsl:text>]</xsl:text>
            </xsl:for-each>
        </xsl:variable>
        <!-- a concatenation of all account element content -->
        <xsl:variable name="elem-content"> 
            <xsl:for-each select=".//text()">
                <xsl:value-of select="."/>
            </xsl:for-each>
        </xsl:variable>
        <!-- a concatenation of all account attributes names -->
        <xsl:variable name="attr-names">
            <xsl:for-each select="self::*/@* | .//*/@*">
                <xsl:text>-</xsl:text>
                <xsl:value-of select="name()"/>
            </xsl:for-each>
        </xsl:variable>
        <!-- a concatenation of all account attribute values -->
        <xsl:variable name="attr-content">
            <xsl:for-each select="self::*/@* | .//*/@*">
                <xsl:text>{</xsl:text>
                <xsl:value-of select="."/>
                <xsl:text>}</xsl:text>
            </xsl:for-each>
        </xsl:variable>
        <!-- the sum of all string-lengths in an account !! estimate only -->
        <xsl:variable name="this-account-size" select="string-length($elem-names) + string-length($elem-content) + string-length($attr-names) + string-length($attr-content)"></xsl:variable>
        <!-- new-sum to pass to next node -->
        <xsl:variable name="new-size">
            <xsl:choose>
                <xsl:when test="$cur-size + $this-account-size &gt; $max-size">
                    <xsl:value-of select="0"/>
                </xsl:when>
                <xsl:otherwise>
                    <xsl:value-of select="$cur-size + $this-account-size"/>
                </xsl:otherwise>
            </xsl:choose>
        </xsl:variable>
        <xsl:copy>
            <xsl:if test="$cur-size + $this-account-size &gt; $max-size or not(preceding-sibling::ACCOUNT)">
                <xsl:attribute name="break-here">
                    <xsl:text>true</xsl:text>
                </xsl:attribute>
            </xsl:if>
            <xsl:copy-of select="@*"/>
            <xsl:copy-of select="node()"/>
        </xsl:copy>
        <xsl:apply-templates select="following-sibling::ACCOUNT[1]">
            <xsl:with-param name="cur-size" select="$new-size"/>
        </xsl:apply-templates>
    </xsl:template>

</xsl:stylesheet>

Open in new window

0
 
Geert BormansInformation ArchitectCommented:
Here is your second stylesheet
it wraps on ACCOUNT having the break-here attribute
this could be optimised, since I do too much walking up and down the preceding and following axes,
but first check wheither this behaves well in your architecture
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:strip-space elements="*"/>
    <xsl:output indent="yes"/>
    <xsl:template match="REQUEST">
        <xsl:copy>
            <xsl:copy-of select="@*"/>
            <xsl:apply-templates select="ACCOUNT[@break-here]">
            </xsl:apply-templates>
        </xsl:copy>
    </xsl:template>
    <xsl:template match="ACCOUNT[@break-here]">
        <xsl:variable name="this-id" select="generate-id()"/>
        <wrapper>
            <xsl:copy-of select="self::* | following-sibling::ACCOUNT[not(@break-here)][generate-id(preceding-sibling::ACCOUNT[@break-here][1]) = $this-id]" />
        </wrapper>
    </xsl:template>

</xsl:stylesheet>

Open in new window

0
 
Geert BormansInformation ArchitectCommented:
please forget my previous stylesheet, that is highly inefficient,
this one beats it in elegance and speed
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:strip-space elements="*"/>
    <xsl:output indent="yes"/>
    <xsl:template match="REQUEST">
        <xsl:copy>
            <xsl:copy-of select="@*"/>
            <xsl:apply-templates select="ACCOUNT[@break-here][1]">
            </xsl:apply-templates>
        </xsl:copy>
    </xsl:template>
    <xsl:template match="ACCOUNT[@break-here]">
        <wrapper>
            <xsl:copy-of select="." />
            <xsl:apply-templates select="following-sibling::ACCOUNT[1][not(@break-here)]" />
        </wrapper>
        <xsl:apply-templates select="following-sibling::ACCOUNT[@break-here][1]"></xsl:apply-templates>
    </xsl:template>
    <xsl:template match="ACCOUNT[not(@break-here)]">
        <xsl:copy-of select="." />
        <xsl:apply-templates select="following-sibling::ACCOUNT[1][not(@break-here)]" />
    </xsl:template>
</xsl:stylesheet>

Open in new window

0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Microsoft Windows 7 Basic

This introductory course to Windows 7 environment will teach you about working with the Windows operating system. You will learn about basic functions including start menu; the desktop; managing files, folders, and libraries.

  • 8
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now