Using Python to automatically update KML file

In the attached KML, I am updating the transparency of my PolyStyle color from 7d (in 7dff0000) to an arbitrary fc (in fcff0000).

Rather than using an arbitrary update value, I'd like to pass a value (somewhere between 00 and ff) in through a Python script, probably connected to the KML via a network link.  The design would be as follows:

1) The KML contains <TimeSpan> begin and end dates for individual decadal censuses (e.g., 1900 and 1910, etc.)
2) When executed, the KML calls the Python script through a <NetworkLink>, passing in the beginning and end dates.
3) The Python script searches a data table to find the population density for Azusa at that census date.
4) Once the density is found, the Python script finds the corresponding 256-bit color transparency, for both the beginning and end dates, and sends it back to the KML.
5) The KML substitutes the two transparency arguments it receives into the <color> tags.

I recognize that this may be difficult if not impossible to do in the way I described, so any alternative approach would also be appreciated.  (P.S. -- the file is actually a kml; I just needed to change the extension to transmit it).

Thanks in advance for your help.     Azusa.txt
cshore12Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

cshore12Author Commented:
I've maximized the point value
peprCommented:
Actually, the attached Azusa.kml does not contain the TimeSpan tags. There are two color elements -- under Document/Style/PolyStyle and under Document/gx:Tour/gx:Playlist/gx:AnimatedUpdate/Update/Change/PolyStyle.

There are several options to proces XML files in Python.  I personally suggest to focus on ElementTree (the xml.etree.ElementTree standard module from Python 2.5 on) instead of using xml.dom.  It is more natural for Python.  The DOM is rather heavy handed because of the forced API that may be acceptable in other languages that are not that much flexible asi Python is (which does not mean that they are worse ;).

Consider the following the first step towards the answer.

import xml.etree.ElementTree as ET

fname = 'Azusa.kml'
 
tree = ET.parse(fname)  # Build the element tree (in memory).

elem = tree.getroot()   # The root element object.
print elem.tag          # See the namespace of the element.

# The child element of the root element.
doc = tree.getroot().find('{http://www.opengis.net/kml/2.2}Document')
print doc

# ... and deeper.
style = doc.find('{http://www.opengis.net/kml/2.2}Style')
print style

# ... and deeper.
polystyle = style.find('{http://www.opengis.net/kml/2.2}PolyStyle')
print polystyle

# ... and deeper -- the first color element.
color = polystyle.find('{http://www.opengis.net/kml/2.2}color')
print color

# It is possible to display a fragment -- here the Style element.
ET.dump(style)


# Use the (limited) XPath capabilities to find certain element.  Here all
# color elements are found and their text is changed.
for color in tree.findall('.//{http://www.opengis.net/kml/2.2}color'):
    print 'Original value of the', color, 'is', color.text
    color.text = 'ffffffff'
    print 'New value is', color.text
    print

# Write the modified structure to another file.
tree.write('Azusa_new.kml', 'utf-8', True)

Open in new window


Notice that part of the problem is that KML files define namespaces that are a kind of abbreviated in the root element. It may not be that obvious at the first time that you have to use them in XPath expressions.  Notice also that that the produced file looks a bit differently.  Anyway, it works with say GoogleEarth.

See http://effbot.org/zone/element-xpath.htm for the XPath support in ElementTree.

Read the "Dive Into Python 3" by Mark Pilgrim, chapter 12. XML (http://diveintopython3.org/xml.html).  You may also want to focus on third party module lxml  (http://diveintopython3.org/xml.html#xml-lxml) -- another ElementTree implementation with different capabilities to work with namespaces.

Specify the next step -- what exactly you want to do knowing the above code.
cshore12Author Commented:
pepr:

I really appreciate the level of detail you provided.  Also, since I'm now restricted to working on this 'weekends only', my replies will be pretty sporadic.

Before I go down the path you suggest, let me expand on the original request.  The intention is to show the city polygon with different transparency gradations, depending on population density at decadal censuses (for the ttrrggbb value of the <color> tag, only the tt will change).  For example, the transparency (and hence perceived color) will be one value in 1900, and will continuously slide to an ending value for 1910, depending on the densities at those two endpoints.  

Now to do this, I can use multiple <TimeSpan> tags, so that shouldn't be a problem.  The real issues have to do with the density data are obtained, and how the corresponding transparency is rendered.  Basically, both these values will be stored in database fields which will be accessed through the Python file.  Again, that part should not be problemmatic.  What is problemmatic is getting the new (beginning and end) transparency values back to the KML so that they can be used in the <TimeSpan> <begin> and <end> tags (yes, and I do realize that the <TimeSpan> tags are not in the kml I sent you).

I think that you're on to something using XPath and the ElementTree namespace.  I could basically modify what you have so that the color.text property is obtained from the database.  However, after making this reassignment (line 34), I'd like to pass this value back into the original Azusa.kml, not create a new file for each decade.

As I said earlier, it doesn't seem like you can pass an argument like that back to an xml file, like you can in a standard programming language.  If you think there's a way of doing it, I'll look through the references you provided or be open to other suggestions.  And, yes, I'm doing all this through Google Earth.      
Your Guide to Achieving IT Business Success

The IT Service Excellence Tool Kit has best practices to keep your clients happy and business booming. Inside, you’ll find everything you need to increase client satisfaction and retention, become more competitive, and increase your overall success.

peprCommented:
cshore12

No problem with sporadic communication.  My above example was only to show how to get the value from an element and to set the new value.  The different filename is no problem here.  This was done just for later comparison of the Azusa.kml and Azusa_new.kml.  If you use the same name for the file, it will write back the new content.  Or it is possible to rename the original file name to some backup name (or to move it to some archive directory).

Also the color 'ffffffff' was used only as an example.

Try the following set of functions to reach your goal.

import xml.etree.ElementTree as ET

def loadKMLFile(fname):
    tree = ET.parse(fname)  # Build the element tree (in memory)...
    return tree             # ... and return the element tree as the result


def saveKMLFile(fname, tree):
    '''Write current content of the tree to the file of the fname.'''    
    tree.write(fname, 'utf-8', True)


def getKMLDates(tree):
    '''Get the beginning and the end dates and return the tuple.'''
    return (1900, 1910)     # dummy, hardwired here
    

def setKMLTransparencyTo(tr, tree):
    '''Set the color transparency for the element tree.
    
    The tr is expected to be the two-char string.
    '''
    
    # Through all the color elements, get its original value, and 
    # put together the wanted transparency value with the tail of the original
    # value.
    for color in tree.findall('.//{http://www.opengis.net/kml/2.2}color'):
        color.text = tr + color.text[2:]  # the tail is from index 2 to the end



# Here is the body to show how the function could be used.
if __name__ == '__main__':
    
    fname = 'Azusa.kml'       # get the filename
    tree = loadKMLFile(fname) # and parse its content
    
    date_start, date_end = getKMLDates(tree)  # extract the dates from inside
    
    # Ask the database for the transparency. The integer transparency can be
    # easily converted to the hexadecimal one. Here only to show principle...
    tr = 'fc'
    
    setKMLTransparencyTo(tr, tree)  # set the wanted transparency
    saveKMLFile(fname, tree     )        # and write everything back to the same file

Open in new window

cshore12Author Commented:
pepr,

First of all, thanks so much for introducing me to the ElementTree technology.  I'm looking forward to getting more in depth on the 3d party software as well.

There's still a basic problem here.  Your code (def saveKMLFile) basically refreshes the kml every time it's called.  So if I want to show discrete changes in transparency every decade for 100 years, I would have to call your program 10 times.

What I really want to do is show continuous transparency changes within each decade, and then across decades.  The following code, from the original kml, works great if I hard-wire the <color> tag:

      <gx:Tour>
              <name>Decades</name>

              <gx:Playlist>

                      <gx:AnimatedUpdate>
                               <gx:duration>5</gx:duration>
                           <Update>
                                  <targetHref></targetHref>
                                  <Change>
                                          <PolyStyle targetId="PolyColor">
                                          <color>AA008800</color>
                                    </PolyStyle>                              
                                     </Change>
                             </Update>
                        </gx:AnimatedUpdate>

                        <gx:Wait>
                          <gx:duration>5</gx:duration>
                        </gx:Wait>
                 </gx:Playlist>
         </gx:Tour>    

(I'm also pretty sure I can get the same functionality using <TimeSpan> tags.)

These tags allow for the linear change of transparency from (say, a beginning value of 00) to an end value of AA.   If I could substitute a new value in the <color> tag, for each decade, based on a table entry, I'd be set.

Now I could call your .py code every 0.1 seconds or so to give the illusion of continuity, but that would obviously not be feasible from a systems standpoint.

Anyway, I think the answer has to lie in using something like DOM or XPATH to substitute a new <color> tag each time.  I'll keep working in that direction.

Incidentally, I got a curious error when I ran your code as an independent script.  The program tripped up on the tree.write method - saying that it was trying to pass 4 arguments, when only 3 are allowed (someone doesn't know how to count!).  When I got rid of the last argument (True), it ran fine.  I couldn't find much documentation on this method, so am not sure what this Boolean represents.

Thanks again!
cshore12Author Commented:
pepr --

If you think, like I do, that this can't be done in a kml (short of hard-coding the actual transparency values), just let me know and I'll go ahead and award you the points.
peprCommented:
Sorry, I did not notice your previous comment.  I am going to look at it soon.
peprCommented:
How do you want to display the result. Do you want to use the Google Earth?

For the error with the number of the argument... I did use Python 2.7 that apparently has more advanced version of the ElementTree -- the interface changed from:

  write(file, encoding="us-ascii", xml_declaration=None, method="xml")

see http://docs.python.org/library/xml.etree.elementtree.html#xml.etree.ElementTree.ElementTree.write

The older Python uses the simpler

  write(file[, encoding])

see http://docs.python.org/release/2.6.6/library/xml.etree.elementtree.html#xml.etree.ElementTree.ElementTree.write for 2.6 and the bottom of http://docs.python.org/release/2.5.4/lib/elementtree-elementtree-objects.html for 2.5.

Still, it may seem that the error message lies about the number of the arguments.  However, the object instance is counted as one argument.  The code for calling the tree method where the tree is the instance of the xml.etree.ElementTree class...

  tree.write(filename, encoding)

is equivalent to

  xml.etree.ElementTree.write(tree, filename, encoding)

The "tree" is the thing that is named "self" in the method definition -- hence the 3 arguments.

 

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
cshore12Author Commented:
pepr:

I think we're on different wavelengths here.  Your responses indicate to me that the only way to refresh the display is through a write() method.  As I said, I don't think this will do, as I'm looking for a continually changing gradient for the feature transparency.  As I said, though, I'll give you the points for the super effort you put in.  
cshore12Author Commented:
pepr tried hard to respond to the question.  However, I came to the conclusion that there's simply no way to create a continuously refreshing gradient for a kml.  His suggestion to continually refesh the whole file, rather than just a single property, doesn't seem feasible for my purposes.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
GIS/GPS Programming

From novice to tech pro — start learning today.