cshore12
asked on
Using Python to automatically update KML file
In the attached KML, I am updating the transparency of my PolyStyle color from 7d (in 7dff0000) to an arbitrary fc (in fcff0000).
Rather than using an arbitrary update value, I'd like to pass a value (somewhere between 00 and ff) in through a Python script, probably connected to the KML via a network link. The design would be as follows:
1) The KML contains <TimeSpan> begin and end dates for individual decadal censuses (e.g., 1900 and 1910, etc.)
2) When executed, the KML calls the Python script through a <NetworkLink>, passing in the beginning and end dates.
3) The Python script searches a data table to find the population density for Azusa at that census date.
4) Once the density is found, the Python script finds the corresponding 256-bit color transparency, for both the beginning and end dates, and sends it back to the KML.
5) The KML substitutes the two transparency arguments it receives into the <color> tags.
I recognize that this may be difficult if not impossible to do in the way I described, so any alternative approach would also be appreciated. (P.S. -- the file is actually a kml; I just needed to change the extension to transmit it).
Thanks in advance for your help. Azusa.txt
Rather than using an arbitrary update value, I'd like to pass a value (somewhere between 00 and ff) in through a Python script, probably connected to the KML via a network link. The design would be as follows:
1) The KML contains <TimeSpan> begin and end dates for individual decadal censuses (e.g., 1900 and 1910, etc.)
2) When executed, the KML calls the Python script through a <NetworkLink>, passing in the beginning and end dates.
3) The Python script searches a data table to find the population density for Azusa at that census date.
4) Once the density is found, the Python script finds the corresponding 256-bit color transparency, for both the beginning and end dates, and sends it back to the KML.
5) The KML substitutes the two transparency arguments it receives into the <color> tags.
I recognize that this may be difficult if not impossible to do in the way I described, so any alternative approach would also be appreciated. (P.S. -- the file is actually a kml; I just needed to change the extension to transmit it).
Thanks in advance for your help. Azusa.txt
Actually, the attached Azusa.kml does not contain the TimeSpan tags. There are two color elements -- under Document/Style/PolyStyle and under Document/gx:Tour/gx:Playli
There are several options to proces XML files in Python. I personally suggest to focus on ElementTree (the xml.etree.ElementTree standard module from Python 2.5 on) instead of using xml.dom. It is more natural for Python. The DOM is rather heavy handed because of the forced API that may be acceptable in other languages that are not that much flexible asi Python is (which does not mean that they are worse ;).
Consider the following the first step towards the answer.
Notice that part of the problem is that KML files define namespaces that are a kind of abbreviated in the root element. It may not be that obvious at the first time that you have to use them in XPath expressions. Notice also that that the produced file looks a bit differently. Anyway, it works with say GoogleEarth.
See http://effbot.org/zone/element-xpath.htm for the XPath support in ElementTree.
Read the "Dive Into Python 3" by Mark Pilgrim, chapter 12. XML (http://diveintopython3.org/xml.html). You may also want to focus on third party module lxml (http://diveintopython3.org/xml.html#xml-lxml) -- another ElementTree implementation with different capabilities to work with namespaces.
Specify the next step -- what exactly you want to do knowing the above code.
There are several options to proces XML files in Python. I personally suggest to focus on ElementTree (the xml.etree.ElementTree standard module from Python 2.5 on) instead of using xml.dom. It is more natural for Python. The DOM is rather heavy handed because of the forced API that may be acceptable in other languages that are not that much flexible asi Python is (which does not mean that they are worse ;).
Consider the following the first step towards the answer.
import xml.etree.ElementTree as ET
fname = 'Azusa.kml'
tree = ET.parse(fname) # Build the element tree (in memory).
elem = tree.getroot() # The root element object.
print elem.tag # See the namespace of the element.
# The child element of the root element.
doc = tree.getroot().find('{http://www.opengis.net/kml/2.2}Document')
print doc
# ... and deeper.
style = doc.find('{http://www.opengis.net/kml/2.2}Style')
print style
# ... and deeper.
polystyle = style.find('{http://www.opengis.net/kml/2.2}PolyStyle')
print polystyle
# ... and deeper -- the first color element.
color = polystyle.find('{http://www.opengis.net/kml/2.2}color')
print color
# It is possible to display a fragment -- here the Style element.
ET.dump(style)
# Use the (limited) XPath capabilities to find certain element. Here all
# color elements are found and their text is changed.
for color in tree.findall('.//{http://www.opengis.net/kml/2.2}color'):
print 'Original value of the', color, 'is', color.text
color.text = 'ffffffff'
print 'New value is', color.text
print
# Write the modified structure to another file.
tree.write('Azusa_new.kml', 'utf-8', True)Notice that part of the problem is that KML files define namespaces that are a kind of abbreviated in the root element. It may not be that obvious at the first time that you have to use them in XPath expressions. Notice also that that the produced file looks a bit differently. Anyway, it works with say GoogleEarth.
See http://effbot.org/zone/element-xpath.htm for the XPath support in ElementTree.
Read the "Dive Into Python 3" by Mark Pilgrim, chapter 12. XML (http://diveintopython3.org/xml.html). You may also want to focus on third party module lxml (http://diveintopython3.org/xml.html#xml-lxml) -- another ElementTree implementation with different capabilities to work with namespaces.
Specify the next step -- what exactly you want to do knowing the above code.
ASKER
pepr:
I really appreciate the level of detail you provided. Also, since I'm now restricted to working on this 'weekends only', my replies will be pretty sporadic.
Before I go down the path you suggest, let me expand on the original request. The intention is to show the city polygon with different transparency gradations, depending on population density at decadal censuses (for the ttrrggbb value of the <color> tag, only the tt will change). For example, the transparency (and hence perceived color) will be one value in 1900, and will continuously slide to an ending value for 1910, depending on the densities at those two endpoints.
Now to do this, I can use multiple <TimeSpan> tags, so that shouldn't be a problem. The real issues have to do with the density data are obtained, and how the corresponding transparency is rendered. Basically, both these values will be stored in database fields which will be accessed through the Python file. Again, that part should not be problemmatic. What is problemmatic is getting the new (beginning and end) transparency values back to the KML so that they can be used in the <TimeSpan> <begin> and <end> tags (yes, and I do realize that the <TimeSpan> tags are not in the kml I sent you).
I think that you're on to something using XPath and the ElementTree namespace. I could basically modify what you have so that the color.text property is obtained from the database. However, after making this reassignment (line 34), I'd like to pass this value back into the original Azusa.kml, not create a new file for each decade.
As I said earlier, it doesn't seem like you can pass an argument like that back to an xml file, like you can in a standard programming language. If you think there's a way of doing it, I'll look through the references you provided or be open to other suggestions. And, yes, I'm doing all this through Google Earth.
I really appreciate the level of detail you provided. Also, since I'm now restricted to working on this 'weekends only', my replies will be pretty sporadic.
Before I go down the path you suggest, let me expand on the original request. The intention is to show the city polygon with different transparency gradations, depending on population density at decadal censuses (for the ttrrggbb value of the <color> tag, only the tt will change). For example, the transparency (and hence perceived color) will be one value in 1900, and will continuously slide to an ending value for 1910, depending on the densities at those two endpoints.
Now to do this, I can use multiple <TimeSpan> tags, so that shouldn't be a problem. The real issues have to do with the density data are obtained, and how the corresponding transparency is rendered. Basically, both these values will be stored in database fields which will be accessed through the Python file. Again, that part should not be problemmatic. What is problemmatic is getting the new (beginning and end) transparency values back to the KML so that they can be used in the <TimeSpan> <begin> and <end> tags (yes, and I do realize that the <TimeSpan> tags are not in the kml I sent you).
I think that you're on to something using XPath and the ElementTree namespace. I could basically modify what you have so that the color.text property is obtained from the database. However, after making this reassignment (line 34), I'd like to pass this value back into the original Azusa.kml, not create a new file for each decade.
As I said earlier, it doesn't seem like you can pass an argument like that back to an xml file, like you can in a standard programming language. If you think there's a way of doing it, I'll look through the references you provided or be open to other suggestions. And, yes, I'm doing all this through Google Earth.
cshore12
No problem with sporadic communication. My above example was only to show how to get the value from an element and to set the new value. The different filename is no problem here. This was done just for later comparison of the Azusa.kml and Azusa_new.kml. If you use the same name for the file, it will write back the new content. Or it is possible to rename the original file name to some backup name (or to move it to some archive directory).
Also the color 'ffffffff' was used only as an example.
Try the following set of functions to reach your goal.
No problem with sporadic communication. My above example was only to show how to get the value from an element and to set the new value. The different filename is no problem here. This was done just for later comparison of the Azusa.kml and Azusa_new.kml. If you use the same name for the file, it will write back the new content. Or it is possible to rename the original file name to some backup name (or to move it to some archive directory).
Also the color 'ffffffff' was used only as an example.
Try the following set of functions to reach your goal.
import xml.etree.ElementTree as ET
def loadKMLFile(fname):
tree = ET.parse(fname) # Build the element tree (in memory)...
return tree # ... and return the element tree as the result
def saveKMLFile(fname, tree):
'''Write current content of the tree to the file of the fname.'''
tree.write(fname, 'utf-8', True)
def getKMLDates(tree):
'''Get the beginning and the end dates and return the tuple.'''
return (1900, 1910) # dummy, hardwired here
def setKMLTransparencyTo(tr, tree):
'''Set the color transparency for the element tree.
The tr is expected to be the two-char string.
'''
# Through all the color elements, get its original value, and
# put together the wanted transparency value with the tail of the original
# value.
for color in tree.findall('.//{http://www.opengis.net/kml/2.2}color'):
color.text = tr + color.text[2:] # the tail is from index 2 to the end
# Here is the body to show how the function could be used.
if __name__ == '__main__':
fname = 'Azusa.kml' # get the filename
tree = loadKMLFile(fname) # and parse its content
date_start, date_end = getKMLDates(tree) # extract the dates from inside
# Ask the database for the transparency. The integer transparency can be
# easily converted to the hexadecimal one. Here only to show principle...
tr = 'fc'
setKMLTransparencyTo(tr, tree) # set the wanted transparency
saveKMLFile(fname, tree ) # and write everything back to the same fileASKER
pepr,
First of all, thanks so much for introducing me to the ElementTree technology. I'm looking forward to getting more in depth on the 3d party software as well.
There's still a basic problem here. Your code (def saveKMLFile) basically refreshes the kml every time it's called. So if I want to show discrete changes in transparency every decade for 100 years, I would have to call your program 10 times.
What I really want to do is show continuous transparency changes within each decade, and then across decades. The following code, from the original kml, works great if I hard-wire the <color> tag:
<gx:Tour>
<name>Decades</name>
<gx:Playlist>
<gx:AnimatedUpdate>
<gx:duration>5</gx:duratio
<Update>
<targetHref></targetHref>
<Change>
<PolyStyle targetId="PolyColor">
<color>AA008800</color>
</PolyStyle>
</Change>
</Update>
</gx:AnimatedUpdate>
<gx:Wait>
<gx:duration>5</gx:duratio
</gx:Wait>
</gx:Playlist>
</gx:Tour>
(I'm also pretty sure I can get the same functionality using <TimeSpan> tags.)
These tags allow for the linear change of transparency from (say, a beginning value of 00) to an end value of AA. If I could substitute a new value in the <color> tag, for each decade, based on a table entry, I'd be set.
Now I could call your .py code every 0.1 seconds or so to give the illusion of continuity, but that would obviously not be feasible from a systems standpoint.
Anyway, I think the answer has to lie in using something like DOM or XPATH to substitute a new <color> tag each time. I'll keep working in that direction.
Incidentally, I got a curious error when I ran your code as an independent script. The program tripped up on the tree.write method - saying that it was trying to pass 4 arguments, when only 3 are allowed (someone doesn't know how to count!). When I got rid of the last argument (True), it ran fine. I couldn't find much documentation on this method, so am not sure what this Boolean represents.
Thanks again!
First of all, thanks so much for introducing me to the ElementTree technology. I'm looking forward to getting more in depth on the 3d party software as well.
There's still a basic problem here. Your code (def saveKMLFile) basically refreshes the kml every time it's called. So if I want to show discrete changes in transparency every decade for 100 years, I would have to call your program 10 times.
What I really want to do is show continuous transparency changes within each decade, and then across decades. The following code, from the original kml, works great if I hard-wire the <color> tag:
<gx:Tour>
<name>Decades</name>
<gx:Playlist>
<gx:AnimatedUpdate>
<gx:duration>5</gx:duratio
<Update>
<targetHref></targetHref>
<Change>
<PolyStyle targetId="PolyColor">
<color>AA008800</color>
</PolyStyle>
</Change>
</Update>
</gx:AnimatedUpdate>
<gx:Wait>
<gx:duration>5</gx:duratio
</gx:Wait>
</gx:Playlist>
</gx:Tour>
(I'm also pretty sure I can get the same functionality using <TimeSpan> tags.)
These tags allow for the linear change of transparency from (say, a beginning value of 00) to an end value of AA. If I could substitute a new value in the <color> tag, for each decade, based on a table entry, I'd be set.
Now I could call your .py code every 0.1 seconds or so to give the illusion of continuity, but that would obviously not be feasible from a systems standpoint.
Anyway, I think the answer has to lie in using something like DOM or XPATH to substitute a new <color> tag each time. I'll keep working in that direction.
Incidentally, I got a curious error when I ran your code as an independent script. The program tripped up on the tree.write method - saying that it was trying to pass 4 arguments, when only 3 are allowed (someone doesn't know how to count!). When I got rid of the last argument (True), it ran fine. I couldn't find much documentation on this method, so am not sure what this Boolean represents.
Thanks again!
ASKER
pepr --
If you think, like I do, that this can't be done in a kml (short of hard-coding the actual transparency values), just let me know and I'll go ahead and award you the points.
If you think, like I do, that this can't be done in a kml (short of hard-coding the actual transparency values), just let me know and I'll go ahead and award you the points.
Sorry, I did not notice your previous comment. I am going to look at it soon.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
pepr:
I think we're on different wavelengths here. Your responses indicate to me that the only way to refresh the display is through a write() method. As I said, I don't think this will do, as I'm looking for a continually changing gradient for the feature transparency. As I said, though, I'll give you the points for the super effort you put in.
I think we're on different wavelengths here. Your responses indicate to me that the only way to refresh the display is through a write() method. As I said, I don't think this will do, as I'm looking for a continually changing gradient for the feature transparency. As I said, though, I'll give you the points for the super effort you put in.
ASKER
pepr tried hard to respond to the question. However, I came to the conclusion that there's simply no way to create a continuously refreshing gradient for a kml. His suggestion to continually refesh the whole file, rather than just a single property, doesn't seem feasible for my purposes.
ASKER