asked on

Python script no longer works - only minor change made

Hello,

I used the script from https://www.experts-exchange.com/questions/23068228/Modify-Python-Script-for-more-automation.html with great success. Only now it doesn't work and I am unsure why. The only thing that has changed is the length of the digit after the diretory names from 2 to 3 (i.e. 01 is now 001) I believe I have updated the code accordingly to account for this change. I also changed the back slashes to forwards slashes. The original code is in the snippet box.

I get this error message when trying to execute the script:

C:\Documents and Settings\user>c:\python24\python "c:\new_donut.py"
Traceback (most recent call last):
File "c:\new_donut.py", line 51, in ?
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
File "c:\new_donut.py", line 40, in allLandcoverDonutAndIdSuffix
assert os.path.isdir(DnumDir)
AssertionError

----

The only output I get in my "tester.txt" is "3"
My Directory has

C:\Giswork\landcoverprep\Donut
C:\Giswork\landcoverprep\LandcoverI
C:\Giswork\landcoverprep\LandcoverL
C:\Giswork\landcoverprep\Results

Donut has folders named "donut_001 - donut_407, donut_431, donut_435, donut_440, donut_447, donut_448

LandcoverI has the same numbering (i.e. LandcoverI_001 - 407, 431, 435, 440, 447)

LandcoverL (is missing about 10)

I also tried not including LandcoverL to the same error message.

Please advise,
Thanks,
MJ

# Multi_Output_Map_Algebra_sample.py
# Description:
# Runs an expression built with the Map Algebra language.
# Requirements: None

import arcgisscripting
import glob
import os

gisworkDir = 'c:\\Giswork\\landcoverprep'

def log(s):
f = open("C:/tester.txt","w")
f.write(s)
f.close()

def allLandcoverDonutAndIdSuffix(gisworkDir):
'''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
log("1\n")
# Prepare the path to the donuts.
donutDir = os.path.join(gisworkDir, 'Donut')
assert os.path.isdir(donutDir)
log("2\n")
# Iterate through sequence of top-level landcover subdirectories.
mask = os.path.normpath(os.path.join(gisworkDir, 'landcover*'))
log(mask)
for LDir in glob.glob(mask): # full paths to LandcoverA, LandcoverB, etc.
subdir = os.path.basename(LDir)
log("3\n")
# Iterate through all numbered landcover subdirs inside.
mask2 = os.path.join(LDir, '*')
for LnumDir in glob.glob(mask2):
# Separate the counter part, the idSuffix and construct
# the related donut numbered directory name.
numSuffix = LnumDir[-3:] # last 3 chars
idSuffix = LnumDir[-5:] # last 5 chars
DnumDir = os.path.normpath(
os.path.join(donutDir, 'DONUT_' + numSuffix))
assert os.path.isdir(DnumDir)
log("4/n")
# Generate another output tuple.
yield (LnumDir, DnumDir, idSuffix)432

# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")

for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
# Construct the expression and the output directory name.
outDir = os.path.join(gisworkDir, 'Results', idSuffix)
log("\n" + outDir)
InExpression = expTemplate % (donut, landcover, landcover)
log(" " + InExpression)

# Check out Spatial Analyst extension license
gp.CheckOutExtension("Spatial")

# Process: MapAlgebra
gp.SingleOutputMapAlgebra_sa(InExpression, outDir)

log("Msg: " + gp.GetMessages())

# Multi_Output_Map_Algebra_sample.py
# Description:
#   Runs an expression built with the Map Algebra language.
# Requirements: None
 
import pdb
import arcgisscripting
import glob
import os
 
gisworkDir = 'c:\Giswork'
 
def log(s):
    f = open("C:/tester.txt","w")
    f.write(s)
    f.close()
 
 
def allLandcoverDonutAndIdSuffix(gisworkDir):
    '''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
    ##log("1\n")
    # Prepare the path to the donuts.
    donutDir = os.path.join(gisworkDir, 'landcoverprep', 'Donut')
    assert os.path.isdir(donutDir)
    ##log("2\n")
    # Iterate through sequence of top-level landcover subdirectories.
    mask = os.path.normpath(os.path.join(gisworkDir, 'Landcoverprep\landcover*'))
 
    ##log(mask)             
    for LDir in glob.glob(mask):      # full paths to LandcoverA, LandcoverB, etc.
        subdir = os.path.basename(LDir)
        ##log("3\n")
        # Iterate through all numbered landcover subdirs inside.
        mask2 = os.path.join(LDir, '*')
#        pdb.set_trace()
        for LnumDir in glob.glob(mask2):
 
            # Separate the counter part, the idSuffix and construct 
            # the related donut numbered directory name.
            numSuffix = LnumDir[-2:]   # last 2 chars
            idSuffix = LnumDir[-4:]    # last 4 chars
            DnumDir = os.path.normpath(
                         os.path.join(donutDir, 'DONUT_' + numSuffix))
            assert os.path.isdir(DnumDir)  
            ##log("4")
            # Generate another output tuple.
            yield (LnumDir, DnumDir, idSuffix)
 
 
# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")
 
#pdb.set_trace()
 
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
    print (landcover,donut,idSuffix)
    pdb.set_trace()
    # Construct the expression and the output directory name.
    outDir = os.path.join(gisworkDir, 'landcoverprep', 'Results', idSuffix)
    log("\n" + outDir)
    InExpression = expTemplate % (donut, landcover, landcover)
    log("  " + InExpression)
 
    # Check out Spatial Analyst extension license
    gp.CheckOutExtension("Spatial")
 
    # Process: MapAlgebra
    gp.SingleOutputMapAlgebra_sa(InExpression, outDir)
 
    log("Msg: " + gp.GetMessages())

Open in new window

mgjust

ASKER

Hello,
Using the python script new_donut.py and my old files with only two digits at the end I get only the final output (in this test case H_71). Which was a problem I had initially with this script back in January.

I had a donut and a landcoverH directory for this trial
i.e. C:\Giswork\landcoverprep\LandcoverA\landcoverA_01 & C:\Giswork\landcoverprep\donut\donut_01

output from log file:

Please advise,
MJ

# Multi_Output_Map_Algebra_sample.py
# Description:
#   Runs an expression built with the Map Algebra language.
# Requirements: None
 
import arcgisscripting
import glob
import os
 
gisworkDir = 'c:/Giswork/landcoverprep'
 
def log(s):
    f = open("C:/tester.txt","w")
    f.write(s)
    f.close()
 
 
def allLandcoverDonutAndIdSuffix(gisworkDir):
    '''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
    log("1\n")
    # Prepare the path to the donuts.
    donutDir = os.path.join(gisworkDir, 'Donut')
    assert os.path.isdir(donutDir)
    log("2\n")
    # Iterate through sequence of top-level landcover subdirectories.
    mask = os.path.normpath(os.path.join(gisworkDir, 'landcover*'))
    log(mask)             
    for LDir in glob.glob(mask):      # full paths to LandcoverA, LandcoverB, etc.
        subdir = os.path.basename(LDir)
        log("3\n")
        # Iterate through all numbered landcover subdirs inside.
        mask2 = os.path.join(LDir, '*')
        for LnumDir in glob.glob(mask2):
            # Separate the counter part, the idSuffix and construct 
            # the related donut numbered directory name.
            numSuffix = LnumDir[-3:]   # last 3 chars
            idSuffix = LnumDir[-5:]    # last 5 chars
            DnumDir = os.path.normpath(
                         os.path.join(donutDir, 'DONUT_' + numSuffix))
            assert os.path.isdir(DnumDir)  
            log("4/n")
            # Generate another output tuple.
            yield (LnumDir, DnumDir, idSuffix)
 
 
# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")
 
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
    # Construct the expression and the output directory name.
    outDir = os.path.join(gisworkDir, 'Results', idSuffix)
    log("\n" + outDir)
    InExpression = expTemplate % (donut, landcover, landcover)
    log("  " + InExpression)
 
    # Check out Spatial Analyst extension license
    gp.CheckOutExtension("Spatial")
 
    # Process: MapAlgebra
    gp.SingleOutputMapAlgebra_sa(InExpression, outDir)
 
    log("Msg: " + gp.GetMessages())

Open in new window

tester1.txt

mgjust

ASKER

Using the code from the original posting:
https://www.experts-exchange.com/questions/23068228/Modify-Python-Script-for-more-automation.html

and my old files with two digits the code works.

# Multi_Output_Map_Algebra_sample.py
# Description:
#   Runs an expression built with the Map Algebra language.
# Requirements: None
 
import arcgisscripting
import glob
import os
 
gisworkDir = 'c:\Giswork'
 
def log(s):
    f = open("C:/tester_new.txt","w")
    f.write(s)
    f.close()
 
 
def allLandcoverDonutAndIdSuffix(gisworkDir):
    '''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
    ##log("1\n")
    # Prepare the path to the donuts.
    donutDir = os.path.join(gisworkDir, 'landcoverprep', 'Donut')
    assert os.path.isdir(donutDir)
    ##log("2\n")
    # Iterate through sequence of top-level landcover subdirectories.
    mask = os.path.normpath(os.path.join(gisworkDir, 'Landcoverprep\landcover*'))
    ##log(mask)             
    for LDir in glob.glob(mask):      # full paths to LandcoverA, LandcoverB, etc.
        subdir = os.path.basename(LDir)
        ##log("3\n")
        # Iterate through all numbered landcover subdirs inside.
        mask2 = os.path.join(LDir, '*')
        for LnumDir in glob.glob(mask2):
            # Separate the counter part, the idSuffix and construct 
            # the related donut numbered directory name.
            numSuffix = LnumDir[-2:]   # last 2 chars
            idSuffix = LnumDir[-4:]    # last 4 chars
            DnumDir = os.path.normpath(
                         os.path.join(donutDir, 'DONUT_' + numSuffix))
            assert os.path.isdir(DnumDir)  
            ##log("4")
            # Generate another output tuple.
            yield (LnumDir, DnumDir, idSuffix)
 
 
# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")
 
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
    # Construct the expression and the output directory name.
    outDir = os.path.join(gisworkDir, 'landcoverprep', 'Results', idSuffix)
    log("\n" + outDir)
    InExpression = expTemplate % (donut, landcover, landcover)
    log("  " + InExpression)
 
    # Check out Spatial Analyst extension license
    gp.CheckOutExtension("Spatial")
 
    # Process: MapAlgebra
    gp.SingleOutputMapAlgebra_sa(InExpression, outDir)
 
    log("Msg: " + gp.GetMessages())

Open in new window

mgjust

ASKER

Then using the script above that worked and changing :

numSuffix = LnumDir[-3:] # last 2 chars
idSuffix = LnumDir[-5:] # last 4 chars

from -2 and -4

& and i changed all the 'landcoverprep' to 'landcoverprepnew'

I get the following error:

C:\Documents and Settings\user>c:\python24\python "c:\new2.py"
Traceback (most recent call last):
File "c:\new2.py", line 51, in ?
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
File "c:\new2.py", line 40, in allLandcoverDonutAndIdSuffix
assert os.path.isdir(DnumDir)
AssertionError

Please advise,
MJ

# Multi_Output_Map_Algebra_sample.py
# Description:
#   Runs an expression built with the Map Algebra language.
# Requirements: None
 
import arcgisscripting
import glob
import os
 
gisworkDir = 'c:\Giswork'
 
def log(s):
    f = open("C:/tester.txt","w")
    f.write(s)
    f.close()
 
 
def allLandcoverDonutAndIdSuffix(gisworkDir):
    '''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
    ##log("1\n")
    # Prepare the path to the donuts.
    donutDir = os.path.join(gisworkDir, 'landcoverprepnew', 'Donut')
    assert os.path.isdir(donutDir)
    ##log("2\n")
    # Iterate through sequence of top-level landcover subdirectories.
    mask = os.path.normpath(os.path.join(gisworkDir, 'Landcoverprepnew\landcover*'))
    ##log(mask)             
    for LDir in glob.glob(mask):      # full paths to LandcoverA, LandcoverB, etc.
        subdir = os.path.basename(LDir)
        ##log("3\n")
        # Iterate through all numbered landcover subdirs inside.
        mask2 = os.path.join(LDir, '*')
        for LnumDir in glob.glob(mask2):
            # Separate the counter part, the idSuffix and construct 
            # the related donut numbered directory name.
            numSuffix = LnumDir[-3:]   # last 2 chars
            idSuffix = LnumDir[-5:]    # last 4 chars
            DnumDir = os.path.normpath(
                         os.path.join(donutDir, 'DONUT_' + numSuffix))
            assert os.path.isdir(DnumDir)  
            ##log("4/n")
            # Generate another output tuple.
            yield (LnumDir, DnumDir, idSuffix)
 
 
# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")
 
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
    # Construct the expression and the output directory name.
    outDir = os.path.join(gisworkDir, 'landcoverprepnew', 'Results', idSuffix)
    log("\n" + outDir)
    InExpression = expTemplate % (donut, landcover, landcover)
    log("  " + InExpression)
 
    # Check out Spatial Analyst extension license
    gp.CheckOutExtension("Spatial")
 
    # Process: MapAlgebra
    gp.SingleOutputMapAlgebra_sa(InExpression, outDir)
 
    log("Msg: " + gp.GetMessages())

Open in new window

pepr

This is why the asserts are good friends. Something have changed and it is not reflected in the code. Ift the command failed...

assert os.path.isdir(DnumDir)

it says thatSays that DnumDir does not exist. As its name is constructed earlier from 2-digits number and now from 3-digits number. If I recall correctly, the DONUT_xx directories are only loosely coupled with the LandcoverN_xx directories. The question now is...

Do you want to protect DONUT_xx (with 2 digits) also for future? Do you want to have both 2digits and 3digits names? How should it behave if the directories are mixed? If you want to have, say, DONUT_12 and DONUT_012, what is preferred?

mgjust

ASKER

Pepr:
From now on all Landcover and Donut directories have three digits (donut_xxx & landcover*_xxx) and will not be mixed. Yes, the same donut (e.g. donut_123) is used for all landcovers (e.g. LandcoverA_123, LandcoverB_123, etc). How come changing the

numSuffix = LnumDir[-3:] # last 2 chars
idSuffix = LnumDir[-5:] # last 4 chars

did not solve the problem?

Finally, does it work by searching for those digits? Only using Donut_001 with Landcover*_001, or if there was no landcover*_001 would Donut_001 be matched with Landcover*_002? If this last piece is more complicated, then I will just use place holder directories to circumvent this, because the automation with 3 digits is what I am really eager for.

Thanks again,
MJ

SOLUTION

Duncan Roe

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

mgjust

ASKER

duncan_roe:
Thanks for the advice,
As posted above I did know the mods between the old and the new. Perhaps my question title "Python script no longer works - only minor change made" was a bit sophomoric but was hoping that it would produce results. I will look into these programs you suggested as I can see there usefulness. On the plus side there is no management to convince just myself. And if you have any suggestions about fixing this script I'd also like to hear it.

Thanks again,
MJ

ASKER CERTIFIED SOLUTION

pepr

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

mgjust

ASKER

pepr:

Sorry for the confusion but the script with the added lines of code works as wanted.

Thanks again for your patience and answers.

Cheers,
MJ

mgjust

ASKER

Thank you.

justearth

It appears the issue has returned. I have posted a new question at:

https://www.experts-exchange.com/questions/23993825/Python-Script-Assertion-Error.html

Thanks,
JE