Link to home
Get AccessLog in
Avatar of mgjust
mgjust

asked on

Python script no longer works - only minor change made

Hello,

I used the script from https://www.experts-exchange.com/questions/23068228/Modify-Python-Script-for-more-automation.html with great success. Only now it doesn't work and I am unsure why.  The only thing that has changed is the length of the digit after the diretory names from 2 to 3 (i.e. 01 is now 001) I believe I have updated the code accordingly to account for this change. I also changed the back slashes to forwards slashes. The original code is in the snippet box.

I get this error message when trying to execute the script:

C:\Documents and Settings\user>c:\python24\python "c:\new_donut.py"
Traceback (most recent call last):
  File "c:\new_donut.py", line 51, in ?
    for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
  File "c:\new_donut.py", line 40, in allLandcoverDonutAndIdSuffix
    assert os.path.isdir(DnumDir)
AssertionError

----

The only output I get in my "tester.txt" is "3"
My Directory has

C:\Giswork\landcoverprep\Donut
C:\Giswork\landcoverprep\LandcoverI
C:\Giswork\landcoverprep\LandcoverL
C:\Giswork\landcoverprep\Results

Donut has folders named "donut_001 - donut_407, donut_431, donut_435, donut_440, donut_447, donut_448

LandcoverI has the same numbering (i.e. LandcoverI_001 - 407, 431, 435, 440, 447)

LandcoverL (is missing about 10)

I also tried not including LandcoverL to the same error message.

Please advise,
Thanks,
MJ


# Multi_Output_Map_Algebra_sample.py
# Description:
#   Runs an expression built with the Map Algebra language.
# Requirements: None
 
import arcgisscripting
import glob
import os
 
gisworkDir = 'c:\\Giswork\\landcoverprep'
 
def log(s):
    f = open("C:/tester.txt","w")
    f.write(s)
    f.close()
 
 
def allLandcoverDonutAndIdSuffix(gisworkDir):
    '''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
    log("1\n")
    # Prepare the path to the donuts.
    donutDir = os.path.join(gisworkDir, 'Donut')
    assert os.path.isdir(donutDir)
    log("2\n")
    # Iterate through sequence of top-level landcover subdirectories.
    mask = os.path.normpath(os.path.join(gisworkDir, 'landcover*'))
    log(mask)            
    for LDir in glob.glob(mask):      # full paths to LandcoverA, LandcoverB, etc.
        subdir = os.path.basename(LDir)
        log("3\n")
        # Iterate through all numbered landcover subdirs inside.
        mask2 = os.path.join(LDir, '*')
        for LnumDir in glob.glob(mask2):
            # Separate the counter part, the idSuffix and construct
            # the related donut numbered directory name.
            numSuffix = LnumDir[-3:]   # last 3 chars
            idSuffix = LnumDir[-5:]    # last 5 chars
            DnumDir = os.path.normpath(
                         os.path.join(donutDir, 'DONUT_' + numSuffix))
            assert os.path.isdir(DnumDir)  
            log("4/n")
            # Generate another output tuple.
            yield (LnumDir, DnumDir, idSuffix)432
 
 
# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")
 
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
    # Construct the expression and the output directory name.
    outDir = os.path.join(gisworkDir, 'Results', idSuffix)
    log("\n" + outDir)
    InExpression = expTemplate % (donut, landcover, landcover)
    log("  " + InExpression)
 
    # Check out Spatial Analyst extension license
    gp.CheckOutExtension("Spatial")
 
    # Process: MapAlgebra
    gp.SingleOutputMapAlgebra_sa(InExpression, outDir)
 
    log("Msg: " + gp.GetMessages())
# Multi_Output_Map_Algebra_sample.py
# Description:
#   Runs an expression built with the Map Algebra language.
# Requirements: None
 
import pdb
import arcgisscripting
import glob
import os
 
gisworkDir = 'c:\Giswork'
 
def log(s):
    f = open("C:/tester.txt","w")
    f.write(s)
    f.close()
 
 
def allLandcoverDonutAndIdSuffix(gisworkDir):
    '''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
    ##log("1\n")
    # Prepare the path to the donuts.
    donutDir = os.path.join(gisworkDir, 'landcoverprep', 'Donut')
    assert os.path.isdir(donutDir)
    ##log("2\n")
    # Iterate through sequence of top-level landcover subdirectories.
    mask = os.path.normpath(os.path.join(gisworkDir, 'Landcoverprep\landcover*'))
 
    ##log(mask)             
    for LDir in glob.glob(mask):      # full paths to LandcoverA, LandcoverB, etc.
        subdir = os.path.basename(LDir)
        ##log("3\n")
        # Iterate through all numbered landcover subdirs inside.
        mask2 = os.path.join(LDir, '*')
#        pdb.set_trace()
        for LnumDir in glob.glob(mask2):
 
            # Separate the counter part, the idSuffix and construct 
            # the related donut numbered directory name.
            numSuffix = LnumDir[-2:]   # last 2 chars
            idSuffix = LnumDir[-4:]    # last 4 chars
            DnumDir = os.path.normpath(
                         os.path.join(donutDir, 'DONUT_' + numSuffix))
            assert os.path.isdir(DnumDir)  
            ##log("4")
            # Generate another output tuple.
            yield (LnumDir, DnumDir, idSuffix)
 
 
# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")
 
#pdb.set_trace()
 
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
    print (landcover,donut,idSuffix)
    pdb.set_trace()
    # Construct the expression and the output directory name.
    outDir = os.path.join(gisworkDir, 'landcoverprep', 'Results', idSuffix)
    log("\n" + outDir)
    InExpression = expTemplate % (donut, landcover, landcover)
    log("  " + InExpression)
 
    # Check out Spatial Analyst extension license
    gp.CheckOutExtension("Spatial")
 
    # Process: MapAlgebra
    gp.SingleOutputMapAlgebra_sa(InExpression, outDir)
 
    log("Msg: " + gp.GetMessages())

Open in new window

Avatar of mgjust
mgjust

ASKER

Hello,
Using the python script new_donut.py and my old files with only two digits at the end I get only the final output (in this test case H_71). Which was a problem I had initially with this script back in January.

I had a donut and a landcoverH directory for this trial
i.e. C:\Giswork\landcoverprep\LandcoverA\landcoverA_01 & C:\Giswork\landcoverprep\donut\donut_01

output from log file:



Please advise,
MJ
# Multi_Output_Map_Algebra_sample.py
# Description:
#   Runs an expression built with the Map Algebra language.
# Requirements: None
 
import arcgisscripting
import glob
import os
 
gisworkDir = 'c:/Giswork/landcoverprep'
 
def log(s):
    f = open("C:/tester.txt","w")
    f.write(s)
    f.close()
 
 
def allLandcoverDonutAndIdSuffix(gisworkDir):
    '''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
    log("1\n")
    # Prepare the path to the donuts.
    donutDir = os.path.join(gisworkDir, 'Donut')
    assert os.path.isdir(donutDir)
    log("2\n")
    # Iterate through sequence of top-level landcover subdirectories.
    mask = os.path.normpath(os.path.join(gisworkDir, 'landcover*'))
    log(mask)             
    for LDir in glob.glob(mask):      # full paths to LandcoverA, LandcoverB, etc.
        subdir = os.path.basename(LDir)
        log("3\n")
        # Iterate through all numbered landcover subdirs inside.
        mask2 = os.path.join(LDir, '*')
        for LnumDir in glob.glob(mask2):
            # Separate the counter part, the idSuffix and construct 
            # the related donut numbered directory name.
            numSuffix = LnumDir[-3:]   # last 3 chars
            idSuffix = LnumDir[-5:]    # last 5 chars
            DnumDir = os.path.normpath(
                         os.path.join(donutDir, 'DONUT_' + numSuffix))
            assert os.path.isdir(DnumDir)  
            log("4/n")
            # Generate another output tuple.
            yield (LnumDir, DnumDir, idSuffix)
 
 
# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")
 
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
    # Construct the expression and the output directory name.
    outDir = os.path.join(gisworkDir, 'Results', idSuffix)
    log("\n" + outDir)
    InExpression = expTemplate % (donut, landcover, landcover)
    log("  " + InExpression)
 
    # Check out Spatial Analyst extension license
    gp.CheckOutExtension("Spatial")
 
    # Process: MapAlgebra
    gp.SingleOutputMapAlgebra_sa(InExpression, outDir)
 
    log("Msg: " + gp.GetMessages())

Open in new window

tester1.txt
Avatar of mgjust

ASKER

Using the code from the original posting:
https://www.experts-exchange.com/questions/23068228/Modify-Python-Script-for-more-automation.html

and my old files with two digits the code works.
# Multi_Output_Map_Algebra_sample.py
# Description:
#   Runs an expression built with the Map Algebra language.
# Requirements: None
 
import arcgisscripting
import glob
import os
 
gisworkDir = 'c:\Giswork'
 
def log(s):
    f = open("C:/tester_new.txt","w")
    f.write(s)
    f.close()
 
 
def allLandcoverDonutAndIdSuffix(gisworkDir):
    '''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
    ##log("1\n")
    # Prepare the path to the donuts.
    donutDir = os.path.join(gisworkDir, 'landcoverprep', 'Donut')
    assert os.path.isdir(donutDir)
    ##log("2\n")
    # Iterate through sequence of top-level landcover subdirectories.
    mask = os.path.normpath(os.path.join(gisworkDir, 'Landcoverprep\landcover*'))
    ##log(mask)             
    for LDir in glob.glob(mask):      # full paths to LandcoverA, LandcoverB, etc.
        subdir = os.path.basename(LDir)
        ##log("3\n")
        # Iterate through all numbered landcover subdirs inside.
        mask2 = os.path.join(LDir, '*')
        for LnumDir in glob.glob(mask2):
            # Separate the counter part, the idSuffix and construct 
            # the related donut numbered directory name.
            numSuffix = LnumDir[-2:]   # last 2 chars
            idSuffix = LnumDir[-4:]    # last 4 chars
            DnumDir = os.path.normpath(
                         os.path.join(donutDir, 'DONUT_' + numSuffix))
            assert os.path.isdir(DnumDir)  
            ##log("4")
            # Generate another output tuple.
            yield (LnumDir, DnumDir, idSuffix)
 
 
# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")
 
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
    # Construct the expression and the output directory name.
    outDir = os.path.join(gisworkDir, 'landcoverprep', 'Results', idSuffix)
    log("\n" + outDir)
    InExpression = expTemplate % (donut, landcover, landcover)
    log("  " + InExpression)
 
    # Check out Spatial Analyst extension license
    gp.CheckOutExtension("Spatial")
 
    # Process: MapAlgebra
    gp.SingleOutputMapAlgebra_sa(InExpression, outDir)
 
    log("Msg: " + gp.GetMessages())

Open in new window

Avatar of mgjust

ASKER

Then using the script above that worked and changing :

numSuffix = LnumDir[-3:]   # last 2 chars
            idSuffix = LnumDir[-5:]    # last 4 chars

from -2 and -4

& and i changed all the 'landcoverprep' to 'landcoverprepnew'

I get the following error:

C:\Documents and Settings\user>c:\python24\python "c:\new2.py"
Traceback (most recent call last):
  File "c:\new2.py", line 51, in ?
    for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
  File "c:\new2.py", line 40, in allLandcoverDonutAndIdSuffix
    assert os.path.isdir(DnumDir)
AssertionError

Please advise,
MJ
# Multi_Output_Map_Algebra_sample.py
# Description:
#   Runs an expression built with the Map Algebra language.
# Requirements: None
 
import arcgisscripting
import glob
import os
 
gisworkDir = 'c:\Giswork'
 
def log(s):
    f = open("C:/tester.txt","w")
    f.write(s)
    f.close()
 
 
def allLandcoverDonutAndIdSuffix(gisworkDir):
    '''Generates all tuples (landcover, donut, idSuffix) from the gisworkDir.'''
    ##log("1\n")
    # Prepare the path to the donuts.
    donutDir = os.path.join(gisworkDir, 'landcoverprepnew', 'Donut')
    assert os.path.isdir(donutDir)
    ##log("2\n")
    # Iterate through sequence of top-level landcover subdirectories.
    mask = os.path.normpath(os.path.join(gisworkDir, 'Landcoverprepnew\landcover*'))
    ##log(mask)             
    for LDir in glob.glob(mask):      # full paths to LandcoverA, LandcoverB, etc.
        subdir = os.path.basename(LDir)
        ##log("3\n")
        # Iterate through all numbered landcover subdirs inside.
        mask2 = os.path.join(LDir, '*')
        for LnumDir in glob.glob(mask2):
            # Separate the counter part, the idSuffix and construct 
            # the related donut numbered directory name.
            numSuffix = LnumDir[-3:]   # last 2 chars
            idSuffix = LnumDir[-5:]    # last 4 chars
            DnumDir = os.path.normpath(
                         os.path.join(donutDir, 'DONUT_' + numSuffix))
            assert os.path.isdir(DnumDir)  
            ##log("4/n")
            # Generate another output tuple.
            yield (LnumDir, DnumDir, idSuffix)
 
 
# Create the Geoprocessor object
gp = arcgisscripting.create()
expTemplate = 'CON(ISNULL(%s), %s, %s * -1)'
log("\n===== Output below: ====\n")
 
for landcover, donut, idSuffix in allLandcoverDonutAndIdSuffix(gisworkDir):
    # Construct the expression and the output directory name.
    outDir = os.path.join(gisworkDir, 'landcoverprepnew', 'Results', idSuffix)
    log("\n" + outDir)
    InExpression = expTemplate % (donut, landcover, landcover)
    log("  " + InExpression)
 
    # Check out Spatial Analyst extension license
    gp.CheckOutExtension("Spatial")
 
    # Process: MapAlgebra
    gp.SingleOutputMapAlgebra_sa(InExpression, outDir)
 
    log("Msg: " + gp.GetMessages())

Open in new window

Avatar of pepr
This is why the asserts are good friends. Something have changed and it is not reflected in the code. Ift the command failed...

assert os.path.isdir(DnumDir)

it says thatSays that DnumDir does not exist. As its name is constructed earlier from 2-digits number and now from 3-digits number. If I recall correctly, the DONUT_xx directories are only loosely coupled with the LandcoverN_xx directories. The question now is...

Do you want to protect DONUT_xx (with 2 digits) also for future? Do you want to have both 2digits and 3digits names? How should it behave if the directories are mixed? If you want to have, say, DONUT_12 and DONUT_012, what is preferred?
Avatar of mgjust

ASKER

Pepr:
From now on all Landcover and Donut directories have three digits (donut_xxx & landcover*_xxx) and will not be mixed. Yes, the same donut (e.g. donut_123) is used for all landcovers (e.g. LandcoverA_123, LandcoverB_123, etc).  How come changing the

numSuffix = LnumDir[-3:]   # last 2 chars
idSuffix = LnumDir[-5:]    # last 4 chars

did not solve the problem?

Finally, does it work by searching for those digits? Only using Donut_001 with Landcover*_001, or if there was no landcover*_001 would Donut_001 be matched with Landcover*_002? If this last piece is more complicated, then I will just use place holder directories to circumvent this, because the automation with 3 digits is what I am really eager for.

Thanks again,
MJ

SOLUTION
Avatar of Duncan Roe
Duncan Roe
Flag of Australia image

Link to home
membership
This content is only available to members.
To access this content, you must be a member of Experts Exchange.
Get Access
Avatar of mgjust

ASKER

duncan_roe:
Thanks for the advice,
As posted above I did know the mods between the old and the new. Perhaps my question title "Python script no longer works - only minor change made" was a bit sophomoric but was hoping that it would produce results. I will look into these programs you suggested as I can see there usefulness. On the plus side there is no management to convince just myself.  And if you have any suggestions about fixing this script I'd also like to hear it.

Thanks again,
MJ
ASKER CERTIFIED SOLUTION
Link to home
membership
This content is only available to members.
To access this content, you must be a member of Experts Exchange.
Get Access
Avatar of mgjust

ASKER

pepr:

Sorry for the confusion but the script with the added lines of code works as wanted.

Thanks again for your patience and answers.

Cheers,
MJ
Avatar of mgjust

ASKER

Thank you.
It appears the issue has returned.  I have posted a new question at:

https://www.experts-exchange.com/questions/23993825/Python-Script-Assertion-Error.html

Thanks,
JE