How to create a menu Python script that runs a few other Python scripts/programs?

import subprocess

cmd = "c:\\file.exe"
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, creationflags=0x08000000)
process.wait()

Select allOpen in new window

I wouldn't reinvent the wheel and execute the SAS Programs rather than attempt to rewrite the SAS program in python.

@David: your syntax won't work as \\ are not valid in pathnames on Unix/Linux

Note: There are no efficiency gains from translating the menu & scripts from bash in python.

Also it doesn't matter in what language each program has been written

os.system()   should be sufficient.

#!env python
import os
os.system("some_command with arguments")

Select allOpen in new window

See also: https://docs.python.org/3/library/os.html

Sub process management can also be used then look into this
https://docs.python.org/3/library/subprocess.html

Thank you, David! 

Unfortunately, SAS is no longer available for this project and the only option is Python. 

The example with bash/SAS was something similar like the:

#1. Menu bash script (menu_script.sh) 

export DIR1=/root/executables

while true
do
clear
echo "Welcome to Files Creation System"
echo "            Control Menu"
echo " "
echo "Please make your selection from the following selections"  
echo "Selection 1: Reading and Verifying Data Files"
echo "Selection 2: Merging Data Files and Creating Final Output Files"
echo "Selection X: Exit System"
echo -e "\n"
echo -e "*** Make a selection from the Control Menu above and press enter ***"
read select

# If selection by a user is invalid
select=$(echo $select | tr '[:lower:]' '[:upper:]')
if [[ (("$select" < "1") || ("$select" > "2")) && ("$select" != "X") ]];then
   echo "Selection is invalid... try again"
   echo " --- Exiting Files Creation System Control Menu" 
   exit
fi
 
# If Selection X: Exit System
if [ "$select" = "X" ]; then
   echo "*** Exit Files Creation System Control Menu***"
   exit
fi

# Selection 1: Reading and Verifying Data Files
echo "select: $select"
if [ "$select" = "1" ]; then
   echo "*** Executing Menu Selection 1: Reading and Verifying Data Files ***"
   ${DIR1}/Selection1_script.sh
   sleep 6
fi

# Selection 2: Reading and Verifying Data Files
echo "select: $select"
if [ "$select" = "2" ]; then
   echo "*** Executing Menu Selection 2: Merging Data Files and Creating Final Output Files ***"
   ${DIR1}/Selection2_script.sh
   sleep 6
fi
done

# End process

Select allOpen in new window

#2. Smaller Bash script (Selection1_script.sh - Selection2_script.sh would be very similar) - initiated by the menu_script.sh

export DIR1=/root/executables
export DIR2=/root/logs
export proglog=${DIR2}/masterlogs.log
export mailist=${DIR2}/mail_list.txt     #this file has list of email addresses

#  Creating Time and Date
 date=`date '+%d-%b-%Y'`
 time=`date '+%H:%M:%S'`
 datemm=`date '+%Y-%m-%d'`
 datemmdd=`date '+%m%d'`

# Set email variables
export mailsubj=" "
statmsg="Success"
altmsg=" "
mailaddr=`<$maillist`

#  Execute Reading_and_Verifying_Data_Files.sas
echo "--- Run Reading_and_Verifying_Data_Files.sas" >> $proglog
sas ${DIR1}/Reading_and_Verifying_Data_Files.sas -sortpgm sas -sortsize 0 -log
 ${DIR2}/Reading_and_Verifying_Data_Files.log -print ${DIR2}/Reading_and_Verifying_Data_Files.lis
    ERRSTAT=$?
    echo "ERRSTAT:" "$ERRSTAT"

#  Check for Errors in SAS
 if [[ $ERRSTAT -ne  0 ]]; then
        echo " *** Process Reading_and_Verifying_Data_Files.sas ==> FAILED – check SAS log *** "
        echo " Date: $date                 Time: $time"
              echo  "*** Process Reading_and_Verifying_Data_Files.sas  ==> FAILED – check SAS log *** " >> $proglog
 
     mailsubj="ERROR!! Selection1:  Reading_and_Verifying_Data_Files.sas ==> FAILED - Check SAS log"
     mailx -a "${DIR2}/masterlogs.log"  -s  "$mailsubj" "$mailaddr"
  exit 1
fi
 
  if [ $ERRSTAT = 0 ]; then
echo " *** Process Reading_and_Verifying_Data_Files.sas == SUCCESS!! *** "
echo " Date: $date                 Time: $time" >> $proglog
 echo "*** SUCCESS!! Selection1:  Reading_and_Verifying_Data_Files.sas ==> COMPLETE ***"  >> $proglog
 mailsubj="SUCCESS!! Selection1:  Reading_and_Verifying_Data_Files.sas ==> COMPLETE"
 mailx -s "$mailsubj" "$mailaddr" < /dev/null
 fi
 exit 0
 
# End Process

Select allOpen in new window

#3. SAS program (Reading_and_Verifying_Data_Files.sas) - run by the Selection1_script.sh

This program basicly has different processes in SAS to include data processing/calculations/manipulations. 

So, this last part #3 I don't really worry about since this would be just replaced Python data processing/calculations/manipulations. 

If anything needs to be clarified then please let me know.  

Can above bash/SAS processes with similar logic be created in Python scripting/programing? If so, how would it look like?  

Thank you!

Hi Noci! It's good to hear from you!! I think last time you were helping me with bash/SAS programming problems a few years back (way before covid). 

So, the long story short. For processing large data, unfortunately, I will not Bolger be able  to use SAS. Pretty much SAS no longer will be available and Python would be the language that would replace all SAS processing. But Unix bash on Linux will still be available to use.  

Back to my above question or bash/SAS examples, would you suggest just leaving bash scripts in place and just run Python programs instead of SAS programs?  Would there be a major change in how bash runs Python programs? 

Thank you and again it is really nice to hear from you! 

Thanks, Noci!

So, basically changing from

sas ${DIR1}/Reading_and_Verifying_Data_Files.sas -sortpgm sas -sortsize 0 -log

Select allOpen in new window

to

python3 ${DIR1}/Reading_and_Verifying_Data_Files.py 

Select allOpen in new window

should be workin? 

OK, by any chance, do you know if bash scripting will also catch error messages in Python as it was keying in SAS (line 20 through 43 in the code)? 

Thank you, Noci!

Should I use just one of those options or multiple options?  And how would I know/find out if that the script catches the errors? Would the errors show only on the screen or also captured in some .log file as it was done in my bash/SAS example?  

I was testing this bash script with a Python program today and it seams to go through without any issues but when I make a mistake in the Python program on purpose - I can only see error messages on the screen on the command line but those error messages looks like not captured anywhere else.   

export DIR1=/root/executables
python3 ${DIR1}/Reading_and_Verifying_Data_Files.py

# OUTPUT variable contains all STDOUT output, STDERR to terminal
OUTPUT=$(python3 ${DIR1}/Reading_and_Verifying_Data_Files.py)       

# OUTPUT variable contains all STDOUT output, STDERR suppressed
OUTPUT=$(python3 ${DIR1}/Reading_and_Verifying_Data_Files.py  2>/dev/null)       

#  OUTPUT  variable contains STDOUT + STDERR
OUTPUT=$(python3 ${DIR1}/Reading_and_Verifying_Data_Files.py 2>&1)        

 #  OUTPUT  variable contains STDERR., STDOUT = suppressed
OUTPUT=$(python3 ${DIR1}/Reading_and_Verifying_Data_Files.py 2>&1  >/dev/null)       

Select allOpen in new window

You can choose the one you like most.

normally input is read from STDIN (fd=0), OUTPUT (in pipes etc.) is sent to STDOUT (fd=1) and Errormessages are sent to STDERR (fd=2).

2>&1   means connect the fd=2 output to the same as fd=1.   

>some-file     means send the STDOUT (fd=1)  to some-file.    1>some-file does the same.

Order does matter, the first met is the first handled....

2>&1    >/dev/null      does first copy all STDERR to the Current STDOUT,  then STDOUT  is sent to the bit-bucket.  The new assignment to STDOUT doesn;t affect STDERR...

>/dev/null 2>&1     FIrst send all stdout to the bit-bucket, and then copies this to stderr as well ie. it is the same as
>/dev/null 2>/dev/null   (suppressing all output).

Specifying this 4 times means running the program 4 times.

You can view the content from OUTPUT using 

echo $OUTPUT

Select allOpen in new window

All the output should be shown in example #3. 

Great. Thank you for your suggestions and examples!