Confusion of scope of python function

I have a list
mylist = ['1008', [27, 29, 30, 31, 32, 33, 34], [27, 28, 30, 32, 33, 37], [27, 28, 29, 30, 31, 32, 33, 34, 37]]

which is operated on by a function euro_activityCnt(mylist,total_euro=0,activityCount=0). This operation changes the value of mylist within the scope of the function - however the function does not return mylist.

However, when I use the function, I find that it creates a new  value for mylist. This confuses me on two levels

1   I always thought that if a function does not return a variable, then that variable would remain unaltered outside the scope of the function.

2 Given that this is possible, why does the function not return the last value of mylist which it created within its scope. It returns
   mylist = [[27, 29, 30, 31, 32, 33, 34], [27, 28, 30, 32, 33, 37], [27, 28, 29, 30, 31, 32, 33, 34, 37]]. The last value of mylist the function created within its scope was
   mylist = [27, 29, 30, 31, 32, 33, 34, 27, 28, 30, 32, 33, 37, 27, 28, 29, 30, 31, 32, 33, 34, 37]   (see line 65 of attached code)

Below is the code containing the function in its enviornment

Thanks for clearing up my confusion

james

#!C:\Python34\python.exe
import collections
import cgi,cgitb
import mysql.connector as conn
def htmlTop():
    print("""Content-type:text/html\n\n
        <!DOCTYPE html>
        <html lang="en">
            <head>
                <meta http-equiv="Content-Type" content="text/html;charset=iso-8859-1">
                <title> My Server-side template</title>
        </head>
        <body>""")

def htmlTail():
    print(""" its done</body>
        </html>""")

def connectDB():
    db=conn.connect(host='localhost' ,user='root' ,passwd='844cheminduplan' ,db='office')
    cursor = db.cursor()
    return db, cursor 

def activityinfo(db,cursor,courseNumber):
    cursor.execute("""select courseNumber,Activity, tariffstwelve, tariffsthirtysix from activities WHERE courseNumber = "%s" """
    %(courseNumber)) 
    activityinfo=cursor.fetchall()
    return activityinfo



def Turn_On_Alibe(db,cursor):
    cursor.execute ("""
       UPDATE registrations
       SET  ALPAID_ytd=1
       WHERE memberID={0}
    """.format(mylist[0]))
    db.commit()
    db.close

def Turn_Off_Alibe(db,cursor):
    cursor.execute ("""
       UPDATE registrations
       SET  ALPAID_ytd=0
       WHERE memberID={0}
    """.format(mylist[0]))
    db.commit()
    db.close
    
def SelectALPAID_ytd(db,cursor):
    cursor.execute("""select ALPAID_ytd from registrations WHERE memberID = "%s" """
    %(memberID)) 
    ALPAID_ytd=cursor.fetchall()
    return ALPAID_ytd

def SelectActivityLibre(db,cursor):
    cursor.execute ("""SELECT activity_libre from fees""")
    activity_libre=cursor.fetchall()
    return activity_libre


def euro_activityCnt(mylist,total_euro = 0,activityCount = 0):
    del mylist[0]
    print(mylist, "inside function after deleting mylist[0]")
    mylist = mylist[0] + mylist[1]  + mylist[2]
    print(mylist, "inside function ..this is my list_FLAT")
    count = collections.Counter(mylist)
    #print(count, "this is the counter")
    for each in count:
        del mylist[0]
        courseNumber = each
        activityCount = activityCount + count[each]
        activityinfo(db,cursor,courseNumber)
        activitydata = activityinfo(db,cursor,courseNumber)
        #print(activitydata, "this is activity data")
        if count[each] != 3:
            total_euro = total_euro + count[each]*activitydata[0][2]
        else:
            total_euro= total_euro + activitydata[0][3]
    return total_euro,activityCount,count

#main program
if __name__== "__main__":
    try:
        htmlTop()
        mylist = ['1008', [27, 29, 30, 31, 32, 33, 34], [27, 28, 30, 32, 33, 37], [27, 28, 29, 30, 31, 32, 33, 34, 37]]
        db,cursor=connectDB()
        total_euro,activityCount,count = euro_activityCnt(mylist,total_euro=0,activityCount=0)
        print(mylist, "outside scope of function this is mylist ??")
        print(total_euro, "this is total euro", activityCount, "this is total activity count", count, "this is counter")
        memberID = 1008
        ALPAID_ytd = SelectALPAID_ytd(db,cursor)
        ALPAID_ytd =ALPAID_ytd[0][0]
        print(ALPAID_ytd, "this is ALPAID_ytd", mylist[0], "this is memberID")
        activity_libre = SelectActivityLibre(db,cursor)
        activity_libre = activity_libre[0][0]
        print(activity_libre, "activity_libre")
       
        #Turn_Off_Alibe(db,cursor)
        
        if ALPAID_ytd == 0:
            print(ALPAID_ytd, "This is ALPAID_ytd")
            db,cursor=connectDB()
            memberID = 1008
            Turn_On_Alibe(db,cursor)
            
            
          
        ALPAID_ytd = SelectALPAID_ytd(db,cursor)
        print(ALPAID_ytd, "this is ALPAID without trim")
        ALPAID_ytd = ALPAID_ytd[0][0]
        print(ALPAID_ytd, "this is ALPAID WITH trim")
        
        
        
        
        htmlTail()
    except:
        cgi.print_exception()

Open in new window

jameskaneAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Walter RitzelSenior Software EngineerCommented:
The short answer for your questions is this:
All parameters (arguments) in the Python language are passed by reference. It means if you change what a parameter refers to within a function, the change also reflects back in the calling function.

So, when you do this:
 mylist = mylist[0] + mylist[1]  + mylist[2]

you have in fact, modified the initial parameter sent to your function.

If you want to avoid that, use a different variable inside the function.
0
peprCommented:
When passing arguments to a Python function, you always pass references to the target object. The object is not copied -- just the reference is. Here the mylist inside the function is just internal name of the reference inside the function (one more reference now). The del mylist[0] at line 63modifies the object that lives outside.

However, the next mylist[0]  + mylist[1]  + mylist[2] generates another list (new one) with elements from lists at indices. When assigning to the mylist name at the line 65, the name is reused as a name for the new list. (The mylist variable stores the reference to the new list. The older list is forgotten, and it is referenced only from outside the function; however, it was modified by the first del.) From now on, you modify the new list, not the passed one.

The del mylist[0] at the line 70 now deletes the items from the new list. I did not try hard to understand the code, so I cannot comment further ;)

In Python, when you want to return the argument, you should use explicit return mylist and assign the function result to the variable that you want.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
jameskaneAuthor Commented:
Thanks very much for your replies - read them carefully - well as carefully as a beginner can !

Would  you agree then that the following are 'good practices' to avoid confusion when using functions ?

1... always create a duplicate of the argument within the function and return the duplicate name.  This worked to preserve the original argument as per attached example

or

2... don't worry about creating a duplicate - just don't RETURN the argument  from the function - assuming that you don't have the objective to change the argument externally to the function in the first place. This worked in the attached example

Would there be any other 'good practices' ?  

If you have any pointers to discussion documents on this, would appreciate it.

AGAIN,  very many thanks for taking the time !!

James

import collections

mylist = ['1008', [27, 29, 30, 31, 32, 33, 34], [27, 28, 30, 32, 33, 37], [27, 28, 29, 30, 31, 32, 33, 34, 37]]

print("---------------------make a copy of mylist in the function---------------------------")

def testing_correct(mylist):
    
    mylist2 = mylist[:]
    del mylist2[0]
    mylist2 = mylist2[0] + mylist2[1]  + mylist2[2]
    count = collections.Counter(mylist2)
    return count,mylist2 

mylist2,count = testing_correct(mylist)
print (count, "this is the count")
print (mylist2,"mylist2")
print (mylist,"mylist - the original list")

 
print("-------------------------don't make a copy of mylist, but do not return it--------------------------------")

    
def testing_wrong(mylist):
    del mylist[0]
    mylist = mylist[0] + mylist[1]  + mylist[2]
    count = collections.Counter(mylist)
    return count 

mylist2,count = testing_correct(mylist)
print (count, "this is the count")
#print (mylist2,"mylist2")
print (mylist,"mylist - the original")

Open in new window

import collections

mylist = ['1008', [27, 29, 30, 31, 32, 33, 34], [27, 28, 30, 32, 33, 37], [27, 28, 29, 30, 31, 32, 33, 34, 37]]

print("-----------------------------------------------------------------------------------")

def testing_correct(mylist):
    
    mylist2 = mylist[:]
    del mylist2[0]
    mylist2 = mylist2[0] + mylist2[1]  + mylist2[2]
    count = collections.Counter(mylist2)
    return count,mylist2 

mylist2,count = testing_correct(mylist)
print (count, "this is the count")
print (mylist2,"mylist2")
print (mylist,"mylist - the original list")


print("--------------------------------------------------------------------------------------------")

    
def testing_wrong(mylist):
    del mylist[0]
    mylist = mylist[0] + mylist[1]  + mylist[2]
    count = collections.Counter(mylist)
    return count 

mylist2,count = testing_correct(mylist)
print (count, "this is the count")
#print (mylist2,"mylist2")
print (mylist,"mylist - the original")

Open in new window

0
peprCommented:
Here is how I would write it:
#!python3

import collections

mylist = [
    '1008',
    [27, 29, 30, 31, 32, 33, 34],
    [27, 28, 30, 32, 33, 37],
    [27, 28, 29, 30, 31, 32, 33, 34, 37]
]

def testing_correct2(mylist):
    mylist = mylist[1] + mylist[2]  + mylist[3]
    count = collections.Counter(mylist)
    return mylist, count  

mylist2, count = testing_correct2(mylist)
print('Original:', mylist)
print('Returned:', mylist2)
print('Counter:', count)

Open in new window

Generally (in theory, if you cannot decide, or if possible), a pure function should not have a side effects. This means it should not modify anything passed inside. The only thing it should do is to return calculated value(s). (This does not hold for member functions also known as methods of a class. They typically modify member variables.) As a rule of thumb, a function should not print (if it is not designed to print -- tradeoffs everywhere).

If you want to write an efficient code, you should know how it works inside. In this case, if you do not want to modify the original list, then do not do it. If you want to modify it, then return it to make this clearer.

If you always duplicate (copy) an object that is passed in, you spend time and memory to do that. So, doing it always, without thinking, is not a good thing. Moreover, in Python, you do a shallow copy only. In other word, if the code would modify say mylist2[1] (that is the sublist from the copied list), it would modify also the original, because the sublist is shared by both the original and the copy.

Your testing_correct() returns the values in a different order than you assign.

From time to time, it is a good thing to read again PEP 0008 -- Style Guide for Python Code.

For understanding Python variables (the idea slightly differs from compiled languages like C++ or Pascal, but also from C# and Java), the article Python illustrated (part 3) may help you.
1
jameskaneAuthor Commented:
Thanks for the input guys. Your info and some googling got me to where I now understand much better.

THANKS
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Python

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.