Python csv errors using Python 2.3 for code written with Python 2.2

Posted on 2004-08-06
Medium Priority
Last Modified: 2007-12-19
I am a new systems administrator (new to my company and new to the field) trying to get some Python code to work- that the previous systems administrator had written.  He used Linux with Python 2.2, importing a csv module to run his code.  I am now trying to run this code on a Macintosh iBook, running Panther (Mac OS 10.3.4) which comes bundled with its own csv module.  

I am getting the following errors when I try to run his code.  (Apparently it used to work fine when he ran it from his laptop on Linux- using Python 2.2)

This code is supposed to mount users' C drives and search for .pst files ( and other files ) and print them out. (The code looks like it may also search for media files and delete non-work related files).  This project came up because my boss would like to know where all the .pst files are (on the Windows NT network that is mounted to our Solaris servers using Samba) Plus he wanted to be able to use the same tools the other administrator used, but  for some reason, I can't get it to run.  

I realize that without being able to connect to my NT network, it will be impossible for anyone to duplicate my exact errors, but thought that someone may have insight into what I can change in this code to get it up and running.  If nothing else, I would like a script to mount user's personal C: drives to search for .pst folders.  I can work on the extra functionality of this code at another time.  

Here are all of the pieces of code that are supposed to work together:

###pstfinder.py calls killjoy.py to find .pst files

import os
import commands
import sys
import csv
from killjoy import Killjoy, Scanner, Writer

class Mailjoy(Killjoy):
    credfilename = '/tmp/pstfinder.cred'
    outputfile = 'pstfinder.csv'

    def execute:
        pass #FIXME use derived Scanner class
class PSTScanner(Scanner):
    badexts = ['.pst']

When I run pstfinder.py I get the following errors:
keroleen:~/Desktop/python/killjoy juliann$ python pstfinder.py
  File "pstfinder.py", line 11
    def execute:
SyntaxError: invalid syntax

Here is the second half of the code that works with pstfinder.py

###killjoy.py looks for files (.pst files) and prints them out- plus does other clean up activity on non-work related files

#!/usr/bin/env python

import os
import commands
import sys
import csv # this uses the python 2.2 CSV module
import re
import threading
import Queue
import time
import popen2
import getopt
import stat
from usermapping import usermapping

class Killjoy:
    """Scan PCs for media files.  Obtains a list of clients from the domain controller and uses smbfs to mount the C$ share.  Writes output as a CSV file."""
    workdir = '/tmp/killjoy'
    domain = 'ICI'
    credentials = """username=Administrator\npassword=h0t^^s@nd\ndomain=ICI"""
    credfilename = '/tmp/killjoy.cred'
    dmb = 'ici-dc1'
    outputfile = 'killjoy.csv'
    numscanners = 10

    def __init__(self):
        """Make a working directory for mount points and write out a credentials file for mount.smbfs."""
        if not (os.path.isdir(self.workdir)):
        print "** writing credentials to %s" % self.credfilename
        creds = open(self.credfilename, 'w').write(self.credentials)

    def getmountpoint(self, client):
        return os.path.join(self.workdir, client)

    def rmcreds(self):
        print "** deleting credential file %s" % self.credfilename

    def getclientlist(self):
        print "** getting client list"
        command = 'smbclient -A %s -L //%s' % (self.credfilename, self.dmb)
        print "** running command: %s" % command
        status, output = commands.getstatusoutput(command)
        if status == 0:
            # special case for "NT-BACKUP"
            clients = [ client for client in output.split() if (client[:2] == 'NT' and client != 'NT-BACKUP') ]
            # add ICI3 to search profiles, handled as a special case in Scanner
            return clients
            print "** no client list:\n%s" % output

    def listnotscanned(self, clientlist):
        """Andy wants to know which machines were in Chester's list but not known by the DC."""
        chesterslist = usermapping.keys()
        missingpcs = [ pc for pc in chesterslist if pc not in clientlist ]
        return missingpcs

    def mount(self, client, driveletter='C'):
        mountpoint = self.getmountpoint(client)
        if not os.path.isdir(mountpoint):
        print "** mounting \\\\%s\\%s$ on %s" % (client, driveletter, mountpoint)
        command = 'mount -t smbfs -o credentials=%s //%s/%s\$ %s' % (self.credfilename, client, driveletter, mountpoint)
        status, output = commands.getstatusoutput(command)
        if status != 0:
            print "status: %d\noutput: %s" % (status, output)
            return None
            return mountpoint
    def umount(self, client):
        mountpoint = self.getmountpoint(client)
        print "** unmounting %s" % mountpoint
        command = 'umount %s' % mountpoint
        status, output = commands.getstatusoutput(command)
        if status != 0:
            print "** command failed. status: %d\noutput: %s" % (status, output)

    def mountall(self, clientlist):
        mounted = []
        notmounted = []
        for client in clientlist:
            driveletter = 'C'
            if client.lower() == 'ici3':
                driveletter = 'J' # search profiles
            if self.mount(client, driveletter):

        return mounted, notmounted

    def umountall(self, clientlist):
        for client in clientlist:

    def execute(self, clientlist, delete=False):
        writequeue = Queue.Queue(0)
        scanqueue = Queue.Queue(0)

        Writer(writequeue, self.outputfile, delete).start()
        for i in range(self.numscanners):
            Scanner(scanqueue, writequeue, self.workdir, self.credfilename, delete).start()

        mounted, notmounted = self.mountall(clientlist)

        for client in mounted:

        for client in notmounted:
            writequeue.put([{'filename':'SCAN FAILED','client':client,'date':'N/A'}])

        notscanned = self.listnotscanned(clientlist)
        for client in notscanned:
            writequeue.put([{'filename':'NOT SCANNED','client':client,'date':'N/A'}])

        # clean up
        for i in range(self.numscanners):

        while True:
            if scanqueue.empty():

class Scanner(threading.Thread):
    # these are compiled into a big regular expression
    innocuous = ['clock\.avi', 'cmos\.ram', 'dna connected\.avi', 'dna searching\.avi', 'sample\.mov', 'findfile\.avi', 'Temporary Internet Files', 'Microsoft Visual Studio', 'RealPlayer', 'WindowsMediaPlayer','Netscape.Users.*Cache']
    #badexts = ['.mp3','.mpg','.mpeg','.wma','.wmv','.mov']
    badexts = ['.pst']
    naughtyfiles = {} # a dict of client:[badfiles]

    def __init__(self, scanqueue, writequeue, workdir, credfilename, delete=False):
        self.__scanqueue = scanqueue
        self.__writequeue = writequeue
        self.credfilename = credfilename
        self.workdir = workdir
        self.delete = delete

        # charge up the innocuous file filter
        reify = [ '.*' + item  for item in self.innocuous ]
        self.ok = re.compile('|'.join(reify), re.IGNORECASE)

    def run(self):
        while True:
            client = self.__scanqueue.get()
            if client is None:
                print "* shutting down thread %s" % self
                break # end of queue
                foundfiles = self.findnaughtyfiles(client)
                if foundfiles:
    def findnaughtyfiles(self, client):
        mountpoint = os.path.join(self.workdir, client)
        filelist = []
        print "** searching %s" % client
        def _visitfunc(filelist, dirname, names):
            for name in names:
                filename = os.path.join(dirname, name)
                if self.ok.match(filename):
                    # if it's ok, we don't care
                if os.path.isfile(filename):
                    ext = os.path.splitext(filename.lower())[1]
                    if ext in self.badexts:
                        if self.delete:
                                print "Deleted",
                            except OSError:
                                print "unable to delete %s" % filename
                        if client.lower() == 'ici3':
                            driveletter = 'J:' # profile drives
                            driveletter = 'C:'
                        mtime = os.stat(filename)[stat.ST_MTIME]
                        filedate = time.strftime('%m/%d/%Y', time.localtime(mtime))
                        filename = filename.replace(mountpoint, driveletter).replace('/','\\')
                  print "%s: %s" % (client, filename)

        os.path.walk(mountpoint, _visitfunc, filelist)
        print '** finished searching %s' % client
        if not filelist:
            filelist = [{'filename':'CLEAN','date':'N/A','client':client}]
        return filelist

class Writer(threading.Thread):
    from usermapping import usermapping
    def __init__(self, queue, outputfile, delete=False):
        self.__queue = queue
        print "** initializing output file %s" % outputfile
        self.fd = open(outputfile, 'w')
        self.parser = csv.parser()

        if delete:
            self.fd.write(self.parser.join(['The following files were deleted:']) + '\n')
        self.fd.write(self.parser.join(['Name','UID','PC','File','Date']) + '\n')

    def writeoutput(self, foundfiles):
        outcsv = []
        for file in foundfiles:
            pc = file['client'].upper()
                pcdata = usermapping[pc]
            except KeyError:
                # provide blank data if pc is not in index
                pcdata = {}
                for i in [ 'cn','uid']:

            cn = pcdata['cn']
            uid = pcdata['uid']

            outcsv.append(self.parser.join([cn, uid, pc, file['filename'], file['date']]))

        #print "** output written to %s" % self.outputfile

    def run(self):
        while True:
            item = self.__queue.get()
            if item is None:
                break # reached end of queue
            foundfiles = item # expand tuple
            if item is None:
                break # reached end of queue
            #print "* Writer: writing output for %s" % client
        print "* Writer: shutting down"

if __name__ == "__main__":
    usage = "Usage: %s [-r] clientlist\n\t-r: delete found files\n"
        opts, args = getopt.getopt(sys.argv[1:], 'r')
    except getopt.GetoptError, e:
        print usage

    delete = False # the default
    for o, v in opts:
        if o == '-r':
            delete = True

    clients = args

    kj = Killjoy()
    if not clients:
        clients = kj.getclientlist()

    kj.execute(clients, delete)

###Here is the (abbreviated) list of users:


usermapping  = {'NT1100': {'cn': 'Tara Meadows', 'uid': 'tmeadows'}, 'NT1101': {'cn': 'Chris Boyland', 'uid': 'cboyland'}, 'NT1108': {'cn': 'Agnes Thomas', 'uid': 'agnes'}, 'NT1109': {'cn': 'Keith Lawson', 'uid': 'lawson'}}

Here is the error message I get when I try to run killjoy.py:

keroleen:~/mbriggs_work/python/killjoy juliann$ python killjoy.py
** writing credentials to /tmp/killjoy.cred
** getting client list
** running command: smbclient -A /tmp/killjoy.cred -L //ici-dc1
** initializing output file killjoy.csv
Traceback (most recent call last):
  File "killjoy.py", line 270, in ?
    kj.execute(clients, delete)
  File "killjoy.py", line 106, in execute
    Writer(writequeue, self.outputfile, delete).start()
  File "killjoy.py", line 208, in __init__
    self.parser = csv.parser()
AttributeError: 'module' object has no attribute 'parser'

Question by:juli_ann
1 Comment

Accepted Solution

rjkimble earned 2000 total points
ID: 11737597
The execute method def statement is wrong. It should be:

    def execute():

Or something like that. My guess is that it's intended to have arguments. To be consistent with its ancestor (Killjoy), it should probably look something like this:

    def execute(self, clientlist, delete=False):

It appears to me that the Mailjoy class programming has not yet been completed.

There was a major change in the csv module from 2.2 to 2.3. His code would have to be run using Python 2.2 in order to work. Otherwise, you'll have to rewrite it, and that will probably be a fairly significant task. Fortunately, it's possible to install multiple versions of python on one machine. I verified the difference in the csv modules by comparing the 2.2 and 2.3 versions on one of my Debian machines.

Hope this helps.

.... Bob

Featured Post

Hire Technology Freelancers with Gigs

Work with freelancers specializing in everything from database administration to programming, who have proven themselves as experts in their field. Hire the best, collaborate easily, pay securely, and get projects done right.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article will show the steps for installing Python on Ubuntu Operating System. I have created a virtual machine with Ubuntu Operating system 8.10 and this installing process also works with upgraded version of Ubuntu OS. For installing Py…
Flask is a microframework for Python based on Werkzeug and Jinja 2. This requires you to have a good understanding of Python 2.7. Lets install Flask! To install Flask you can use a python repository for libraries tool called pip. Download this f…
Learn the basics of modules and packages in Python. Every Python file is a module, ending in the suffix: .py: Modules are a collection of functions and variables.: Packages are a collection of modules.: Module functions and variables are accessed us…
Learn the basics of while and for loops in Python.  while loops are used for testing while, or until, a condition is met: The structure of a while loop is as follows:     while <condition>:         do something         repeate: The break statement m…
Suggested Courses

864 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question