Solved

Python function to handle multiple file input types for an argument

Posted on 2011-03-21
9
520 Views
Last Modified: 2012-05-11
I previously posted http://www.experts-exchange.com/Programming/Languages/Scripting/Python/Q_26881841.html but I would now like to take a file, folder or wildcards via command line using a single option/arg. How can the same be achieved using a function? Thanks
0
Comment
Question by:adamshields
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
9 Comments
 
LVL 29

Expert Comment

by:pepr
ID: 35183716
If I understand you well, then you want to get a single argument, and depending on whether it is a directory, a file, or a mask, you want to have a function that returns filenames from the directory, the file, or the files got by expanding the mask.  Is that what you want?

If yes, I suggest to create a generator function that produces the filenames independently on what you pass.  Or it could be no file, or the single file, or the files got via expanded mask, or the files from inside the directory.  (I know I am repeating the same using the other words.)

Then if you get the argument myArg, the you will use it this way:

for fname in myFiles(myArg):
     process(fname)

Open in new window


Do you want something like this?
0
 
LVL 3

Author Comment

by:adamshields
ID: 35183757
@pepr: how would this work with my current logic? I am only going to be using the options.filename and doing away with the options.directory argument.

    filenames_or_wildcards = []

# remove next line if you do not want allow to run the script without the -f -d
# option, but with arguments
    filenames_or_wildcards = args # take all filenames passed in the command line

# if -f was specified add them (in current working directory)
    if options.filename is not None:
        filenames_or_wildcards.append(options.filename)

# if -d was specified or nothing at all add files from dir
    if options.directory is not None:
        filenames_or_wildcards.append( os.path.join(options.directory, "*") )

# Now expand all wildcards
# glob.glob transforms a filename or a wildcard in a list of all matches
# a non existing filename will be 'dropped'
    all_files = []
    for filename_or_wildcard in filenames_or_wildcards:
        all_files.extend( glob.glob(filename_or_wildcard) )

Open in new window

0
 
LVL 29

Accepted Solution

by:
pepr earned 500 total points
ID: 35184026
If yes, try the following code (test data and the script attached in the zip):

a.py
import glob
import os
import sys


def myFiles(arg):
    '''A generator function that diggs all files from the argument.'''
    
    if os.path.isfile(arg):                 # for a single file, return the file
        yield arg
        
    elif os.path.isdir(arg):                # for a directory, return all files
        for item in os.listdir(arg):        # produces bare names
            fname = os.path.join(arg, item) # path + bare name
            if os.path.isfile(fname):       # only the files are returned
                yield fname  # no normalization of slashes/backslashes here
    else:
        for fname in glob.glob(arg):        # it could be a mask -- expand it
            if os.path.isfile(fname):       # only the files are returned
                yield os.path.normpath(fname)  # normalize slashes/backslashes


myArg = ''
if len(sys.argv) > 1:
    myArg = sys.argv[1]
    
print 'For the explicitly given argument:', repr(myArg)
for fname in myFiles(myArg):
    print fname
    
# For other tests the returned filenames are printed as one list.    
print '-' * 70
arg = ''                   # empty argument
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'testDir'            # existing directory
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'nodirDir'           # non-existing directory
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'testDir/a.txt'      # existing file
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'testDir/nofile.txt'  # non-existing file
print 'Test for', repr(arg)
print list(myFiles(arg))
    
print '-' * 70
arg = 'testDir/.'           # existing directory -- another form
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'noDir/*.txt'         # mask expanding to no file
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'testDir/*'           # all files from the existing directory
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'testDir/?.txt'       # one-letter + extension
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'testDir/a*.txt'      # a-prefixed filenames
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'testDir/*a.txt'      # a-suffixed filenames
print 'Test for', repr(arg)
print list(myFiles(arg))

Open in new window


For the directory like (Czech interface):

C:\tmp\_Python\adamshields\Q_26901398\testDir>ls -l
celkem 5
-rwx------+ 1 Petr None 10 Mar 21 20:18 a.txt
-rwx------+ 1 Petr None 11 Mar 21 20:18 aa.txt
drwx------+ 1 Petr None  0 Mar 21 20:35 ad
drwx------+ 1 Petr None  0 Mar 21 20:35 adir
-rwx------+ 1 Petr None 14 Mar 21 20:19 ax.txt
-rwx------+ 1 Petr None 14 Mar 21 20:19 ba.txt
drwx------+ 1 Petr None  0 Mar 21 20:35 subdir
-rwx------+ 1 Petr None 10 Mar 21 20:34 x.txt
drwx------+ 1 Petr None  0 Mar 21 20:35 z

Open in new window


It prints the following on my console:

c:\tmp\_Python\adamshields\Q_26901398>python a.py testdir/b*
For the explicitly given argument: 'testdir/b*'
testdir\ba.txt
----------------------------------------------------------------------
Test for ''
[]
----------------------------------------------------------------------
Test for 'testDir'
['testDir\\a.txt', 'testDir\\aa.txt', 'testDir\\ax.txt', 'testDir\\ba.txt', 'testDir\\x.txt']
----------------------------------------------------------------------
Test for 'nodirDir'
[]
----------------------------------------------------------------------
Test for 'testDir/a.txt'
['testDir/a.txt']
----------------------------------------------------------------------
Test for 'testDir/nofile.txt'
[]
----------------------------------------------------------------------
Test for 'testDir/.'
['testDir/.\\a.txt', 'testDir/.\\aa.txt', 'testDir/.\\ax.txt', 'testDir/.\\ba.txt', 'testDir/.\\x.txt']
----------------------------------------------------------------------
Test for 'noDir/*.txt'
[]
----------------------------------------------------------------------
Test for 'testDir/*'
['testDir\\a.txt', 'testDir\\aa.txt', 'testDir\\ax.txt', 'testDir\\ba.txt', 'testDir\\x.txt']
----------------------------------------------------------------------
Test for 'testDir/?.txt'
['testDir\\a.txt', 'testDir\\x.txt']
----------------------------------------------------------------------
Test for 'testDir/a*.txt'
['testDir\\a.txt', 'testDir\\aa.txt', 'testDir\\ax.txt']
----------------------------------------------------------------------
Test for 'testDir/*a.txt'
['testDir\\a.txt', 'testDir\\aa.txt', 'testDir\\ba.txt']

Open in new window


Feel free to ask on what you want to have explained. (Strange English here, righ?  My fault ;)
Q-26901398.zip
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 29

Expert Comment

by:pepr
ID: 35184061
Well, I was typing meanwhile. The myFiles() processes a single argument but it can easily be adjusted to proces or a single argument or a list of arguments.  Any single argument could be a file, a dir, a mask.
0
 
LVL 3

Author Closing Comment

by:adamshields
ID: 35184080
Thanks for the quick response!
0
 
LVL 29

Expert Comment

by:pepr
ID: 35184178
Have a look at the modified function:

b.py
import glob
import os
import sys


def myFiles(arg):
    '''A generator function that diggs all files from the argument.'''
    
    if isinstance(arg, str):     # single string
        lst = [ arg ]            # convert to the list
    elif isinstance(arg, list):  # list is OK,
        lst = arg                # no conversion
    else:
        print 'Unexpected arg:', repr(arg)
        sys.exit(1)
    # Warning: the arg identifier reused here (not ideal).
    for arg in lst:
        if os.path.isfile(arg):                 # for a single file, return the file
            yield arg
            
        elif os.path.isdir(arg):                # for a directory, return all files
            for item in os.listdir(arg):        # produces bare names
                fname = os.path.join(arg, item) # path + bare name
                if os.path.isfile(fname):       # only the files are returned
                    yield fname  # no normalization of slashes/backslashes here
        else:
            for fname in glob.glob(arg):        # it could be a mask -- expand it
                if os.path.isfile(fname):       # only the files are returned
                    yield os.path.normpath(fname)  # normalize slashes/backslashes

print '-' * 70
arg = 'testDir/a.txt'      # existing file
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = 'testDir/.'           # existing directory -- another form
print 'Test for', repr(arg)
print list(myFiles(arg))

print '-' * 70
arg = ['noDir/*.txt',       # mask expanding to no file
       'testDir/*',         # all files from the existing directory
       'testDir/?.txt',     # one-letter + extension
       'testDir/a*.txt',    # a-prefixed filenames
       'testDir/*a.txt',    # a-suffixed filenames
      ]
print 'Test for', repr(arg)
print list(myFiles(arg))

Open in new window


It prints:
C:\tmp\_Python\adamshields\Q_26901398>python b.py
----------------------------------------------------------------------
Test for 'testDir/a.txt'
['testDir/a.txt']
----------------------------------------------------------------------
Test for 'testDir/.'
['testDir/.\\a.txt', 'testDir/.\\aa.txt', 'testDir/.\\ax.txt', 'testDir/.\\ba.txt', 'testDir/.\\x.txt']
----------------------------------------------------------------------
Test for ['noDir/*.txt', 'testDir/*', 'testDir/?.txt', 'testDir/a*.txt', 'testDir/*a.txt']
['testDir\\a.txt', 'testDir\\aa.txt', 'testDir\\ax.txt', 'testDir\\ba.txt', 
'testDir\\x.txt', 'testDir\\a.txt', 'testDir\\x.txt', 'testDir\\a.txt', 
'testDir\\aa.txt', 'testDir\\ax.txt', 'testDir\\a.txt', 'testDir\\aa.txt', 
'testDir\\ba.txt']

Open in new window


Notice that the names are not uniqued in the case.
0
 
LVL 3

Author Comment

by:adamshields
ID: 35184197
Great thanks, lastly, if I were using:

def main()

Where would def myFiles(arg): be located, before, after or in the main()?

Thanks
0
 
LVL 29

Expert Comment

by:pepr
ID: 35184230
It does not matter in Python.  But it may be good to keep conventions from other languages and put it earlier than main().

Actually, Python interprets the .py file from beginning to the end.  However, the first and the only interpretation means compilation.  Any def, class or such is converted to the internal object, any other command is interpreted.
0
 
LVL 3

Author Comment

by:adamshields
ID: 35184281
Okay thanks!
0

Featured Post

How Do You Stack Up Against Your Peers?

With today’s modern enterprise so dependent on digital infrastructures, the impact of major incidents has increased dramatically. Grab the report now to gain insight into how your organization ranks against your peers and learn best-in-class strategies to resolve incidents.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Remove all hidden metadata properties of MS .Docx Files 7 50
Python 3.5.2 - Script Help 3 93
Problem to copy file 14 51
TypeError jquery issues with site - CSS conflicts 3 28
Sequence is something that used to store data in it in very simple words. Let us just create a list first. To create a list first of all we need to give a name to our list which I have taken as “COURSE” followed by equals sign and finally enclosed …
A quick Powershell script I wrote to find old program installations and check versions of a specific file across the network.
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…

733 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question