Pythoon loop through several csv files

I have several (57) csv files.  They are all setup with the same data structure.  I want to loop through all files and if the value in column 3 is null/blank, I want save the value in column 1 to a file.  I wrote the following to address this in one file, but how would I loop through all 57 files and save the output.

#!/usr/bin/python

import csv  
data = csv.reader(open('data.csv'))  
fields = data.next()  
for row in data:  
      if row[2] == '':
            print row[0]
abuhaneefAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

prashanthdCommented:
try the following, set the rootdir path to the dir where csv files are present
#!/usr/bin/python
import csv  
import os

rootdir='c:\Your\Path'

for files in os.walk(rootdir):

for file in files:

data = csv.reader(open(file))  
fields = data.next()  
for row in data:  
      if row[2] == '':
            print row[0]

Open in new window

0
peprCommented:
The bacslashes in the path should be doubled, or replaced by normal slashes, or r'The raw string' should be used.

The indentation matters.

The 'file' is better not to be used (already used for something else).

The os.walk() returns tuples like (dirpath, dirnames, filenames).

The open file should be closed.

Does the fields = data.next() skip the heading?

0
abuhaneefAuthor Commented:
Yes, data.next() skips the heading.
0
peprCommented:
Do you have explicit list of the csv files, are they in a single directory, or are they in more directories?
0
peprCommented:
Try the followind code (attached in the zip with the test data files)

import csv
import glob
import os

def extract(reader, writer):
    for row in reader:
        ##print row
        if row[2] == '':
            writer.writerow(row[:1])  # the sublist with only a single element
          

myPath = 'mydir'
myMask = os.path.join(myPath, '*.csv')  # more reliable way to get 'mydir/*.csv'
##print myMask

names = glob.glob(myMask)         # alternatively can be filled differently
##print names

fout = open('output.csv', 'wb')   # binary mode is important
writer = csv.writer(fout)

for name in names:                # could be any generator of names here
    if os.path.isfile(name):      # could be left out for simplicity
        print name
        fin = open(name, 'rb')    # binary mode again
        reader = csv.reader(fin)  # open the reader wrapped around the fin
        extract(reader, writer)   # call the above function
        fin.close()               # close the input file
        
fout.close()                      # close the output file

Open in new window

Q-26893106.zip
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Python

From novice to tech pro — start learning today.