philsivyer
asked on
filter out unwanted rows
Hello
Trying to remove unwanted rows from txt file.
eg
import csv
data = open("X:/r_dhwm.csv")
ma=[]
r = csv.DictReader(data,['Date ','TEST',' TEST2','TE ST3'])
for a in r:
if "{TEST}" == 'TEST':
break
else:
ma.append("{TEST}".format( **a))
print(ma)
data.close()
RESULT =
['TEST', '', '', '', '', '', '', '', '-0.000412682', '-0.000393387']
What I want is
['-0.000412682', '-0.000393387']
Trying to remove unwanted rows from txt file.
eg
import csv
data = open("X:/r_dhwm.csv")
ma=[]
r = csv.DictReader(data,['Date
for a in r:
if "{TEST}" == 'TEST':
break
else:
ma.append("{TEST}".format(
print(ma)
data.close()
RESULT =
['TEST', '', '', '', '', '', '', '', '-0.000412682', '-0.000393387']
What I want is
['-0.000412682', '-0.000393387']
ASKER
OK
I have attached a csv file - I want to be able to pick any column and return data to an array but with no headers (as in row 1 of csv file) or any cells that are null or blank (no values).
So, if I want to return data into array from column "TEST1" as in the attached my firsdt value world be: -0.011027536
Regards
I have attached a csv file - I want to be able to pick any column and return data to an array but with no headers (as in row 1 of csv file) or any cells that are null or blank (no values).
So, if I want to return data into array from column "TEST1" as in the attached my firsdt value world be: -0.011027536
Regards
I can see no attachment.
ASKER
Sorry
ee.csv
ee.csv
Try the following script:
a.py
It prints on my console:
If your Python does not support the with construct, try the older way:
a.py
import csv
def columnNonEmptyValues(csvFileName, columnName):
with open(csvFileName, 'rb') as f:
for d in csv.DictReader(f):
value = d.get(columnName, '') # default if not present
if value:
yield value
if __name__ == '__main__':
# Now you can iterate through the values of the chosen column.
for value in columnNonEmptyValues('ee.csv', 'TEST1'):
print value
# Or you can pass the iterator to the list constructor.
lst = list(columnNonEmptyValues('ee.csv', 'TEST1'))
print '-' * 70
print lst
It prints on my console:
c:\tmp\_Python\philsivyer\Q_27640149>python a.py
-0.011027536
-0.004086121
-0.012063901
-0.010020955
-0.002050665
-0.004273241
-0.009383166
-0.013938726
...
-0.055930797
-0.066374625
-0.070332921
-0.05798936
----------------------------------------------------------------------
['-0.011027536', '-0.004086121', '-0.012063901', '-0.010020955', '-0.002050665',
'-0.004273241', '-0.009383166', '-0.013938726', '-0.03455181', '-0.045187455',
'-0.045987727', '-0.043084829', '-0.040607066', '-0.041589412', '-0.031441661',
'-0.032485216', '-0.015742733', '-0.01418973', '-0.012148488', '-0.010371504', '
-0.008392192', '-0.009299116', '-0.009473792', '-0.00851022', '-0.009960434', '-
0.009133718', '-0.005845811', '-0.005267109', '-0.0102875', '-0.014345424', '-0.
010280687', '-0.010643898', '-0.008002167', '-0.007472288', '-0.006297341', '-0.
017303705', '-0.021429973', '-0.013058342', '-0.002337027', '0.002957021', '-0.0
04385036', '-0.010007584', '-0.019457466', '-0.048419451', '-0.073353606', '-0.0
55930797', '-0.066374625', '-0.070332921', '-0.05798936']
If your Python does not support the with construct, try the older way:
...
def columnNonEmptyValues(csvFileName, columnName):
f = open(csvFileName, 'rb')
for d in csv.DictReader(f):
value = d.get(columnName, '') # default if not present
if value:
yield value
f.close()
ASKER
Thanks - works a treat.
Question - excuse my ignorance but as a newbie to Python ...
can you explain this bit .. if __name__ == '__main__':
and ...
how does it know how to ignore null values or empty strings and not include the JARF header?
Regards
Question - excuse my ignorance but as a newbie to Python ...
can you explain this bit .. if __name__ == '__main__':
and ...
how does it know how to ignore null values or empty strings and not include the JARF header?
Regards
ASKER
Sorry - one more question
How can I write results to txt file
outfile = open("myresults.txt2,"wb")
etc ??
Regards
How can I write results to txt file
outfile = open("myresults.txt2,"wb")
etc ??
Regards
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
If the csv.DictReader(f) does not get the list of columns, it interprets the first line as the record with the column names.
If the d is a dictionary, the d['TEST1'] returns the value for the key 'TEST1'. But it would fail if the key was not in the dictionary. The d.get('TEST1', default) is the alternative that returns the same if the key exist or the given default if the item does not exist in the dictionary.
The object behaves as a boolean value in so called boolean context. In other words, there is a boolean expression expected after the if command. If it is not a boolean expression, the object interprets itself as a boolean value. For strings, lists, and other sequence or containers, the empty value is interpreted as False, a non-empty value is interpreted as True. This way, if the value was or read or set by default as empty string, it is interpreted as False, and it is not yielded by the generator (i.e. ignored).
If the d is a dictionary, the d['TEST1'] returns the value for the key 'TEST1'. But it would fail if the key was not in the dictionary. The d.get('TEST1', default) is the alternative that returns the same if the key exist or the given default if the item does not exist in the dictionary.
The object behaves as a boolean value in so called boolean context. In other words, there is a boolean expression expected after the if command. If it is not a boolean expression, the object interprets itself as a boolean value. For strings, lists, and other sequence or containers, the empty value is interpreted as False, a non-empty value is interpreted as True. This way, if the value was or read or set by default as empty string, it is interpreted as False, and it is not yielded by the generator (i.e. ignored).
ASKER
Thanks
Great response - helps a lot
Great response - helps a lot
ASKER
Many thanks
You are welcome. Have a good day. ;)
The csv.DictReader() may be overkill for the purpose. Anyway, the file must be opened for reading in binary mode. Also the data is not a good identifier for the file object.
What version of Python do you use? If it is new enough, you should prefer the with construct (the automatic file object close is done).
The following code
Open in new window
... is equivalent to
Open in new window
as the "{TEST}" == 'TEST' never holds.