We help IT Professionals succeed at work.

Python if statement exclude "--" values

AndyC1000
AndyC1000 asked
on
Dear all,

I'm reading data from a netcdf file, I've noticed some fields are written to my file as "--".

When I print the row to the console its value is:
[-33.939999, 151.03918, masked, masked, masked, masked, masked, masked, masked]

I've tried the below code block to check if the row doesn't contain masked or "--" and write those values.  Its not working!

How do I exclude or check for those values?

Thanks

if "masked" not in content:
   outputwriter.writerow(content)
   print content

Open in new window

Comment
Watch Question

The question is not clear. There is not enough information to answer. Please, show some example.

Author

Commented:
Admin, I have provided a response at http:Q_28627100.html

To clarify this question the below code block writes a csv file.   My issue is when there is no data in the content string for a lat/lon, the row is outputted to the csv file as

[-33.939999, 151.03918, --, --, --, --, --, --, --]

or to the console as

[-33.939999, 151.03918, masked, masked, masked, masked, masked, masked, masked]

I tried wrapping the outputwriter around  an if statement i.e.

if "masked" not in content:
   outputwriter.writerow(content)
   print content

with open(r'C:/output.csv', 'wb') as csvFile:
    outputwriter = csv.writer(csvFile, delimiter=',')
    for date_val in date_strings:
        header.append(date_val)
    outputwriter.writerow(header)
    for lat_index, lat in enumerate(lats):
        for lon_index, lon in enumerate(lons):
            content = [lat,lon]
            for time_index, time in enumerate(times[:]): 
                data = value[time_index,lat_index,lon_index]
                content.append(data)
            outputwriter.writerow(content) 

Open in new window

Top Expert 2014

Commented:
please post a sample file

Author

Commented:
from netCDF4 import Dataset, num2date
import csv

filename = "C:/Test.nc"

nc = Dataset(filename, 'r', Format='NETCDF4')

# get coordinates variables
lats = nc.variables['latitude'][:] 
lons = nc.variables['longitude'][:]

if nc.variables['latitude'].units != 'degrees_north':
    raise AttributeError('latitude units attribute not what was expected')

if nc.variables['longitude'].units != 'degrees_east':
    raise AttributeError('longitude units attribute not what was expected')

val = nc.variables['Min_SFC'][:]

times = nc.variables['time']
dates = num2date(times[:],times.units)
date_strings = [date.strftime('%d-%m-%Y') for date in dates]

header =  ['Latitude', 'Longitude']

with open(r'C:/output.csv', 'wb') as csvFile:
    outputwriter = csv.writer(csvFile, delimiter=',')
    for date_val in date_strings:
        header.append(date_val)
    outputwriter.writerow(header)
    for lat_index, lat in enumerate(lats):
        for lon_index, lon in enumerate(lons):
            content = [lat,lon]
            for time_index, time in enumerate(times[:]): 
                data = val[time_index,lat_index,lon_index]
                content.append(data)
            #outputwriter.writerow(content) 
            if "masked" not in content:
                outputwriter.writerow(content)
                print content

# close netcdf 
nc.close()

Open in new window

Top Expert 2014

Commented:
Please a sample DATA file.  Sorry for the confusion.

Author

Commented:
Attached is the output data file, I'm not able to post the input data file (netcdf).
Output.csv
Top Expert 2014

Commented:
This will filter the csv file.  It is crude, but it does seem to work.  Of course, you would replace the print statement with a file-writing statement.
    import csv

    filename = "C:\Users\aikimark\Downloads\Output.csv"
    csvfile= open(filename, 'r')
    f=csv.reader(csvfile)
    for line in f:
        disposition = "drop"
        for item in line[2:]:
            if item <> "--":
                disposition = "keep"
                break
                
        print disposition, line

Open in new window

Top Expert 2014
Commented:
This is a little more elegant, using a comprehension:
    import csv

    filename = "C:\Users\aikimark\Downloads\Output.csv"
    csvfile= open(filename, 'r')
    f=csv.reader(csvfile)
    for line in f:
        disposition = "drop"
        if False in ["--" == xx for xx in line[2:]]:
            disposition = "keep"
                
        print disposition, line

Open in new window