Python - how to delete an item from a nested list, file input

The python readlines() function gives me a list, output[], from file and I want to delete any line from output[] that is short the required number of list elements (14).  I can print the index values of the lines concerned, but I can't delete the lines themselves - the objects are either unsubscriptable or don't support item deletion. Help!

output = src.readlines()
 
index = 0
for index, lines in enumerate(output):
   lines = lines.split(',', 13)
   if (len(lines) < 14):
       print index  # prints correct index value 
       del index     # or output2[index] etc, fails
   else:
       index +=1

Open in new window

sara_bellumAsked:
Who is Participating?
 
efnCommented:
If you use the second approach I suggested, where you make a copy of the list, you can make a copy of everything in the list starting from index 4 if you use list_all[4:] instead of list_all[:].  In this approach, you are removing list elements by value, so it won't matter that the indexes in the list being checked and the list being changed are not the same.

There are, of course, other solutions.
0
 
ghostdog74Commented:
assign to another array. Here's a list comprehension
a = [ l for n,l in enumerate(open("file")) if len(l.split(",")) == 14 ]
print a   

Open in new window

0
 
sara_bellumAuthor Commented:
I have added your code to my script but still can't delete the lines that are missing list elements.  I tried several options but my attempts may confuse you, so I simply added some comments to demonstrate what I am trying to do.
file = open('badger_start.dat')
list_all = file.readlines()
 
# check to see if I'm picking up the right errors
list_errors = [ l for n, l in enumerate(file) if len(l.split(",")) < 14 ]
print list_errors # this is hard to read but I know how to fix that
 
for index, line in enumerate(list_all):
#   if the index value matches a line with an error 
#   delete the line
   else:
       index += 1 #the index should only increment for valid lines
 
# check your result
print list_all # hard to read but I know how to fix that
file.close()

Open in new window

0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

 
ghostdog74Commented:
can you post a sample of your badger_start.dat file, and then describe what you actually want to see as output. Its much easier this way
0
 
efnCommented:
It's not going to work well to delete from a list while you are in the middle of iterating through it.

It is possible to fix your example to work with the technique ghostdog74 suggested.  The idea is that instead of trying to delete the elements you don't want from the list, you construct a new list with only the elements you do want.  In your code, you have all the input lines in the list_all list.  In this case, you don't care about the indexes, so there is no need to use the enumerate function.  You can just write an expression to select the lines you want from the list_all list:

wanted = [ line for line in list_all if len(line.split(",")) >= 14 ]

If you want to construct a list of the error lines as in your example, you can just change the condition tested.

Another way to do it is to iterate over a copy of the list, removing elements from the original list.  list_all[:] makes a copy of the list and you can delete from the original list by value.  This is longer, but perhaps easier to read.

for line in list_all[:] :
    if len(line.split(",")) < 14:
        list_all.remove(line)
0
 
mish33Commented:
A) As was said do NOT modify list you iterate on
B) enumerate does +=1 for you
C) be careful with variable names
output = src.readlines()
valid = []
for index, line in enumerate(output):
   fields = line.split(',')
   if len(fields) < 14:
       print index  # prints correct index value 
   else:
       valid.append(line)
# use valid lines

Open in new window

0
 
efnCommented:
> B) enumerate does +=1 for you

Nitpick:  actually, "for ... in ..." does the +=1 for you.  But mish33 did show another way that will work.
0
 
mish33Commented:
efn: I was referring to index += 1 in the OP code
0
 
efnCommented:
So was I.  The idea of the "index += 1" statement was apparently to iterate through the list;  I think we agree that this statement is unnecessary.  The "for ... in ..." statement is what iterates through the various lists in all of the code on this page.  The calls to enumerate are really not needed at all, except for debugging displays of index values.  The enumerate function just returns a list that just sits there, not adding anything to anything.  But this is not really an important point.
0
 
mish33Commented:
enumerate is needed to print indexes of non-complaining lines,
but having both enumerate and manual index counting is um... unnecessary
0
 
sara_bellumAuthor Commented:
Thanks very much!  efn and mish33's solutions do work for me, but I'm trying to start at an index value of 4 to exclude the header information from the sample data (the header has a different format).  I tried inserting index = 4 before the for loop(s) but that fails. I had assumed, apparently incorrectly, that I could start a for loop at any index value in the list.   Let me know if there's a simple answer to this, or if I need to start a new question, thanks.
0
 
mish33Commented:
That approach will keep index (line number) of printed lines right:
output = src.readlines()
valid = []
for index, line in enumerate(output):
   if index < 4: continue  # skip first 4 lines
   fields = line.split(',')
   if len(fields) < 14:
       print index  # prints correct index value 
   else:
       valid.append(line)
# use valid lines

Open in new window

0
 
sara_bellumAuthor Commented:
For reasons which pass my understanding, mish33's solution didn't work the 2d etc time I tried it...so I gave all the points to efn - thanks very much!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.