Python 15 columns to single column

I am trying to take lines from a .txt file that has 15 columns, and subset out a certain range of lines and then write them in one long column. The text file represents a lattitude/longitude grid that puts a band of latittude in 24 lines, 15 columns and I'm trying to get a certain area. If i can read the band of latitude in a column I can grab the areas by their longitude.

My text file looks like:

1  2  3  4  ...  15
16  17  18 etc
 and needs to look like:
1
2
3
etc.

here's part of the code:
 LatCount = 1
DayFile =  open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'.txt', 'r')
LatFile= open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'_'+LatStart+'-'+LatEnd+'.txt', 'w')
for line in DayFile:
    if LatCount >=LatStartLine and LatCount <= LatEndLine:
        LatFile.write(line)
    LatCount +=1

It's writing the file just fine. I just can't figure out the easiest way to read it into a single column.
Shaft960Asked:
Who is Participating?
 
SuperdaveCommented:
You might just need to keep a counter and when it gets up to 360, have it print a linefeed.
What do you want for row separators and column separators?
Something like this might work to finish what you have above:

LatFile = open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/NLF_daily_200111_01_13-20.txt','r')
LatLonFile = open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'_subset_final.txt', 'w')
counter=0
for line in LatFile:
    linesplit= line.split()
    for line in linesplit:
        OutFile.write(line)        
        counter += 1
        if counter=360:
            counter=0
            OutFile.write('\n')
        else:
            OutFile.write('\t')   # or ','
       
0
 
SuperdaveCommented:
If you have tab-separated values in the file LatFile, you could do:

re.split('\r?\n|\t',read(LatFile))

to read in the whole file and split it by both lines and tabs to get a list.
0
 
Shaft960Author Commented:
Ok I did some thinking about my data and what I need to do is take the first 24 lines of the text file and make that column 1. Then the next 24 lines are column 2 etc. How do I do that?
0
Cloud Class® Course: Microsoft Azure 2017

Azure has a changed a lot since it was originally introduce by adding new services and features. Do you know everything you need to about Azure? This course will teach you about the Azure App Service, monitoring and application insights, DevOps, and Team Services.

 
SuperdaveCommented:
Is it one item per line now or still 15 per line (so that 24*15 items all go in column 1)?
Then do you want to write them out again?  I want to understand this right before I try coming up with code.
0
 
Shaft960Author Commented:
Sorry for the confusion.
I don't know why they gave me the data in 15 columns. Its not very intuitive. However, the text file reads 1-15 in columns and every 24 rows represents one band of latitude. So I want to take those 24 rows and read them down one column so that that column 1 has 360 values (15columns*24 rows = 360). then I want to take the next 24 rows and make column 2. The new output should be one column for every band of latitude selected by the user (in this case I have 8) so 8 columns and 360 rows (one for every degree of longitude). this will orient the grid with North to the right, and 0 degrees longitude at the top. From this I can select rows for whatever longitude by selecting its row #.
I hope this helps. Its hard to explain. Thanks
0
 
SuperdaveCommented:
lats=[]
for column in range(0,8):
        lats.append([])
        for i in range(0,24):
                print i
                lats[column] += DayFile.readline().strip().split('\t')
#now lats should be an 8x360 list
0
 
Shaft960Author Commented:
Hmmm...It doesn't seem to be working. I get a list of 0 to 24 that repeats 8 times with no breaks. What I need to do is take a text file that is tab-delimited and has 15 columns, and make every 24 rows into one column and then the next 24 rows one column etc. for as many 24 rows as the text file has.
0
 
SuperdaveCommented:
Sorry about that, the print statement was for debugging.  Take it out.  Then you're left with the array lats to do what you want.  It will be sideways so the first subscript is the column like lats[columnnum][row] if you want eight columns.  So you could do something like (after the code from my last post):

for row in range(0,360):
    for column in range(0,8):
        LatFile.write(lats[column][row]+'\t')
    LatFile.write('\n')

Something like that.  I didn't test it and I'm not sure if that's what you want to do with the information once you've read it in.
0
 
Shaft960Author Commented:
Ok, I couldn't get that to work. I'm a beginner, can you tell? I can get it to write everything in one long column. I want to separate that column every 360 rows and make it a new column. I'm at this point:

LatFile = open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/NLF_daily_200111_01_13-20.txt','r')
LatLonFile = open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'_subset_final.txt', 'w')
for line in LatFile:
    linesplit= line.split()
    for line in linesplit:
       
0
 
Shaft960Author Commented:
That puts it into 360 columns and 8 rows but no worries! I can subset by column instead of row.  I arrived at a similar solution but didn't know the syntax. Thanks SuperDave!!
0
 
Shaft960Author Commented:
Here is the full code if you were wondering. Maybe i should have posted this first.



##THIS PORTION OF THE CODE SUBSETS DAILY DATA
##REQUIRES INPUT OF DAY, MONTH, YEAR, PARAMETER TYPE: DLF,NLF,LWCRF
##INPUT BOUNDING BOX COORDINATES AS LATSTART FROM SOUTHERN BOUND, LATEND IS NORTHERN BOUND
##LONGSTART IS WESTERN BOUND, LONGEND IS EASTERN BOUND

##Input parameters for subset:
Data= 'NLF' ##Longwave parameters = DLF, NLF, LWCR
Day= '01' ##Has to be 2 digits  !!!Change to lists for multiple!!!!
Month= '11' ##2 digits
Year= '2001' ##4 digits
InFilename= Data+'_'+'daily'+'_'+Year+Month
print InFilename +'_' + Day

##Lattitude starts at South Pole -90, South values are negative, North is positive
LatStart= '13'
LatEnd= '20'

LongStart= '73'
LongEnd= '81'


## Bounding box
##20N,73E-------20N,81E
##19N,73E-------19N,81E
##.....................
##.....................
##.....................
##.....................
##.....................
##13N,73E-------13N,81E


## Each day has 180(Latitude -90,90) x 360 (0,180,0) cells = 64,800/15columns =4320 lines
## Every 4320 lines = 1 day

##Subset Day
DayEndLine = int(Day)*4320
DayStartLine= DayEndLine-4319
DayCount = 1
MonthFile = open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/'+InFilename+'.ascii', 'r')
DayFile =  open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'.txt', 'w')
for line in MonthFile:
    if DayCount >= DayStartLine and DayCount <= DayEndLine: ##Write lines DayStartLine:DayEndLine
        DayFile.write(str(line)) ##Write Day in 15 column matrix
        DayFile.close
    DayCount +=1


####For Day, subset bands of lattitude
##Every band of lattitude is 24 lines, 15 columns
LatEndLine= (90+int(LatEnd))*24
LatStartLine=((90+int(LatStart))*24)+1
LatCount = 1
DayFile =  open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'.txt', 'r')
LatFile= open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'_'+LatStart+'-'+LatEnd+'.txt', 'w')
for line in DayFile:
    if LatCount >=LatStartLine and LatCount <= LatEndLine:
        LatFile.write(line) ##Write Lat in SINGLE COLUMN
    LatCount +=1

##Takes the subsetted lattitude bands and reads them into 360 columns x lines of lattitude (S->N)
LatFile = open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/NLF_daily_200111_'+Day+'_'+LatStart+'-'+LatEnd+'.txt','r')
LatLonFile = open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'_subset_lat.txt', 'w')
count=0
for line in LatFile:
    linesplit= line.split()

    for line in linesplit:
        LatLonFile.write(line)
        count +=1
        if count >=360:
            count=0
            LatLonFile.write('\n')
        else:
            LatLonFile.write(',')

##Takes subsetted lines of lattitude and subsets out lines of longitude (W->E)
import csv
reader = csv.reader(open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'_subset_lat.txt', 'r'))
Grid= open('F:/SRB/srb_rel2.5_qclw_daily-1995-2005/test/lw_subsets/'+InFilename+'_'+Day+'_grid.txt', 'w')
for line in reader:
    for column in range(int(LongStart),int(LongEnd)):
        Grid.write(str(line[column]+','))
    Grid.write('\n')


0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.