using perl or shell delete column 2, 3, & 5 in text file and make binned rolling averages per month

Alright. See complete file attached. The files starts like this below. The first column is the date and the second column is the time which can be deleted.  Columns 3 & 6 are not necessary and can be deleted. The 4th  column is uncorrected data which can also be deleted. Aside from deleting columns 2-4 & 6, I want to create a binned rolling average which will be added in a third column.So the third column won't start until month 12 since 12 data points are needed so rows 1-11 will only have two columns of data and the 3rd column of data begins from row 12.




Oulu Neutron Monitor database query - http://cosmicrays.oulu.fi/
Start: 1979/01/01 00:00 End: 2012/01/31 23:00
Units for date column: fractional years
Columns: #1, #2 -date and time, #3 - fractional date, #4, #5 - uncorrected and corrected (for pressure and efficiency ) count rates [counts/min],
#6 - barometric pressure [mbar]
Effectiveness correction factors:
1 (from 1964-01-01)
1.00674 (from 1985-10-01)
1.01147 (from 1995-01-01)
1.00914 (from 2000-01-01)
1.00813 (from 2003-06-01)
1.0029 (from 2008-08-01)
1.0019 (from 2009-11-01)
Pressure correction factors: -0.0074 (from 1964-01-01)
Resolution: 1 months
Note: query times are adjusted to 1979-01-01 00:00 - 2012-01-01 00:00
1979.01.01 00:00:00 1979.0000000 5612 6179 1013.86
1979.02.01 00:00:00 1979.0849315 5810 6152 1009.25
1979.03.01 00:00:00 1979.1616438 5854 6077 1005.37
cray.txt
libertyforall2Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

farzanjCommented:
What needs to be done to the non numeric rows?
Could you please give a sample output?
0
Pierre Fran├žoisSenior consultantCommented:
This shell script gives you the result you want:

#!/bin/bash

sed '1,/^Note:/d' cray.txt \
| grep -v seconds \
| cut -d ' ' -f 1,5 \
| awk -F ' ' '{
	if (++j > 12) {
		j = 1
	};
	if (++i < 12) {
		last12 [j] = $2; 
		tot += $2; 
		printf ("%s %d\n", $1, $2);
	} else {
		tot -= last12 [j]; 
		last12 [j] = $2;
		tot += $2;
		printf ("%s %d %d\n", $1, $2, tot/12);
	}
}'

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
libertyforall2Author Commented:
Works.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.