Anthony Mellor
asked on
AWK Code gives empty file, why?
At my wit's end with this one:
code is meant to reorder columns.
cat in.csv | awk 'BEGIN{FS="," ; OFS=","} {print $1,$33,$6,$7,$8,$9,$10,$34,$11,$12,$35,$13,$2,$3,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$31,$4,$5,$14,$30,$32,$36,$37,$38;}' > out.csv
code is meant to reorder columns.
ASKER
I just ran it through dos2unix , so I'll try it on an on unchanged file.
Wonder if all the line eol ends are wrong or even missing, but it looks ok in textedit
this not the first issue with my data files, which originated on a Windows machine possibly donkeys' years ago.
awk version 20070501
Sample attached
I prefer not to use cat as I am even less familiar with it than awk.
running on macos Yosemite 10.12
it would be easier if my awk code was wrong...
sample.txt
Wonder if all the line eol ends are wrong or even missing, but it looks ok in textedit
this not the first issue with my data files, which originated on a Windows machine possibly donkeys' years ago.
awk version 20070501
Sample attached
I prefer not to use cat as I am even less familiar with it than awk.
running on macos Yosemite 10.12
it would be easier if my awk code was wrong...
sample.txt
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Now I get the same.
!!
Thank you.
!!
Thank you.
Hi Anthony,
Well don't blame me for fixing it, because I don't think I've changed anything significant. But thanks for the points!
It's a mystery to me what was wrong in the first place.
BTW, you don't need the ';' before the '}'. You probably never do with awk.
Here's a Perl alternative which is shorter, partly because I can represent ranges of fields with '..'.
perl -F, -ane 'print join(",",@F[0,32,5..9,33,1 0,11,34,12 ,1,2,14..2 8,30,3,4,1 3,29,31,35 ..37])' sample.txt >out.txt
or:
perl -F, -ape '$_=join(",",@F[0,32,5..9, 33,10,11,3 4,12,1,2,1 4..28,30,3 ,4,13,29,3 1,35..37]) ' sample.txt >out.txt
And with Perl (and sed), you can overwrite the original file with the output, by using the '-i' switch:
perl -i -F, -ape '$_=join(",",@F[0,32,5..9, 33,10,11,3 4,12,1,2,1 4..28,30,3 ,4,13,29,3 1,35..37]) ' *.txt
And the above will process ALL files with .txt extensions.
Or you can automatically backup the original before processing, like this:
perl -i.old -F, -ape '$_=join(",",@F[0,32,5..9, 33,10,11,3 4,12,1,2,1 4..28,30,3 ,4,13,29,3 1,35..37]) ' *.txt
When processing sample.txt (for example), the above would back it up to sample.txt.old, and put the output into sample.txt.
The only difference in output between these and your awk solution, is awk seems to be putting a line ending (LF) at the end of the last line, even though is none in the input file (sample.txt).
BTW, what operating system are you using?
Well don't blame me for fixing it, because I don't think I've changed anything significant. But thanks for the points!
It's a mystery to me what was wrong in the first place.
BTW, you don't need the ';' before the '}'. You probably never do with awk.
Here's a Perl alternative which is shorter, partly because I can represent ranges of fields with '..'.
perl -F, -ane 'print join(",",@F[0,32,5..9,33,1
or:
perl -F, -ape '$_=join(",",@F[0,32,5..9,
And with Perl (and sed), you can overwrite the original file with the output, by using the '-i' switch:
perl -i -F, -ape '$_=join(",",@F[0,32,5..9,
And the above will process ALL files with .txt extensions.
Or you can automatically backup the original before processing, like this:
perl -i.old -F, -ape '$_=join(",",@F[0,32,5..9,
When processing sample.txt (for example), the above would back it up to sample.txt.old, and put the output into sample.txt.
The only difference in output between these and your awk solution, is awk seems to be putting a line ending (LF) at the end of the last line, even though is none in the input file (sample.txt).
BTW, what operating system are you using?
ASKER
macos yosemite 10.12.3 - think you've just earned those points, I'm pondering it all :-)
MacOS comes with Perl installed, right?
BTW, in case it wasn't obvious, those field numbers listed in my Perl one-liners were zero based (0 - 37, not 1 - 38).
Perl is a full-blown language, and is more flexible than awk. But for tasks like your one there isn't a huge difference. (Most) Perl code is probably more portable though, i.e. the same code should run on a lot of operating systems.
BTW, in case it wasn't obvious, those field numbers listed in my Perl one-liners were zero based (0 - 37, not 1 - 38).
Perl is a full-blown language, and is more flexible than awk. But for tasks like your one there isn't a huge difference. (Most) Perl code is probably more portable though, i.e. the same code should run on a lot of operating systems.
ASKER
continued here:
https://www.experts-exchange.com/questions/29006443/Perl-Versus-AWK.html
can't have you engaged with no prospect of points :-)
https://www.experts-exchange.com/questions/29006443/Perl-Versus-AWK.html
can't have you engaged with no prospect of points :-)
Sounds beyond me and the amount of time I'm willing to put in to it, sorry Anthony.
But thanks for the invite.
But out of interest, what Perl version, etc, info do you get when you run these commands?:
perl -v
perl -V
But thanks for the invite.
But out of interest, what Perl version, etc, info do you get when you run these commands?:
perl -v
perl -V
ASKER
no problem at all, happily I have the time at this time, I completely understand your constraint, hope that's an acceptable word.
to answer your question
gosh, much info:
MacPro:FinalProcess ADM$ perl -v
This is perl 5, version 18, subversion 2 (v5.18.2) built for darwin-thread-multi-2level
(with 2 registered patches, see perl -V for more detail)
Copyright 1987-2013, Larry Wall
Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.
MacPro:FinalProcess ADM$ perl -V
Summary of my perl5 (revision 5 version 18 subversion 2) configuration:
Platform:
osname=darwin, osvers=16.0, archname=darwin-thread-mul ti-2level
uname='darwin osx300.apple.com 16.0 darwin kernel version 15.0.0: wed apr 6 00:55:38 pdt 2016; root:xnu-3247.1.106.2.8~1d evelopment _x86_64 x86_64 '
config_args='-ds -e -Dprefix=/usr -Dccflags=-g -pipe -Dldflags= -Dman3ext=3pm -Duseithreads -Duseshrplib -Dinc_version_list=none -Dcc=cc'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-arch x86_64 -arch i386 -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector',
optimize='-Os',
cppflags='-g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector'
ccversion='', gccversion='4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc -mmacosx-version-min=10.12 .2', ldflags ='-arch x86_64 -arch i386 -fstack-protector'
libpth=/usr/lib /usr/local/lib
libs=
perllibs=
libc=, so=dylib, useshrplib=true, libperl=libperl.dylib
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags='-arch x86_64 -arch i386 -bundle -undefined dynamic_lookup -fstack-protector'
Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TI ME_HARD
PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV PERL_SAWAMPERSAND USE_64_BIT_ALL
USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
USE_REENTRANT_API
Locally applied patches:
/Library/Perl/Updates/<ver sion> comes before system perl directories
installprivlib and installarchlib points to the Updates directory
Built under darwin
Compiled at Jul 30 2016 16:59:44
@INC:
/Library/Perl/5.18/darwin- thread-mul ti-2level
/Library/Perl/5.18
/Network/Library/Perl/5.18 /darwin-th read-multi -2level
/Network/Library/Perl/5.18
/Library/Perl/Updates/5.18 .2
/System/Library/Perl/5.18/ darwin-thr ead-multi- 2level
/System/Library/Perl/5.18
/System/Library/Perl/Extra s/5.18/dar win-thread -multi-2le vel
/System/Library/Perl/Extra s/5.18
.
MacPro:FinalProcess ADM$
to answer your question
gosh, much info:
MacPro:FinalProcess ADM$ perl -v
This is perl 5, version 18, subversion 2 (v5.18.2) built for darwin-thread-multi-2level
(with 2 registered patches, see perl -V for more detail)
Copyright 1987-2013, Larry Wall
Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.
MacPro:FinalProcess ADM$ perl -V
Summary of my perl5 (revision 5 version 18 subversion 2) configuration:
Platform:
osname=darwin, osvers=16.0, archname=darwin-thread-mul
uname='darwin osx300.apple.com 16.0 darwin kernel version 15.0.0: wed apr 6 00:55:38 pdt 2016; root:xnu-3247.1.106.2.8~1d
config_args='-ds -e -Dprefix=/usr -Dccflags=-g -pipe -Dldflags= -Dman3ext=3pm -Duseithreads -Duseshrplib -Dinc_version_list=none -Dcc=cc'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-arch x86_64 -arch i386 -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector',
optimize='-Os',
cppflags='-g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector'
ccversion='', gccversion='4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc -mmacosx-version-min=10.12
libpth=/usr/lib /usr/local/lib
libs=
perllibs=
libc=, so=dylib, useshrplib=true, libperl=libperl.dylib
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags='-arch x86_64 -arch i386 -bundle -undefined dynamic_lookup -fstack-protector'
Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TI
PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV PERL_SAWAMPERSAND USE_64_BIT_ALL
USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
USE_REENTRANT_API
Locally applied patches:
/Library/Perl/Updates/<ver
installprivlib and installarchlib points to the Updates directory
Built under darwin
Compiled at Jul 30 2016 16:59:44
@INC:
/Library/Perl/5.18/darwin-
/Library/Perl/5.18
/Network/Library/Perl/5.18
/Network/Library/Perl/5.18
/Library/Perl/Updates/5.18
/System/Library/Perl/5.18/
/System/Library/Perl/5.18
/System/Library/Perl/Extra
/System/Library/Perl/Extra
.
MacPro:FinalProcess ADM$
You've definitely got Perl, then. Version 5.18.2.
ASKER
ta :-)
Might be a difference in awk version, or might be because your input data has issues.
Here's my awk version:
$ awk -V
GNU Awk 4.0.2
Copyright (C) 1989, 1991-2012 Free Software Foundation.
...etc...
And here's the input file:
$ cat -vet in.csv
Open in new window
Please ignore the "$" at the end which is a newline (line-feed) and is displayed that way because of cat's "-vet" option.And here's the output I get:
Open in new window
Note how I didn't use "cat" in the command above, to feed in.csv into awk. I just redirected it using a "<" at the end of the command. But this is just another (slightly more concise) way of doing it, and shouldn't make any difference to the result.$ cat -vet out.csv
Open in new window
Please attach a sample input file and show us your awk version.