Link to home
Start Free TrialLog in
Avatar of Anthony Mellor
Anthony MellorFlag for United Kingdom of Great Britain and Northern Ireland

asked on

AWK Code gives empty file, why?

At my wit's end with this one:

cat in.csv   | awk 'BEGIN{FS="," ; OFS=","}  {print $1,$33,$6,$7,$8,$9,$10,$34,$11,$12,$35,$13,$2,$3,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$31,$4,$5,$14,$30,$32,$36,$37,$38;}' > out.csv

Open in new window


code is meant to reorder columns.
Avatar of tel2
tel2
Flag of New Zealand image

Seems to work for me, Anthony.

Might be a difference in awk version, or might be because your input data has issues.

Here's my awk version:
$ awk -V
GNU Awk 4.0.2
Copyright (C) 1989, 1991-2012 Free Software Foundation.
...etc...

And here's the input file:
$ cat -vet in.csv
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38$

Open in new window

Please ignore the "$" at the end which is a newline (line-feed) and is displayed that way because of cat's "-vet" option.

And here's the output I get:
$ awk 'BEGIN{FS=","; OFS=","} {print $1,$33,$6,$7,$8,$9,$10,$34,$11,$12,$35,$13,$2,$3,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$31,$4,$5,$14,$30,$32,$36,$37,$38;}' <in.csv >out.csv

Open in new window

Note how I didn't use "cat" in the command above, to feed in.csv into awk.  I just redirected it using a "<" at the end of the command.  But this is just another (slightly more concise) way of doing it, and shouldn't make any difference to the result.

$ cat -vet out.csv
1,33,6,7,8,9,10,34,11,12,35,13,2,3,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,31,4,5,14,30,32,36,37,38$

Open in new window


Please attach a sample input file and show us your awk version.
Avatar of Anthony Mellor

ASKER

I just ran it through dos2unix , so I'll try it on an on unchanged file.
Wonder if all the line eol ends are wrong or even missing, but it looks ok in textedit

this not the first issue with my data files, which originated on a Windows machine possibly donkeys' years ago.

awk version 20070501

Sample attached

I prefer not to use cat as I am even less familiar with it than awk.
running on macos Yosemite 10.12

it would be easier if my awk code was wrong...
sample.txt
ASKER CERTIFIED SOLUTION
Avatar of tel2
tel2
Flag of New Zealand image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Now I get the same.
!!
Thank you.
Hi Anthony,

Well don't blame me for fixing it, because I don't think I've changed anything significant.  But thanks for the points!
It's a mystery to me what was wrong in the first place.

BTW, you don't need the ';' before the '}'.  You probably never do with awk.

Here's a Perl alternative which is shorter, partly because I can represent ranges of fields with '..'.
    perl -F, -ane 'print join(",",@F[0,32,5..9,33,10,11,34,12,1,2,14..28,30,3,4,13,29,31,35..37])' sample.txt >out.txt
or:
    perl -F, -ape '$_=join(",",@F[0,32,5..9,33,10,11,34,12,1,2,14..28,30,3,4,13,29,31,35..37])' sample.txt >out.txt

And with Perl (and sed), you can overwrite the original file with the output, by using the '-i' switch:
    perl -i -F, -ape '$_=join(",",@F[0,32,5..9,33,10,11,34,12,1,2,14..28,30,3,4,13,29,31,35..37])' *.txt
And the above will process ALL files with .txt extensions.
Or you can automatically backup the original before processing, like this:
    perl -i.old -F, -ape '$_=join(",",@F[0,32,5..9,33,10,11,34,12,1,2,14..28,30,3,4,13,29,31,35..37])' *.txt
When processing sample.txt (for example), the above would back it up to sample.txt.old, and put the output into sample.txt.

The only difference in output between these and your awk solution, is awk seems to be putting a line ending (LF) at the end of the last line, even though is none in the input file (sample.txt).

BTW, what operating system are you using?
macos yosemite 10.12.3    - think you've just earned those points, I'm pondering it all  :-)
MacOS comes with Perl installed, right?

BTW, in case it wasn't obvious, those field numbers listed in my Perl one-liners were zero based (0 - 37, not 1 - 38).

Perl is a full-blown language, and is more flexible than awk.  But for tasks like your one there isn't a huge difference.  (Most) Perl code is probably more portable though, i.e. the same code should run on a lot of operating systems.
continued here:
https://www.experts-exchange.com/questions/29006443/Perl-Versus-AWK.html

can't have you engaged with no prospect of points :-)
Sounds beyond me and the amount of time I'm willing to put in to it, sorry Anthony.
But thanks for the invite.
But out of interest, what Perl version, etc, info do you get when you run these commands?:
  perl -v
  perl -V
no problem at all, happily I have the time at this time, I completely understand your constraint, hope that's an acceptable word.

to answer your question

gosh, much info:

MacPro:FinalProcess ADM$ perl -v

This is perl 5, version 18, subversion 2 (v5.18.2) built for darwin-thread-multi-2level
(with 2 registered patches, see perl -V for more detail)

Copyright 1987-2013, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

MacPro:FinalProcess ADM$ perl -V
Summary of my perl5 (revision 5 version 18 subversion 2) configuration:
   
  Platform:
    osname=darwin, osvers=16.0, archname=darwin-thread-multi-2level
    uname='darwin osx300.apple.com 16.0 darwin kernel version 15.0.0: wed apr 6 00:55:38 pdt 2016; root:xnu-3247.1.106.2.8~1development_x86_64 x86_64 '
    config_args='-ds -e -Dprefix=/usr -Dccflags=-g  -pipe  -Dldflags= -Dman3ext=3pm -Duseithreads -Duseshrplib -Dinc_version_list=none -Dcc=cc'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-arch x86_64 -arch i386 -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector',
    optimize='-Os',
    cppflags='-g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector'
    ccversion='', gccversion='4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc -mmacosx-version-min=10.12.2', ldflags ='-arch x86_64 -arch i386 -fstack-protector'
    libpth=/usr/lib /usr/local/lib
    libs=
    perllibs=
    libc=, so=dylib, useshrplib=true, libperl=libperl.dylib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-arch x86_64 -arch i386 -bundle -undefined dynamic_lookup -fstack-protector'


Characteristics of this binary (from libperl):
  Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
                        PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
                        PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
                        PERL_PRESERVE_IVUV PERL_SAWAMPERSAND USE_64_BIT_ALL
                        USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
                        USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
                        USE_REENTRANT_API
  Locally applied patches:
      /Library/Perl/Updates/<version> comes before system perl directories
      installprivlib and installarchlib points to the Updates directory
  Built under darwin
  Compiled at Jul 30 2016 16:59:44
  @INC:
    /Library/Perl/5.18/darwin-thread-multi-2level
    /Library/Perl/5.18
    /Network/Library/Perl/5.18/darwin-thread-multi-2level
    /Network/Library/Perl/5.18
    /Library/Perl/Updates/5.18.2
    /System/Library/Perl/5.18/darwin-thread-multi-2level
    /System/Library/Perl/5.18
    /System/Library/Perl/Extras/5.18/darwin-thread-multi-2level
    /System/Library/Perl/Extras/5.18
    .
MacPro:FinalProcess ADM$
You've definitely got Perl, then.  Version 5.18.2.
ta :-)