Solved

gawk field separators

Posted on 2014-02-16
2
251 Views
Last Modified: 2014-02-16
I have a shell script (below) that scans through directories, finding files ending in a .arf file extension (sample below), and then writes data from the file to the database (sample also below).  Everything works correctly except there are two time fields which have colons in the middle of the field so gawk is stripping them as well.  How can I change the field separator so that it will correctly process the fields with colons in them as well?

Shell script
#!/bin/bash

echo `date`
find . -name "*.arf" | while read f; do
  newpath="$(basename $(dirname "$f"))"
#/$(basename $f)"
  cat "$f" | gawk -F'[:"]' -v p="$newpath" '{ 
      nlist=nlist "`, `" $1;
      gsub("'\''","'\\\\\''", $3);
      vlist=vlist ", '\''" $3 "'\''";
  }
  END { 
    printf "insert into `database`.`archives` (`NEWPATH%s`) values ('\''%s'\''%s);\n", nlist, p, vlist;
  }' >> myinsertfile4.sql
#| tee -a myinsertfile4.sql
  cnt=$((cnt+1))
  [ $(($cnt%100)) -eq 0 ] && echo "File #$cnt: $f"
done

echo "Total Files: $cnt"

echo `date`

Open in new window


Sample .arf file:
FILEID: "TIF460222"
PATH: "/optical/incoming/TIF460222"
TYPE: "TIF"
SECLEV: "10"
STATID: ""
USRID: "admin"
REQDATE: "03/15/2012"
REQTIME: "08:43:32"
GENDATE: "03/16/2012"
GENTIME: "08:43:32"
PROGID: ""
GROUPID: "Drive up rec's"
DESC: "March"

Open in new window


Sample Output:
insert into `database`.`archives` (`NEWPATH`, `FILEID`, `PATH`, `TYPE`, `SECLEV`, `STATID`, `USRID`, `REQDATE`, `REQTIME`, `GENDATE`, `GENTIME`, `PROGID`, `GROUPID`, `DESC`) values ('.', 'TIF460222', '/optical/incoming/TIF460222', 'TIF', '10', '', 'admin', '03/15/2012', '08', '03/16/2012', '08', '', 'Drive up rec\'s', 'March');

Open in new window

0
Comment
Question by:bdhtechnology
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
2 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 39863171
New line #7:

cat "$f" | gawk -F': |"' -v p="$newpath" '{
...
...

It's a different way to specify the separator. The above means "colon space OR double quote"
0
 
LVL 1

Author Comment

by:bdhtechnology
ID: 39863222
Perfect, thanks again!
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Utilizing an array to gracefully append to a list of EmailAddresses
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

710 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question