Solved

gawk field separators

Posted on 2014-02-16
2
239 Views
Last Modified: 2014-02-16
I have a shell script (below) that scans through directories, finding files ending in a .arf file extension (sample below), and then writes data from the file to the database (sample also below).  Everything works correctly except there are two time fields which have colons in the middle of the field so gawk is stripping them as well.  How can I change the field separator so that it will correctly process the fields with colons in them as well?

Shell script
#!/bin/bash

echo `date`
find . -name "*.arf" | while read f; do
  newpath="$(basename $(dirname "$f"))"
#/$(basename $f)"
  cat "$f" | gawk -F'[:"]' -v p="$newpath" '{ 
      nlist=nlist "`, `" $1;
      gsub("'\''","'\\\\\''", $3);
      vlist=vlist ", '\''" $3 "'\''";
  }
  END { 
    printf "insert into `database`.`archives` (`NEWPATH%s`) values ('\''%s'\''%s);\n", nlist, p, vlist;
  }' >> myinsertfile4.sql
#| tee -a myinsertfile4.sql
  cnt=$((cnt+1))
  [ $(($cnt%100)) -eq 0 ] && echo "File #$cnt: $f"
done

echo "Total Files: $cnt"

echo `date`

Open in new window


Sample .arf file:
FILEID: "TIF460222"
PATH: "/optical/incoming/TIF460222"
TYPE: "TIF"
SECLEV: "10"
STATID: ""
USRID: "admin"
REQDATE: "03/15/2012"
REQTIME: "08:43:32"
GENDATE: "03/16/2012"
GENTIME: "08:43:32"
PROGID: ""
GROUPID: "Drive up rec's"
DESC: "March"

Open in new window


Sample Output:
insert into `database`.`archives` (`NEWPATH`, `FILEID`, `PATH`, `TYPE`, `SECLEV`, `STATID`, `USRID`, `REQDATE`, `REQTIME`, `GENDATE`, `GENTIME`, `PROGID`, `GROUPID`, `DESC`) values ('.', 'TIF460222', '/optical/incoming/TIF460222', 'TIF', '10', '', 'admin', '03/15/2012', '08', '03/16/2012', '08', '', 'Drive up rec\'s', 'March');

Open in new window

0
Comment
Question by:bdhtechnology
2 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 39863171
New line #7:

cat "$f" | gawk -F': |"' -v p="$newpath" '{
...
...

It's a different way to specify the separator. The above means "colon space OR double quote"
0
 
LVL 1

Author Comment

by:bdhtechnology
ID: 39863222
Perfect, thanks again!
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Fine Tune your automatic Updates for Ubuntu / Debian
Google Drive is extremely cheap offsite storage, and it's even possible to get extra storage for free for two years.  You can use the free account 15GB, and if you have an Android device..when you install Google Drive for the first time it will give…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.

816 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now