Solved

gawk field separators

Posted on 2014-02-16
2
244 Views
Last Modified: 2014-02-16
I have a shell script (below) that scans through directories, finding files ending in a .arf file extension (sample below), and then writes data from the file to the database (sample also below).  Everything works correctly except there are two time fields which have colons in the middle of the field so gawk is stripping them as well.  How can I change the field separator so that it will correctly process the fields with colons in them as well?

Shell script
#!/bin/bash

echo `date`
find . -name "*.arf" | while read f; do
  newpath="$(basename $(dirname "$f"))"
#/$(basename $f)"
  cat "$f" | gawk -F'[:"]' -v p="$newpath" '{ 
      nlist=nlist "`, `" $1;
      gsub("'\''","'\\\\\''", $3);
      vlist=vlist ", '\''" $3 "'\''";
  }
  END { 
    printf "insert into `database`.`archives` (`NEWPATH%s`) values ('\''%s'\''%s);\n", nlist, p, vlist;
  }' >> myinsertfile4.sql
#| tee -a myinsertfile4.sql
  cnt=$((cnt+1))
  [ $(($cnt%100)) -eq 0 ] && echo "File #$cnt: $f"
done

echo "Total Files: $cnt"

echo `date`

Open in new window


Sample .arf file:
FILEID: "TIF460222"
PATH: "/optical/incoming/TIF460222"
TYPE: "TIF"
SECLEV: "10"
STATID: ""
USRID: "admin"
REQDATE: "03/15/2012"
REQTIME: "08:43:32"
GENDATE: "03/16/2012"
GENTIME: "08:43:32"
PROGID: ""
GROUPID: "Drive up rec's"
DESC: "March"

Open in new window


Sample Output:
insert into `database`.`archives` (`NEWPATH`, `FILEID`, `PATH`, `TYPE`, `SECLEV`, `STATID`, `USRID`, `REQDATE`, `REQTIME`, `GENDATE`, `GENTIME`, `PROGID`, `GROUPID`, `DESC`) values ('.', 'TIF460222', '/optical/incoming/TIF460222', 'TIF', '10', '', 'admin', '03/15/2012', '08', '03/16/2012', '08', '', 'Drive up rec\'s', 'March');

Open in new window

0
Comment
Question by:bdhtechnology
2 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 39863171
New line #7:

cat "$f" | gawk -F': |"' -v p="$newpath" '{
...
...

It's a different way to specify the separator. The above means "colon space OR double quote"
0
 
LVL 1

Author Comment

by:bdhtechnology
ID: 39863222
Perfect, thanks again!
0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Over the last ten+ years I have seen Linux configuration tools come and go. In the early days there was the tried-and-true, all-powerful linuxconf that many thought would remain the one and only Linux configuration tool until the end of times. Well,…
SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…

820 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question