[Webinar] Streamline your web hosting managementRegister Today

x
?
Solved

gawk field separators

Posted on 2014-02-16
2
Medium Priority
?
270 Views
Last Modified: 2014-02-16
I have a shell script (below) that scans through directories, finding files ending in a .arf file extension (sample below), and then writes data from the file to the database (sample also below).  Everything works correctly except there are two time fields which have colons in the middle of the field so gawk is stripping them as well.  How can I change the field separator so that it will correctly process the fields with colons in them as well?

Shell script
#!/bin/bash

echo `date`
find . -name "*.arf" | while read f; do
  newpath="$(basename $(dirname "$f"))"
#/$(basename $f)"
  cat "$f" | gawk -F'[:"]' -v p="$newpath" '{ 
      nlist=nlist "`, `" $1;
      gsub("'\''","'\\\\\''", $3);
      vlist=vlist ", '\''" $3 "'\''";
  }
  END { 
    printf "insert into `database`.`archives` (`NEWPATH%s`) values ('\''%s'\''%s);\n", nlist, p, vlist;
  }' >> myinsertfile4.sql
#| tee -a myinsertfile4.sql
  cnt=$((cnt+1))
  [ $(($cnt%100)) -eq 0 ] && echo "File #$cnt: $f"
done

echo "Total Files: $cnt"

echo `date`

Open in new window


Sample .arf file:
FILEID: "TIF460222"
PATH: "/optical/incoming/TIF460222"
TYPE: "TIF"
SECLEV: "10"
STATID: ""
USRID: "admin"
REQDATE: "03/15/2012"
REQTIME: "08:43:32"
GENDATE: "03/16/2012"
GENTIME: "08:43:32"
PROGID: ""
GROUPID: "Drive up rec's"
DESC: "March"

Open in new window


Sample Output:
insert into `database`.`archives` (`NEWPATH`, `FILEID`, `PATH`, `TYPE`, `SECLEV`, `STATID`, `USRID`, `REQDATE`, `REQTIME`, `GENDATE`, `GENTIME`, `PROGID`, `GROUPID`, `DESC`) values ('.', 'TIF460222', '/optical/incoming/TIF460222', 'TIF', '10', '', 'admin', '03/15/2012', '08', '03/16/2012', '08', '', 'Drive up rec\'s', 'March');

Open in new window

0
Comment
Question by:bdhtechnology
2 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 2000 total points
ID: 39863171
New line #7:

cat "$f" | gawk -F': |"' -v p="$newpath" '{
...
...

It's a different way to specify the separator. The above means "colon space OR double quote"
0
 
LVL 1

Author Comment

by:bdhtechnology
ID: 39863222
Perfect, thanks again!
0

Featured Post

Receive 1:1 tech help

Solve your biggest tech problems alongside global tech experts with 1:1 help.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Fine Tune your automatic Updates for Ubuntu / Debian
I have written articles previously comparing SARDU and YUMI.  I also included a couple of lines about Easy2boot (easy2boot.com).  I have now been using, and enjoying easy2boot as my sole multiboot utility for some years and realize that it deserves …
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Suggested Courses

607 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question