Improve company productivity with a Business Account.Sign Up

x
?
Solved

gawk field separators

Posted on 2014-02-16
2
Medium Priority
?
271 Views
Last Modified: 2014-02-16
I have a shell script (below) that scans through directories, finding files ending in a .arf file extension (sample below), and then writes data from the file to the database (sample also below).  Everything works correctly except there are two time fields which have colons in the middle of the field so gawk is stripping them as well.  How can I change the field separator so that it will correctly process the fields with colons in them as well?

Shell script
#!/bin/bash

echo `date`
find . -name "*.arf" | while read f; do
  newpath="$(basename $(dirname "$f"))"
#/$(basename $f)"
  cat "$f" | gawk -F'[:"]' -v p="$newpath" '{ 
      nlist=nlist "`, `" $1;
      gsub("'\''","'\\\\\''", $3);
      vlist=vlist ", '\''" $3 "'\''";
  }
  END { 
    printf "insert into `database`.`archives` (`NEWPATH%s`) values ('\''%s'\''%s);\n", nlist, p, vlist;
  }' >> myinsertfile4.sql
#| tee -a myinsertfile4.sql
  cnt=$((cnt+1))
  [ $(($cnt%100)) -eq 0 ] && echo "File #$cnt: $f"
done

echo "Total Files: $cnt"

echo `date`

Open in new window


Sample .arf file:
FILEID: "TIF460222"
PATH: "/optical/incoming/TIF460222"
TYPE: "TIF"
SECLEV: "10"
STATID: ""
USRID: "admin"
REQDATE: "03/15/2012"
REQTIME: "08:43:32"
GENDATE: "03/16/2012"
GENTIME: "08:43:32"
PROGID: ""
GROUPID: "Drive up rec's"
DESC: "March"

Open in new window


Sample Output:
insert into `database`.`archives` (`NEWPATH`, `FILEID`, `PATH`, `TYPE`, `SECLEV`, `STATID`, `USRID`, `REQDATE`, `REQTIME`, `GENDATE`, `GENTIME`, `PROGID`, `GROUPID`, `DESC`) values ('.', 'TIF460222', '/optical/incoming/TIF460222', 'TIF', '10', '', 'admin', '03/15/2012', '08', '03/16/2012', '08', '', 'Drive up rec\'s', 'March');

Open in new window

0
Comment
Question by:bdhtechnology
2 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 2000 total points
ID: 39863171
New line #7:

cat "$f" | gawk -F': |"' -v p="$newpath" '{
...
...

It's a different way to specify the separator. The above means "colon space OR double quote"
0
 
LVL 1

Author Comment

by:bdhtechnology
ID: 39863222
Perfect, thanks again!
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Virtualization software lets you run different versions of Windows, Ubuntu Linux and other versions of Linux all at the same time, rather than running each one directly from your computer's hard drive.
Cron is one of the most popular and basic utilities found on Unix systems. Combined with other tools, cron makes it exceptionally easy to automate a broad range of tasks on your server.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

589 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question