asked on

Bash shell script - batch process multiple files by command line

I have a bash script that works great when i process a single file by command line. I use

check 234234.txt

and it processes it great. I have a large directory full of .txt files that I want the script to process one by one and append all results to one single file for mysql import. I've tried doing check *.txt but it will only do the first file in the directory. Any ideas? Here is my current partial script.

if test $# -lt 1; then

echo "usage: check <filename>";
exit 1
fi

Thanks in advance!

ravenpl

You can iterate through args like

while [ -n "$1" ]; do
echo $1;
# do more job on $1
shift
done

cybrthug

ASKER

Hmm i tried working with this, it seems to see all the .txt files but it only processes the first one still, it does echo out each correct filename, but starts to reprocess the first .txt file in the directory after each echo. One thing that I do is append the filename to the end of each line when the script processes it, it uses the filename as an ID so i know where the line came from.

ravenpl

Can You show us the script itself, there's propably somewhere saved filename, so it uses same in each cycle.

cybrthug

ASKER

Here she is:

#!/bin/bash

FILE="$1"
stringZ=$FILE
FILEA=`echo ${stringZ:(+1)}`
FILEB=`echo ${FILEA:0:5}`

if test $# -lt 1; then
echo "usage: tabcon <filename>";
exit 1
fi

while [ -n "$1" ]; do
echo $1;
shift

#read the file

while read DATA ; do

#parse the leading line
PITEM=`echo "$DATA" | cut -d ' ' -f1`
PID=`echo "$DATA" | cut -d ' ' -f3`
QUANTITY=`echo "$DATA" | cut -d ' ' -f4`

DATA=`echo "$DATA" | cut -d ' ' -f11-` # -f5-

while [ "$QUANTITY" = "" ]; do
QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
done
while [ 1 ]; do
GREPCHECK=`echo "$QUANTITY" | grep ","`
if [ -z "$GREPCHECK" ]; then
QUANTITYFIX=`echo "$QUANTITY" | cut -d '.' -f1`
else
QUANTITYFIX=`echo "${QUANTITY//,/}" | cut -d '.' -f1`
fi
expr $QUANTITYFIX + 0 >/dev/null 2>&1
if [ $? -ne 0 ]; then
PID="$PID $QUANTITY"

QUANTITY=`echo "$DATA" | cut -d ' ' -f2`

DATA=`echo "$DATA" | cut -d ' ' -f2-`
while [ "$QUANTITY" = "" ]; do
QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
done
else
break
fi
done
UM=`echo "$DATA" | cut -d ' ' -f4-`

#read the next line
read DATA

#test for a blank line
while [ "$DATA" != "" ]; do
#parse the item
CID=`echo "$DATA" | cut -d ' ' -f1`
CNTR=`echo "$DATA" | cut -d ' ' -f2`
DATA=`echo "$DATA" | cut -d ' ' -f3-`
while [ "$CNTR" = " " ]; do
CNTR=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
done
QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
while [ "$QUANTITY" = " " ]; do
QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
done
while [ 1 ]; do

GREPCHECK=`echo "$QUANTITY" | grep ","`
if [ -z "$GREPCHECK" ]; then
QUANTITYFIX=`echo "$QUANTITY" | cut -d '.' -f1`
else
QUANTITYFIX=`echo "${QUANTITY//,/}" | cut -d '.' -f1`
fi
expr $QUANTITYFIX + 1 >/dev/null 2>&1

if [ $? -ne 0 ]; then
CNTR="$CNTR $QUANTITY"
QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
while [ "$QUANTITY" = "" ]; do
QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
done
else
break
fi
done
CHKPR=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
while [ "$CHKPR" = "" ]; do
CHKPR=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
done
TOTAL=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
while [ "$TOTAL" = "" ]; do
TOTAL=`echo "$DATA" | cut -d ' ' -f1`
DATA=`echo "$DATA" | cut -d ' ' -f2-`
done
echo "$CID;$PITEM;$QUANTITY;$UM;$CHKPR;$FILEB"

#read the next line
read DATA
done

#blank line
# echo ""

done < "$FILE"

#done

done

exit 0

ASKER CERTIFIED SOLUTION

ravenpl

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

cybrthug

ASKER

Simple enough, thank you so much, worked like a charm, just needed placement.