Bash shell script - batch process multiple files by command line

I have a bash script that works great when i process a single file by command line. I use

check 234234.txt

and it processes it great. I have a large directory full of .txt files that I want the script to process one by one and append all results to one single file for mysql import. I've tried doing check *.txt but it will only do the first file in the directory. Any ideas? Here is my current partial script.

if test $# -lt 1; then

        echo "usage: check <filename>";
        exit 1
fi


Thanks in advance!
cybrthugAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ravenplCommented:
You can iterate through args like

while [ -n "$1" ]; do
    echo $1;
    # do more job on $1
    shift
done
0
cybrthugAuthor Commented:
Hmm i tried working with this, it seems to see all the .txt files but it only processes the first one still, it does echo out each correct filename, but starts to reprocess the first .txt file in the directory after each echo. One thing that I do is append the filename to the end of each line when the script processes it, it uses the filename as an ID so i know where the line came from.
0
ravenplCommented:
Can You show us the script itself, there's propably somewhere saved filename, so it uses same in each cycle.
0
Cloud Class® Course: Microsoft Windows 7 Basic

This introductory course to Windows 7 environment will teach you about working with the Windows operating system. You will learn about basic functions including start menu; the desktop; managing files, folders, and libraries.

cybrthugAuthor Commented:
Here she is:

#!/bin/bash

FILE="$1"
stringZ=$FILE
FILEA=`echo ${stringZ:(+1)}`
FILEB=`echo ${FILEA:0:5}`

if test $# -lt 1; then
        echo "usage: tabcon <filename>";
        exit 1
fi


while [ -n "$1" ]; do
    echo $1;
    shift

#read the file

while read DATA ; do

  #parse the leading line
  PITEM=`echo "$DATA" | cut -d ' ' -f1`
  PID=`echo "$DATA" | cut -d ' ' -f3`
  QUANTITY=`echo "$DATA" | cut -d ' ' -f4`

  DATA=`echo "$DATA" | cut -d ' ' -f11-`  # -f5-

  while [ "$QUANTITY" = "" ]; do
    QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
    DATA=`echo "$DATA" | cut -d ' ' -f2-`
  done
  while [ 1 ]; do
  GREPCHECK=`echo "$QUANTITY" | grep ","`
  if [ -z "$GREPCHECK" ]; then
    QUANTITYFIX=`echo "$QUANTITY" | cut -d '.' -f1`
  else
    QUANTITYFIX=`echo "${QUANTITY//,/}" | cut -d '.' -f1`
  fi
  expr $QUANTITYFIX + 0 >/dev/null 2>&1
  if [ $? -ne 0 ]; then
    PID="$PID $QUANTITY"

    QUANTITY=`echo "$DATA" | cut -d ' ' -f2`

    DATA=`echo "$DATA" | cut -d ' ' -f2-`
    while [ "$QUANTITY" = "" ]; do
      QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
      DATA=`echo "$DATA" | cut -d ' ' -f2-`
    done
  else
 break
  fi
  done
  UM=`echo "$DATA" | cut -d ' ' -f4-`

#read the next line
  read DATA

  #test for a blank line
  while [ "$DATA" != "" ]; do
    #parse the item
    CID=`echo "$DATA" | cut -d ' ' -f1`
    CNTR=`echo "$DATA" | cut -d ' ' -f2`
    DATA=`echo "$DATA" | cut -d ' ' -f3-`
    while [ "$CNTR" = " " ]; do
      CNTR=`echo "$DATA" | cut -d ' ' -f1`
      DATA=`echo "$DATA" | cut -d ' ' -f2-`
    done
    QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
    DATA=`echo "$DATA" | cut -d ' ' -f2-`
    while [ "$QUANTITY" = " " ]; do
      QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
      DATA=`echo "$DATA" | cut -d ' ' -f2-`
    done
  while [ 1 ]; do

  GREPCHECK=`echo "$QUANTITY" | grep ","`
  if [ -z "$GREPCHECK" ]; then
    QUANTITYFIX=`echo "$QUANTITY" | cut -d '.' -f1`
  else
    QUANTITYFIX=`echo "${QUANTITY//,/}" | cut -d '.' -f1`
  fi
  expr $QUANTITYFIX + 1 >/dev/null 2>&1

  if [ $? -ne 0 ]; then
  CNTR="$CNTR $QUANTITY"
  QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
  DATA=`echo "$DATA" | cut -d ' ' -f2-`
  while [ "$QUANTITY" = "" ]; do
    QUANTITY=`echo "$DATA" | cut -d ' ' -f1`
    DATA=`echo "$DATA" | cut -d ' ' -f2-`
  done
else
break
fi
done
    CHKPR=`echo "$DATA" | cut -d ' ' -f1`
    DATA=`echo "$DATA" | cut -d ' ' -f2-`
    while [ "$CHKPR" = "" ]; do
      CHKPR=`echo "$DATA" | cut -d ' ' -f1`
      DATA=`echo "$DATA" | cut -d ' ' -f2-`
    done
    TOTAL=`echo "$DATA" | cut -d ' ' -f1`
    DATA=`echo "$DATA" | cut -d ' ' -f2-`
    while [ "$TOTAL" = "" ]; do
      TOTAL=`echo "$DATA" | cut -d ' ' -f1`
      DATA=`echo "$DATA" | cut -d ' ' -f2-`
    done
   echo "$CID;$PITEM;$QUANTITY;$UM;$CHKPR;$FILEB"

 #read the next line
    read DATA
  done

  #blank line
#  echo ""

done < "$FILE"

#done

done

exit 0
0
ravenplCommented:
> FILE="$1"
> stringZ=$FILE
> FILEA=`echo ${stringZ:(+1)}`
> FILEB=`echo ${FILEA:0:5}`
here You set FILE variable, which is used then in all cycles. Right? Move this line to after the 'while [ -n "$1" ]; do'

also, 'shift' should be last statement in the 'while [ -n "$1" ]; do' loop, or at least should be after FILE names initializations. The 'shift' statement renames parameters $2 -> $1, $3 -> $2 and so on... so You always use $1 ;)

Is it clear enought?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
cybrthugAuthor Commented:
Simple enough, thank you so much, worked like a charm, just needed placement.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.