I have a daily job that recieves a file created by another group. The file is dropped into a directory for me to pickup and process using a unix korn shell script, KSH.
In the ksh scripts the file gets messaged (AWK) and then bcp'd in to sybase DB.
the file is supposed to have unique rows based on 4 fields. Sometimes we get "dup" recs. i need to check in the ksh for a dup.
if a dup occurs i will email the contact's and tell them to create a dup free file.
my main concern is how to identify the "dups"!