asked on

Check for files then combine .txt files

Hello,

I have a server where a company uploads multiple .txt files of inventory.

The problem is that I have to run a script for each file, and they are constantly adding them.

They are all the exact csv format.

Is it possible to check the folder for .txt files, and combine them into one file? I am concerned about the first line of the files as they are the column names.

Here is what I am using to load them one by one if this helps.

$loadsql = 'LOAD DATA INFILE "../../../var/www/vhosts/domain.com/web_users/company/inventory_1.txt" INTO TABLE data_processing
FIELDS TERMINATED BY "\",\"" LINES TERMINATED BY "\"\n\"" IGNORE 1 LINES (column_1,...

ahoffmann

ls *csv.txt|awk '(NR==1){print "cat "$0;next}{print "tail -n +2 "$0}'|sh>new-inventory.txt

movieprodw

ASKER

interesting, I am using a CRON to load a PHP file, is there a way for me to do that?

Locate any .txt file in the ../../../var/www/vhosts/domain.com/web_users/company directory then combine them removing the and save as compiled.txt without the first lines?

Thank you for the help

SOLUTION

ahoffmann

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ASKER CERTIFIED SOLUTION

Ray Paseur

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

movieprodw

ASKER

The client currently dumps 3 files (soon to be 20+) at 4am and the cron runs at 4:30am, currently I have it setup to select one at a time by name, I guess I could just put the file time in a db, stick them in an array then do a foreach.

Thanks for your help talking through this guys

movieprodw

ASKER

Thanks