Avatar of movieprodw
movieprodw
 asked on

Check for files then combine .txt files

Hello,

I have a server where a company uploads multiple .txt files of inventory.

The problem is that I have to run a script for each file, and they are constantly adding them.

They are all the exact csv format.

Is it possible to check the folder for .txt files, and combine them into one file? I am concerned about the first line of the files as they are the column names.

Here is what I am using to load them one by one if this helps.

$loadsql = 'LOAD DATA INFILE "../../../var/www/vhosts/domain.com/web_users/company/inventory_1.txt" INTO TABLE data_processing
FIELDS TERMINATED BY "\",\"" LINES TERMINATED BY "\"\n\"" IGNORE 1 LINES (column_1,...
PHP

Avatar of undefined
Last Comment
movieprodw

8/22/2022 - Mon
ahoffmann

ls *csv.txt|awk '(NR==1){print "cat "$0;next}{print "tail -n +2 "$0}'|sh>new-inventory.txt
movieprodw

ASKER
interesting, I am using a CRON to load a PHP file, is there a way for me to do that?

Locate any .txt file in the ../../../var/www/vhosts/domain.com/web_users/company directory then combine them removing the and save as compiled.txt without the first lines?

Thank you for the help
SOLUTION
ahoffmann

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question
ASKER CERTIFIED SOLUTION
Log in to continue reading
Log In
Sign up - Free for 7 days
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
movieprodw

ASKER
The client currently dumps 3 files (soon to be 20+) at 4am and the cron runs at 4:30am, currently I have it setup to select one at a time by name, I guess I could just put the file time in a db, stick them in an array then do a foreach.

Thanks for your help talking through this guys
Experts Exchange has (a) saved my job multiple times, (b) saved me hours, days, and even weeks of work, and often (c) makes me look like a superhero! This place is MAGIC!
Walt Forbes
movieprodw

ASKER
Thanks