Linux scripting to consolidate separate files


   We get some postgres db command files (Insert, Update, Create, Drop etc) with the DB commands in them on a location(\usr\DB Scripts) in our Linux environment.
We need to come up with a script, which when ran,
   -> creates a consolidated file, with the statements in each file, ended with a GO statement
   -> Move the individual files used in the above consolidated file, into an archive directory, with timestamp in the name.
   -> Leave any unused files in the original location (\usr\DB Scripts); These will be the files which may have been added just after the script was run
   -> Run another script which compiles the consolidated file, against the database.

Can someone kindly help with this? I'm fairly new to the linux world.
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

nociSoftware EngineerCommented:
say all files are named: pg-cmd-nnnnnnn.sql
where nnnnnnn is a unique item in some directory the the following will do::

Something along....


mkdir t
echo >t/sqlscript.sql
DATE=$( date +%Y%m%d-%H%M%S )
find . -name 'pg-cmd-*.sql'  -exec bash -c "cat {} >>t/sqlscript.sql " \;  -exec mv '{}'  /archive/$DATE-'{}' \;
(echo "" ; echo "GO;" ) >> t/sqlscript.sql
RUNCOMMAND t/sqlscript.sql

Open in new window


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Some minor comments on noci's solution:

In line 4 noci is:
    echo >t/sqlscript.sql
That will create the file with a newline in it.  If there is no need to start the file with a newline, you can do this instead:
    echo -n >t/sqlscript.sql
or even just:
both of which will create an empty file.

And line 7 is:
    (echo "" ; echo "GO;" ) >> t/sqlscript.sql
can be more simply be written like this:
    echo -e "\nGO;" >>t/sqlscript.sql

But I doubt that any of the above will make any difference to whether noci's solution works.
nociSoftware EngineerCommented:
@tel2, yes youre remarks are right.
Except that i wanted to keep it as simple as possible... and not pull too many tricks in the equation.
For the trailing go an empty line feed either echo "" or echo -e "\n" is needed because not all files end with a linefeed before end-of-file.

Also not mentioned before is that the asker should put his own activation script where RUNSCRIPT is mentioned here.
Hi noci,

> "Except that i wanted to keep it as simple as possible... and not pull too many tricks in the equation."
OK, but it's debatable whether this:
    (echo "" ; echo "GO;" ) >> t/sqlscript.sql
is more simple than this:
    echo -e "\nGO;" >>t/sqlscript.sql
I find the later simpler...and more concise.  But I accept that others may have different opinions.
pvsbandiAuthor Commented:
Thank you, both!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.