Linux RedHat, Inserting lines into file with a list from other file.

Hi experts, I need to do an Insert of many lines into a file called test.ctl, before APPEND line:
LOAD DATA
----> here should be new lines <-----
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
SCADA_ID CHAR,
SCADA_SISTEMA CHAR,
SCADA_OPERATION_CLASS CHAR,
SCADA_OPERATION_SUBCLASS CHAR,
SCADA_OPERATION_DATE ,
SCADA_DATE_OFFSET ,
SCADA_OPERATION_TIMESTAMP "TO_TIMESTAMP(RPAD(SUBSTR(:SCADA_OPERATION_TIMESTAMP,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_PROCESS_STATUS ,
CERTA_PROCESS_DATE "TO_TIMESTAMP(RPAD(SUBSTR(:CERTA_PROCESS_DATE,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_OPERATION_ID ,
CERTA_NOTES CHAR,
INT_ID
 )

Open in new window

But the lines must be obtained from a file called list.lst.
He has lines like:
Transfer_data.20141118194103
Transfer_data.20141118194132

Open in new window

concatenated with string INFILE. then it should look like this:
LOAD DATA
INFILE Transfer_data.20141118194103
INFILE Transfer_data.20141118194132  
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
SCADA_ID CHAR,
SCADA_SISTEMA CHAR,
SCADA_OPERATION_CLASS CHAR,
SCADA_OPERATION_SUBCLASS CHAR,
SCADA_OPERATION_DATE ,
SCADA_DATE_OFFSET ,
SCADA_OPERATION_TIMESTAMP "TO_TIMESTAMP(RPAD(SUBSTR(:SCADA_OPERATION_TIMESTAMP,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_PROCESS_STATUS ,
CERTA_PROCESS_DATE "TO_TIMESTAMP(RPAD(SUBSTR(:CERTA_PROCESS_DATE,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_OPERATION_ID ,
CERTA_NOTES CHAR,
INT_ID
 )

Open in new window

I tried with a script ksh:
ls -1 Transfer_data* > lista.lst
line=$(cat test.ctl | grep -n 'APPEND' | grep -o '^[0-9]*')
line=$((line -1))
for f in `cat  lista.lst`
   do
       sed -i ${line}' i\INFILE "$f"' 'test.ctl'
   done

Open in new window

But the lines inserted are in any place, and with wrong sintaxis:
INFILE "$f"
INFILE "$f"
LOAD DATA
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
etc...

Open in new window

Note ' INFILE "$f" 'on first and second place.

Can you help me find an alternative?
Thanks and regards
carlino70Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Gerwin Jansen, EE MVETopic Advisor Commented:
Try this:

sed -i '/LOAD DATA/ r list.lst' test.ctl

This will add the lines in list.lst into test.ctl after the LOAD DATA line.
0
woolmilkporcCommented:
awk '/APPEND/ {l=$0; while ((getline < "list.lst") > 0) {print "INFILE",$0}; print l; next} {print}' test.ctl
0
woolmilkporcCommented:
Since awk cannot change files "in place" add ">testnew.ctl" to write the changed content to a new file:

awk '/APPEND/ {l=$0; while ((getline < "list.lst") > 0) {print "INFILE",$0}; print l; next} {print}' test.ctl > testnew.ctl

By the way, a few modifications applied (mostly " instead of ' and the subtraction removed) your script will work, too:

ls -1 Transfer_data* > lista.lst
line=$(grep -n 'APPEND' test.ctl | grep -o '^[0-9]*')
for f in $(cat  lista.lst)
   do
       sed  -i "${line} i\INFILE $f" test.ctl
   done
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
The Ultimate Tool Kit for Technolgy Solution Provi

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy for valuable how-to assets including sample agreements, checklists, flowcharts, and more!

carlino70Author Commented:
woolmilkporc, your suggestion is ok, with a modification:
ls -1 Transfer_data* > lista.lst
line=$(cat test.ctl | grep -n 'APPEND' | grep -o '^[0-9]*')
line=$((line -0))                                                                   #here I do a change
     for f in $(cat  lista.lst)
        do
           sed  -i ${line}" i\INFILE $f" test.ctl
          # sqlldr epe/epe@xa21 control=test.ctl log=sqlldr.log
        done

Open in new window

But when test.ctl is filled, the order is distinct that lista.lst:
see lista.ctl
Transfer_data.20141118194103
Transfer_data.20141118194132

Open in new window

and see the order inside test.ctl
INFILE Transfer_data.20141118194132
INFILE Transfer_data.20141118194103

Open in new window

How I can reverse the order?
Thanks
0
woolmilkporcCommented:
It's quite normal that the order gets reversed - we read the list file line by line and insert that respective line just above "APPEND", i.e we read top-down and insert bottom-up.

We can use the "tac" utiity (on Linux only) instead of "cat" to reverse the order of the lines as they're presented to "for".
Line #4 of the last version you posted should then read:

 for f in $(tac  lista.lst)

Another option: Reverse the order in lista.lst by instructing "ls" to display the newest file first: "ls -t1 ...".

Obviously you should use just one of the above suggestions at a time!

By the way, why would you want to subtract "0" from "line"? The variable $line should be numeric ab ovo, so no need for an arithmetic NOP.

And how about my "awk" solution? Here the file is inserted as is above "APPEND", so the order will be kept automatically. Is it because "awk" cannot change "in place"?

Finally, the "sqlldr" command should not appear inside the "for" loop, because test.ctl will not contain all "INFILE" lines from lista.lst before the last iteration has completed!
0
Gerwin Jansen, EE MVETopic Advisor Commented:
The 'sed' suggestion I made does not reverse order and is much shorter as well ;) So why don't you use that?
0
carlino70Author Commented:
woolmilkporc,

I appreciate your attention and suggestions. Finally, use the option "ls -t1", and so far it works.

Also, I took the loop sentence "sqlldr ...." was my mistake.

Thanks again.
0
carlino70Author Commented:
Gerwin Jansen, your suggestion not insert lines in test.ctl

Thanks anyway
0
Gerwin Jansen, EE MVETopic Advisor Commented:
>> Gerwin Jansen, your suggestion not insert lines in test.ctl
Sure it did, did you try my suggestion at all?

[gerwin@localhost ~]$ cat test.ctl
LOAD DATA
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
SCADA_ID CHAR,
SCADA_SISTEMA CHAR,
SCADA_OPERATION_CLASS CHAR,
SCADA_OPERATION_SUBCLASS CHAR,
SCADA_OPERATION_DATE ,
SCADA_DATE_OFFSET ,
SCADA_OPERATION_TIMESTAMP "TO_TIMESTAMP(RPAD(SUBSTR(:SCADA_OPERATION_TIMESTAMP,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_PROCESS_STATUS ,
CERTA_PROCESS_DATE "TO_TIMESTAMP(RPAD(SUBSTR(:CERTA_PROCESS_DATE,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_OPERATION_ID ,
CERTA_NOTES CHAR,
INT_ID
 )
[gerwin@localhost ~]$ cat list.lst
Transfer_data.20141118194103
Transfer_data.20141118194132
[gerwin@localhost ~]$ sed -i '/LOAD DATA/ r list.lst' test.ctl
[gerwin@localhost ~]$ cat test.ctl
LOAD DATA
Transfer_data.20141118194103
Transfer_data.20141118194132
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
SCADA_ID CHAR,
SCADA_SISTEMA CHAR,
SCADA_OPERATION_CLASS CHAR,
SCADA_OPERATION_SUBCLASS CHAR,
SCADA_OPERATION_DATE ,
SCADA_DATE_OFFSET ,
SCADA_OPERATION_TIMESTAMP "TO_TIMESTAMP(RPAD(SUBSTR(:SCADA_OPERATION_TIMESTAMP,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_PROCESS_STATUS ,
CERTA_PROCESS_DATE "TO_TIMESTAMP(RPAD(SUBSTR(:CERTA_PROCESS_DATE,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_OPERATION_ID ,
CERTA_NOTES CHAR,
INT_ID
 )
[gerwin@localhost ~]$

Open in new window

0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.