Go Premium for a chance to win a PS4. Enter to Win

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 196
  • Last Modified:

Linux RedHat, Inserting lines into file with a list from other file.

Hi experts, I need to do an Insert of many lines into a file called test.ctl, before APPEND line:
LOAD DATA
----> here should be new lines <-----
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
SCADA_ID CHAR,
SCADA_SISTEMA CHAR,
SCADA_OPERATION_CLASS CHAR,
SCADA_OPERATION_SUBCLASS CHAR,
SCADA_OPERATION_DATE ,
SCADA_DATE_OFFSET ,
SCADA_OPERATION_TIMESTAMP "TO_TIMESTAMP(RPAD(SUBSTR(:SCADA_OPERATION_TIMESTAMP,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_PROCESS_STATUS ,
CERTA_PROCESS_DATE "TO_TIMESTAMP(RPAD(SUBSTR(:CERTA_PROCESS_DATE,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_OPERATION_ID ,
CERTA_NOTES CHAR,
INT_ID
 )

Open in new window

But the lines must be obtained from a file called list.lst.
He has lines like:
Transfer_data.20141118194103
Transfer_data.20141118194132

Open in new window

concatenated with string INFILE. then it should look like this:
LOAD DATA
INFILE Transfer_data.20141118194103
INFILE Transfer_data.20141118194132  
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
SCADA_ID CHAR,
SCADA_SISTEMA CHAR,
SCADA_OPERATION_CLASS CHAR,
SCADA_OPERATION_SUBCLASS CHAR,
SCADA_OPERATION_DATE ,
SCADA_DATE_OFFSET ,
SCADA_OPERATION_TIMESTAMP "TO_TIMESTAMP(RPAD(SUBSTR(:SCADA_OPERATION_TIMESTAMP,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_PROCESS_STATUS ,
CERTA_PROCESS_DATE "TO_TIMESTAMP(RPAD(SUBSTR(:CERTA_PROCESS_DATE,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_OPERATION_ID ,
CERTA_NOTES CHAR,
INT_ID
 )

Open in new window

I tried with a script ksh:
ls -1 Transfer_data* > lista.lst
line=$(cat test.ctl | grep -n 'APPEND' | grep -o '^[0-9]*')
line=$((line -1))
for f in `cat  lista.lst`
   do
       sed -i ${line}' i\INFILE "$f"' 'test.ctl'
   done

Open in new window

But the lines inserted are in any place, and with wrong sintaxis:
INFILE "$f"
INFILE "$f"
LOAD DATA
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
etc...

Open in new window

Note ' INFILE "$f" 'on first and second place.

Can you help me find an alternative?
Thanks and regards
0
carlino70
Asked:
carlino70
  • 3
  • 3
  • 3
1 Solution
 
Gerwin Jansen, EE MVETopic Advisor Commented:
Try this:

sed -i '/LOAD DATA/ r list.lst' test.ctl

This will add the lines in list.lst into test.ctl after the LOAD DATA line.
0
 
woolmilkporcCommented:
awk '/APPEND/ {l=$0; while ((getline < "list.lst") > 0) {print "INFILE",$0}; print l; next} {print}' test.ctl
0
 
woolmilkporcCommented:
Since awk cannot change files "in place" add ">testnew.ctl" to write the changed content to a new file:

awk '/APPEND/ {l=$0; while ((getline < "list.lst") > 0) {print "INFILE",$0}; print l; next} {print}' test.ctl > testnew.ctl

By the way, a few modifications applied (mostly " instead of ' and the subtraction removed) your script will work, too:

ls -1 Transfer_data* > lista.lst
line=$(grep -n 'APPEND' test.ctl | grep -o '^[0-9]*')
for f in $(cat  lista.lst)
   do
       sed  -i "${line} i\INFILE $f" test.ctl
   done
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
carlino70Author Commented:
woolmilkporc, your suggestion is ok, with a modification:
ls -1 Transfer_data* > lista.lst
line=$(cat test.ctl | grep -n 'APPEND' | grep -o '^[0-9]*')
line=$((line -0))                                                                   #here I do a change
     for f in $(cat  lista.lst)
        do
           sed  -i ${line}" i\INFILE $f" test.ctl
          # sqlldr epe/epe@xa21 control=test.ctl log=sqlldr.log
        done

Open in new window

But when test.ctl is filled, the order is distinct that lista.lst:
see lista.ctl
Transfer_data.20141118194103
Transfer_data.20141118194132

Open in new window

and see the order inside test.ctl
INFILE Transfer_data.20141118194132
INFILE Transfer_data.20141118194103

Open in new window

How I can reverse the order?
Thanks
0
 
woolmilkporcCommented:
It's quite normal that the order gets reversed - we read the list file line by line and insert that respective line just above "APPEND", i.e we read top-down and insert bottom-up.

We can use the "tac" utiity (on Linux only) instead of "cat" to reverse the order of the lines as they're presented to "for".
Line #4 of the last version you posted should then read:

 for f in $(tac  lista.lst)

Another option: Reverse the order in lista.lst by instructing "ls" to display the newest file first: "ls -t1 ...".

Obviously you should use just one of the above suggestions at a time!

By the way, why would you want to subtract "0" from "line"? The variable $line should be numeric ab ovo, so no need for an arithmetic NOP.

And how about my "awk" solution? Here the file is inserted as is above "APPEND", so the order will be kept automatically. Is it because "awk" cannot change "in place"?

Finally, the "sqlldr" command should not appear inside the "for" loop, because test.ctl will not contain all "INFILE" lines from lista.lst before the last iteration has completed!
0
 
Gerwin Jansen, EE MVETopic Advisor Commented:
The 'sed' suggestion I made does not reverse order and is much shorter as well ;) So why don't you use that?
0
 
carlino70Author Commented:
woolmilkporc,

I appreciate your attention and suggestions. Finally, use the option "ls -t1", and so far it works.

Also, I took the loop sentence "sqlldr ...." was my mistake.

Thanks again.
0
 
carlino70Author Commented:
Gerwin Jansen, your suggestion not insert lines in test.ctl

Thanks anyway
0
 
Gerwin Jansen, EE MVETopic Advisor Commented:
>> Gerwin Jansen, your suggestion not insert lines in test.ctl
Sure it did, did you try my suggestion at all?

[gerwin@localhost ~]$ cat test.ctl
LOAD DATA
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
SCADA_ID CHAR,
SCADA_SISTEMA CHAR,
SCADA_OPERATION_CLASS CHAR,
SCADA_OPERATION_SUBCLASS CHAR,
SCADA_OPERATION_DATE ,
SCADA_DATE_OFFSET ,
SCADA_OPERATION_TIMESTAMP "TO_TIMESTAMP(RPAD(SUBSTR(:SCADA_OPERATION_TIMESTAMP,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_PROCESS_STATUS ,
CERTA_PROCESS_DATE "TO_TIMESTAMP(RPAD(SUBSTR(:CERTA_PROCESS_DATE,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_OPERATION_ID ,
CERTA_NOTES CHAR,
INT_ID
 )
[gerwin@localhost ~]$ cat list.lst
Transfer_data.20141118194103
Transfer_data.20141118194132
[gerwin@localhost ~]$ sed -i '/LOAD DATA/ r list.lst' test.ctl
[gerwin@localhost ~]$ cat test.ctl
LOAD DATA
Transfer_data.20141118194103
Transfer_data.20141118194132
APPEND
INTO TABLE "INT_OPERATIONS"
FIELDS TERMINATED BY '!'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
SCADA_ID CHAR,
SCADA_SISTEMA CHAR,
SCADA_OPERATION_CLASS CHAR,
SCADA_OPERATION_SUBCLASS CHAR,
SCADA_OPERATION_DATE ,
SCADA_DATE_OFFSET ,
SCADA_OPERATION_TIMESTAMP "TO_TIMESTAMP(RPAD(SUBSTR(:SCADA_OPERATION_TIMESTAMP,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_PROCESS_STATUS ,
CERTA_PROCESS_DATE "TO_TIMESTAMP(RPAD(SUBSTR(:CERTA_PROCESS_DATE,1,21),24,'0'),'dd/mm/yyyy hh24:mi:ss.ff)')",
CERTA_OPERATION_ID ,
CERTA_NOTES CHAR,
INT_ID
 )
[gerwin@localhost ~]$

Open in new window

0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

  • 3
  • 3
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now