Solved

need help with awk or sed writing a bash script to find a delete an object in a flat database file

Posted on 2012-03-14
9
624 Views
Last Modified: 2012-06-06
I need help removing the following text from a file. It's an object in a flat file database. I need to find     : (VSXSERVER_VSNAME      and delete the contents from : to the closing bracket )

I usually do this in vi by putting the cursor on the : and then hitting d then %

This datbase has a number of other objects that I want to leave alone.

I want to define VSXSERVER_VSNAME as an enviroment variable so I can create a loop to run through a list object names.

I have this script that I used to delete the contents between ( and ) for :certificates but I'm not familar enough with awk to modify it to complete this task.

please help!

cat $FWDIR/conf/objects_5_0.C | awk 'BEGIN {c=0};  /:certificates \(\)/ {print;next}; /:certificates \(/ {c=1;print;next}; c>0 &&  /\(/ {c++}; c>0 && /\)/ {c--}; c<=0 {print}' >> $FWDIR/conf/objects_5_0.C.new




: (VSXSERVER_VSNAME
                  :AdminInfo (
                        :chkpf_uid ("{15886D8F-31B2-491A-BE18-D63DE2AA8599}")
                        :ClassName (vs_cluster_member)
                        :table (network_objects)
                        :LastModified (
                              :Time ("Sun Dec 18 08:08:07 2011")
                              :last_modified_utc (1324195687)
                              :By (localhost)
                              :From (GUICLIENT)
                        )
                        :icon ("NetworkObjects/vsx/vs_member")
                        :name (VSXSERVER_VSNAME)
                  )
                  :certificates ()
                  :edges ()
                  :free_fields ()
                  :interfaces (
                        :0 (
                              :AdminInfo (
                                    :chkpf_uid ("{9115D02E-94C6-4514-9DA5-2D3E6A61E0BF}")
                                    :ClassName (interface)
                              )
                              :edges ()
                              :bandwidth ()
                              :comments ()
                              :description ()
                              :dual_wan (false)
                              :dynamic_ip (false)
                              :ifindex (0)
                              :ip_pool_allocation ()
                              :ipaddr (192.168.1.1)
                              :ipv6_address ()
                              :ipv6_prefix (64)
                              :monitored_by_cluster (false)
                              :multicast_enforcement ()
                              :netmask (255.255.255.240)
                              :officialname (wrp1536)
                              :antispoof (true)
                              :netaccess (
                                    :AdminInfo (
                                          :chkpf_uid ("{DA3E1225-98EE-4E44-96F8-569056DE6710}")
                                          :ClassName (netaccess)
                                    )
                                    :access (undefined)
                                    :allowed ()
                                    :dmz (false)
                                    :enable_overlapping_nat (false)
                                    :force_policy (true)
                                    :internal_type (undefined)
                                    :leads_to_internet (false)
                                    :log (log)
                                    :monitor_anti_spoofing (false)
                                    :overlap_nat_dst_ipaddr ()
                                    :overlap_nat_netmask (255.255.255.0)
                                    :overlap_nat_src_ipaddr ()
                                    :perform_anti_spoofing (false)
                              )
                              :security (
                                    :AdminInfo (
                                          :chkpf_uid ("{9115D02E-94C6-4514-9DA5-2D3E6A61E0BF}")
                                          :ClassName (interface_security)
                                    )
                              )
                              :security_zone ()
                              :shared (true)
                              :vpn_trusted (false)
                        )
                        :1 (
                              :AdminInfo (
                                    :chkpf_uid ("{117509D3-EC8B-4F95-9DAB-8C5BFCE2BAC9}")
                                    :ClassName (interface)
                              )
                              :edges ()
                              :bandwidth ()
                              :comments ()
                              :description ()
                              :dual_wan (false)
                              :dynamic_ip (false)
                              :ifindex (1)
                              :ip_pool_allocation ()
                              :ipaddr (192.168.196.18)
                              :ipv6_address ()
                              :ipv6_prefix (64)
                              :monitored_by_cluster (false)
                              :multicast_enforcement ()
                              :netmask (255.255.255.240)
                              :officialname (Lan1.210)
                              :antispoof (true)
                              :netaccess (
                                    :AdminInfo (
                                          :chkpf_uid ("{1A1DA713-7619-4CBE-B264-81C3FB8BD345}")
                                          :ClassName (netaccess)
                                    )
                                    :access (undefined)
                                    :allowed ()
                                    :dmz (false)
                                    :enable_overlapping_nat (false)
                                    :force_policy (true)
                                    :internal_type (undefined)
                                    :leads_to_internet (false)
                                    :log (log)
                                    :monitor_anti_spoofing (false)
                                    :overlap_nat_dst_ipaddr ()
                                    :overlap_nat_netmask (255.255.255.0)
                                    :overlap_nat_src_ipaddr ()
                                    :perform_anti_spoofing (false)
                              )
                              :security (
                                    :AdminInfo (
                                          :chkpf_uid ("{117509D3-EC8B-4F95-9DAB-8C5BFCE2BAC9}")
                                          :ClassName (interface_security)
                                    )
                              )
                              :security_zone ()
                              :shared (true)
                              :vpn_trusted (false)
                        )
                        :2 (
                              :AdminInfo (
                                    :chkpf_uid ("{B8642EC1-CE35-434B-8B6F-20935B16ED49}")
                                    :ClassName (interface)
                              )
                              :edges ()
                              :bandwidth ()
                              :comments ()
                              :description ()
                              :dual_wan (false)
                              :dynamic_ip (false)
                              :ifindex (2)
                              :ip_pool_allocation ()
                              :ipaddr (192.168.1.2)
                              :ipv6_address ()
                              :ipv6_prefix (64)
                              :monitored_by_cluster (false)
                              :multicast_enforcement ()
                              :netmask (255.255.255.240)
                              :officialname (Lan1.280)
                              :antispoof (true)
                              :netaccess (
                                    :AdminInfo (
                                          :chkpf_uid ("{79A2A5FF-29FC-4F5A-9A8D-4066EB98D6E2}")
                                          :ClassName (netaccess)
                                    )
                                    :access (undefined)
                                    :allowed ()
                                    :dmz (false)
                                    :enable_overlapping_nat (false)
                                    :force_policy (true)
                                    :internal_type (undefined)
                                    :leads_to_internet (false)
                                    :log (log)
                                    :monitor_anti_spoofing (false)
                                    :overlap_nat_dst_ipaddr ()
                                    :overlap_nat_netmask (255.255.255.0)
                                    :overlap_nat_src_ipaddr ()
                                    :perform_anti_spoofing (false)
                              )
                              :security (
                                    :AdminInfo (
                                          :chkpf_uid ("{B8642EC1-CE35-434B-8B6F-20935B16ED49}")
                                          :ClassName (interface_security)
                                    )
                              )
                              :security_zone ()
                              :shared (true)
                              :vpn_trusted (false)
                        )
                  )
                  :Machine_weight (1)
                  :NAT ()
                  :accessible_from_smc (true)
                  :add_adtr_rule (false)
                  :color (black)
                  :comments ()
                  :connection_state (initialized)
                  :ics_update_supported (false)
                  :ip_pool_nat (false)
                  :ip_pool_office_mode (false)
                  :ip_pool_securemote_allocation_name ()
                  :ipaddr (0.0.0.0)
                  :is_bridge (false)
                  :junction (false)
                  :mgmt_ip (10.0.0.251)
                  :om_ippool ()
                  :owner ()
                  :sic_identifier (
                        :AdminInfo (
                              :chkpf_uid ("{6232F2DE-F76D-1180-9785-0A6015905355}")
                              :ClassName (gw_sic_identifier)
                        )
                        :id_type (ip_addr)
                        :id_value ()
                  )
                  :sic_name ("CN=VSXSERVER_VSNAME,O=MAIL-TEST.web.mgmtserver.org.selse4")
                  :type (cluster_member)
                  :vs_cluster_member (true)
            )
0
Comment
Question by:Dooglave
  • 4
  • 3
  • 2
9 Comments
 
LVL 76

Expert Comment

by:arnold
Comment Utility
The items within // are a pattern match. \ is an escape to note that the character that follows should be taken as is and not converted/interpreted.
So the match is for :certificates () the second looks for items in parenthesis.

You can use /:VX

The info you posted, is that the entire data that should be deleted?
0
 
LVL 6

Author Comment

by:Dooglave
Comment Utility
yeah the whole thing should be deleted. Eveything between : and the close bracket )
0
 
LVL 76

Expert Comment

by:arnold
Comment Utility
Is the structure the same as you have
:(entry
 dat
   subdata 
     subsubdata)
:(new entry
 dat
   subdata 
     subsubdata
)
:(another entry

Open in new window

both examples are valid as well as another where there is a space between the last data entry and the closing parenthesis?
I.e. starting at the line :(pattern all the way till the first new pattern :(newpattern.
Personally I find using PERL simpler to deal with text modifications.
Is that an option?
0
 
LVL 6

Author Comment

by:Dooglave
Comment Utility
sure I need to learn me some pearl anyways just haven't had the time. This would be a good start.

Yes the structure is the same.

:(object name
)
:(another object name

But their are other variations of object feilds I don't want to touch like for example there are indexes like. :2 (, and :something() and so on.

So I think I need to find the :(specific name
then count the open and close brackets.

for example ( opens the parameters for the object and their might be () or (()) but they close so by the time you get to the final ) the count would just be plus 1 and the final ) would bring it back to zero to close out this specific object. hope that makes sense
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 
LVL 76

Expert Comment

by:arnold
Comment Utility
Counting opening,closing parenthesis is unnecessary and could lead to compound errors i.e. if there are mismatched parenthesis in the data, your code attempting to match count, could have an issue.
0
 
LVL 51

Expert Comment

by:ahoffmann
Comment Utility
> .. and delete the contents from : to the closing bracket )
do you mean that you want to delete all lines 'till the matching ) bracket?

if so, can you ensure that all ( are balance with ) ?
in particulat can you ensure not to have something like
   
                          :icon ("crazy/file)name")

if not, you need a context sensitive parser for your data
0
 
LVL 6

Author Comment

by:Dooglave
Comment Utility
sed '\^..\:.\(VSXSERVER,\^..\)/d' objects_5_0.C.test
sed: -e expression #1, char 27: Unterminated address regex

Whaaa, what's wrong?


Quoted Text: (http://lowfatlinux.com/linux-sed.html)
Now let's try an example involving deletion of lines. The values for start and end can be either a line number or a pattern to match. All lines from the start line to the end line are removed from the output. This example will delete starting at line 2, up to and including line 3:

sed '2,3d' poem.txt
Mary had a little lamb
Jill had a lamb spamwich

This example will delete starting at line 1, up to and including the next line containing Jack:

sed '1,/Jack/d' poem.txt
Jill had a lamb spamwich

The most common use of sed is to change one string of text to another string of text. But I should mention that the strings that sed uses for search and delete are actually regular expressions. This means you can use pattern matching, just as with grep. Although you'll probably never need to do anything like this, here's an example anyway. To change any occurrences of lamb at the end of a line to ham, and save the results in a new file, enter this:
0
 
LVL 76

Expert Comment

by:arnold
Comment Utility
You never open the block leaving a single /d near the end.
Your first \ should be /.
0
 
LVL 51

Accepted Solution

by:
ahoffmann earned 500 total points
Comment Utility
> Whaaa, what's wrong?
sed uses / as regex separator, however, GNU's sed may use any charcter also, but with \ you most likely will hav a problem 'cause it's the shell's meta character for escape

I'm not sure what you want to archive with your sed command
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now