Link to home
Start Free TrialLog in
Avatar of David Aldridge
David AldridgeFlag for United States of America

asked on

Replacing words in a line of a file.

replacing parts of a line in a file (/etc/group).

If I have the following line in my group file, I want to remove all of the "sdb" entries, but leave anything else.  For instance:

sdba::1014:sdb,sdb,david,sdb

would need to be changed to:

sdba::1014:david

I'm trying to do this with perl -pi -e if I can.  This SORT of gets me there, but removes the "david" entry.

egrep "^sdba:" $TEMP && perl -pi -e "s/^sdba::.*/sdba::1014:/g" $TEMP

Thanks!
David
Avatar of Travis Martinez
Travis Martinez
Flag of United States of America image

I know you're wanting to do this in perl but leveraging sed will do the job a lot easier.  I've just tested the following:

cat group | sed 's/sdb,sdb,david,sdb/david/g'

Returns:

sdba::1014:david

Also using egrep for the beginning of line ^sdba and piping to sed works the same

egrep ^sdba group | sed 's/sdb,sdb,david,sdb/david/g'
Avatar of David Aldridge

ASKER

I could use sed, because the actual executable is a ksh script.  The problem with this is it might not be "david".  It might be any other name.  I just want to get rid of the "sdb" entries, of couse leaving the group name of sdba.

Thanks!
David
And, it needs to only make the change on the sdba line.  "sdb" might be under another group, and I would want that to stay.
SOLUTION
Avatar of Travis Martinez
Travis Martinez
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Wow, I was really overthinking this one. I was about to go to Python. Thanks!
David
Thanks again! Sometimes I need to just get up and walk away.

David
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Sorry to be the bearer of bad news, but Travis's solution will not handle cases like this:
$ cat group
sdba::1014:sdb,sdb,david,sdb
sdba::1015:sdb,test2,david,sdb
sdbb::1016:test1,test2,david,sdb

Here's what happens:
$ egrep ^sdba group | sed 's/sdb//g' | sed 's/,//g' | sed 's/a::/sdba::/g'
sdba::1014:david
sdba::1015:test2david

Note how there is no comma between test2 & david.
ozo's 2nd Perl solution doesn't have this problem.

Also, commands like this:
    cat group | sed 's/sdb,sdb,david,sdb/david/g'
can be simplified to this (no cat, pipe and in this case I doubt the /g would be necessary):
    sed 's/sdb,sdb,david,sdb/david/' group

And commands like this:
    egrep ^sdba group | sed 's/sdb//g' | sed 's/,//g' | sed 's/a::/sdba::/g'
can be simplified to this (no need for egrep when grep will work, and remove some pipes and seds):
    grep ^sdba group | sed 's/sdb//g; s/,//g; s/a::/sdba::/g'
and maybe even this (no grep):
    sed '/^sdba/ {s/sdb//g;s/,//g;s/a::/sdba::/g}' group
although the latter retains lines which don't start with "sdba".

Questions for David:
Q1. When you said this:
   "If I have the following line in my group file..."
you didn't state what the line was.  What was it?  Anything starting with "sdba:"?
Q2. What did you want to happen with the lines which didn't start with "sdba"?  Remove them completely or leave them intact?
Q3. And did you want to overwrite the original file, as your use of Perl's -i switch suggests, or what?

Here's a simpler Perl solution, but whether it does what you want depends on your answers to my questions above:
    perl -pe  '/^sdba:/ && s/,sdb|sdb,//g' group
Or in sed:
    sed '/^sdba:/ {s/,sdb//g; s/sdb,//g}' group
The output of these can differ from ozo's 2nd Perl solution.  For example, for this input:
    sdba:sdb:1012:test1,sdb,sdb,david
my Perl & sed solutions above would output this:
    sdba:sdb:1012:test1,david
and ozo's would output this:
    sdba::1012:test1,david
But that kind of input may never happen so it may not make any difference.
I realized after playing with it a bit that it needed some modification and ended up with this solution, which seems to be working out:

echo $line |grep "^sdba" | sed 's/:sdb/:/g' | sed 's/,sdb,/,/g' | sed 's/sdb$//g' | sed 's/,$//g' | sed 's/:,/:/g' >> $TEMP
What should the result be from
sdba::1014:sdb1,test1,sdb,sdba,sdb,david,testsdb
?
sdba::1014:sdb1,test1,sdba,david,testsdb  

Although, I would never see anything like that.  Here's the most radical example that I've tested it with and it works fine so far:

sdba::1014:sdb,sdb,david,sdb,billy,sdb,sue,sdb

ends up:

sdba::1014:david,billy,sue
Hi David,

ozo seems to be demonstrating a shortcoming of your latest solution.  You've said above that this:
    sdba::1014:sdb1,test1,sdb,sdba,sdb,david,testsdb
should become this:
    sdba::1014:sdb1,test1,sdba,david,testsdb
but as you can see when you process it:
    echo "sdba::1014:sdb1,test1,sdb,sdba,sdb,david,testsdb" | grep "^sdba" | sed 's/:sdb/:/g' | sed 's/,sdb,/,/g' | sed 's/sdb$//g' | sed 's/,$//g' | sed 's/:,/:/g'
it becomes this:
    sdba::1014:1,test1,sdba,david,test
which is quite different.

That kind of input may not happen, but maybe something like this is more realistic:
    echo "sdba::1014:sdbb,sdb,sdb,david" | grep "^sdba" | sed 's/:sdb/:/g' | sed 's/,sdb,/,/g' | sed 's/sdb$//g' | sed 's/,$//g' | sed 's/:,/:/g'
Here's the output you'd get:
    sdba::1014:b,sdb,david
That "b" ain't pretty, is it?

I suggest you go for one of the shorter, more robust, and possibly simpler solutions provided by ozo or myself.  Here are mine again:
    perl -pe  '/^sdba:/ && s/,sdb|sdb,//g' group
Or in sed:
    sed '/^sdba:/ {s/,sdb//g; s/sdb,//g}' group
Or here's a slightly different version:
    perl -pe  's/,sdb|sdb,//g if /^sdba:/' group
If there's some reason those aren't what you want, please explain why and we might be able to sort it out.

Also, what were the answers to my questions Q1-Q3, David?
Thank you for the continued interest.  While I'm sure that what I have would WORK, it most certainly, as you point out, isn't the best solution.  In the real world, I'm most likely going to only want to get rid of the "sdb" user if it is attached to the sdba group.  I really don't see any others being in there, but I was just trying to take that into account. I think I will do as you suggest and work with one of the better solutions that you've suggested, just in case!  The answers to your questions are:

1.  The exact line I originally put down, although "david" was added as a test:   sdba::1014:sdb,sdb,david,sdb
2.  I don't want to do anything to a line that doesn't start with sdba, i.e., ^sdba:
3.  I took this project over from someone else who has been working on it for a while and needed help so I was starting to work with what he already had there.  What I ended up doing with the sed solution is writing to a temp file and I'll just copy rename it when the process is over.

Again, thanks for your help.  I wish I could give you and ozo some points on it.  I was just in a hurry and jumped at the first thing that worked with my original request.

Thanks,
David
So if you're interested, this whole question was actually part of a do while loop in ksh.  It does quite a bit to the group file.  Here's the entire loop.  It's working as it is, but the "sdba" and "dazel" pieces were the biggest problem.  I'm sure this could be done a LOT easier and more efficient than what I came up with:

        TEMP="temp"$(date '+%d%m%Y')
        touch $TEMP

        while read line
        do
                if [[ $(echo $line|grep "mysql::70:") ]]
                then
                        MYSQL=found
                        echo $line >> $TEMP
                elif [[ $(echo $line | grep "^mlocate") ]]
                then
                        MLOCATE=found
                        echo $line >> $TEMP
                elif [[ $(echo $line | grep "^slocate") ]]
                then
                        continue
                elif [[ $(echo $line | grep "^sdba") ]]
                then
                        echo $line |grep "^sdba" | sed 's/:sdb/:/g' | sed 's/,sdb,/,/g' | sed 's/sdb$//g' | sed 's/,$//g' | sed 's/:,/:/g' >> $TEMP
                elif [[ $(echo $line | grep "^dazel") ]]
                then
                        echo $line |grep "^dazel" | sed 's/:dazel/:/g' | sed 's/,dazel,/,/g' | sed 's/dazel$//g' | sed 's/,$//g' | sed 's/:,/:/g' >> $TEMP
                else
                        grep -q "$line" $TEMP || echo $line >> $TEMP
                fi
        done < /etc/group
        [ ! $MYSQL ] && echo "mysql::70:" >> $TEMP
        [ ! $MLOCATE ] && echo mlocate::95: >> $TEMP

        # mv $TEMP /etc/group
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Done.  We'll see what the moderators say.  Considering the expanse of the question, I would think it would be acceptable for me to open another question, posting the entire loop.  You and ozo have been a great help.
Sorry about all of the point shuffling guys.  I really appreciate all of the help you gave.  I also could have been much clearer in my original question.  I ended up using this line from tel2's comment:

echo $line | sed 's/,sdb//g; s/sdb,//g' >> $TEMP
Thanks, David.

Did you also use this line that I also suggested?:
    echo $line | sed 's/,dazel//g; s/dazel,//g' >> $TEMP

Why do you feel the need to open another question?  Do you need more help?
For the "dazel" entry, I just used the one below because there would only ever be dazel::206:dazel   There shouldn't be ANY users with dazel as a secondary group.  Now that I think about it, I guess I could just overwrite the line with dazel::206: no matter what's there.  But here's what I used:

                elif [[ $(echo $line | grep "^dazel") ]]
                then
                        echo $line | sed 's/:dazel/:/g' >> $TEMP

The reason I mentioned possibly opening another question was because I closed this one too quickly and received so much more help from you and ozo that I wanted to be able to reward points to the both of you as well.  Your suggestion worked and they let me reopen it though.  Thanks once again!

David
OK David, I guess that will work, but technically you don't need the /g (global replace), because you're replacing the 1st (and only) match alone.  So this:
    echo $line | sed 's/:dazel/:/g' >> $TEMP
Could change to this:
    echo $line | sed 's/:dazel/:/' >> $TEMP