Solved

Parsing a file using ksh

Posted on 2016-07-28
10
47 Views
Last Modified: 2016-07-29
I have a file that looks like this:

uid=name
userpassword={crypt}nJEKEgaHjJZY2
uid=john
userpassword={crypt}nJEKEgaHjJZY2
uid=bill
userpassword={crypt}nJEKEgaHjJZY2

Of course the hash will be different on the userpasswords.  What I want to do is read this file and write it out to another file in this format:

name      nJEKEgaHjJZY2
john        nJEKEgaHjJZY2
bill           nJEKEgaHjJZY2

Any help would be appreciated.  Again, of course the userpasswords are unique, but they all have the {crypt} in front of them.  I got this far, but I can't get it to read through the whole file.

servername-[root]/home/share/david/work> jane=$(cat file | cut -d= -f2 | sed s/{crypt}//g)
servername-[root]/home/share/david/work> echo $jane
oracle nJEKEgaHjJZY2

Thanks!
David
0
Comment
Question by:David Aldridge
  • 6
  • 4
10 Comments
 
LVL 1

Author Comment

by:David Aldridge
Comment Utility
I figured it out.  If someone has a better way, I'd be glad to award the points for it.  I'll leave it open for a bit.

#/usr/bin/ksh
# set -x
count=1

while read -r line
do
        if [[ $count = 1 ]]
        then
                first=$(echo $line | cut -d= -f2 | sed s/{crypt}//g)
                count=$(( $count + 1))
                continue
        else
                second=$(echo $line | cut -d= -f2 | sed s/{crypt}//g)
                echo "$first\t$second"
                count=1
        fi
done < file
0
 
LVL 11

Expert Comment

by:tel2
Comment Utility
Hi David,

I don't know whether this is "better", but it's more concise and should be a lot faster:

sed 'N;s/\n//;s/uid=//;s/userpassword={crypt}/\t/' file

Output:
name        nJEKEgaHjJZY2
john        nJEKEgaHjJZY2
bill        nJEKEgaHjJZY2

Put it in a ksh script if you wish.

If you don't like the above, will you accept Perl solutions?

tel2
1
 
LVL 11

Expert Comment

by:tel2
Comment Utility
P.S. If you decide to stick with your solution, note:
- The /g modifiers in the sed commands are not necessary.  Try: sed s/{crypt}//
- If you want a real tab character to appear between output fields, use the -e switch: echo -e "$first\t$second"
- The "continue" line is not necessary in this case.
- Your "count=$(( $count + 1))" line could be simplified to:
    count=$((count + 1))
   OR:
    let count=$count+1
- Your "while read -r line" line could be just "while read line" unless the input could contain back-slashes.
- I used to use ksh, but I'd recommend use bash if it is available to you.  It's more modern and powerful, and I think most ksh code will run in bash.
1
 
LVL 11

Accepted Solution

by:
tel2 earned 500 total points
Comment Utility
Here's a slightly different alternative sed approach:

sed 'N;s/uid=\(.*\)\nuserpassword={crypt}\(.*\)/\1\t\2/' file
1
 
LVL 11

Expert Comment

by:tel2
Comment Utility
P.S. If you ever want to trace the execution of an entire script, you don't need to put this at the start:
    set -x
You can just run the script like this:
    ksh -x scriptname
or:
    bash -x scriptname
This means you don't need to waste time editing the script before and after you do your debugging.
But "set -x" still has its place, because it allows you to turn on debugging from a specific point in the code.
1
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 1

Author Comment

by:David Aldridge
Comment Utility
Great stuff!  If you have time or care to shoot a Perl solution, that would be great.  If you don't, that's fine.  I understand the time thing!

Thanks again!
David
0
 
LVL 1

Author Closing Comment

by:David Aldridge
Comment Utility
Excellent!
0
 
LVL 11

Expert Comment

by:tel2
Comment Utility
Thanks for the points, David.

Here are 3 Perl solutions:

# Read file line by line removing unwanted text
perl -pe 's/^uid=(.*)\n/$1\t/;s/^userpassword={crypt}//' file

# Read entire file (slurp) then remove unwanted text with a global substitution
perl -0pe 's/^uid=(.+)\nuserpassword={crypt}(.+)/$1\t$2/mg' file

# Read entire file (slurp) then print wanted parts a while loop
perl -0ne 'print "$1\t$2\n" while(/^uid=(.+)\nuserpassword={crypt}(.+)/mg)' file

But each of the above could also be written as traditional Perl scripts (rather than one-liners), and could also be embedded in a shell script as a one-liner or an in-line Perl script (here document).

Too bad I don't understand the time thing yet.    8)

Any questions about the above?
0
 
LVL 1

Author Comment

by:David Aldridge
Comment Utility
Absolutely brilliant!  Thank you so much for your help. I've had a subscription here since Jan 3, 2001.  It's answers like these that make every dollar with it.  Thank you my friend.
0
 
LVL 11

Expert Comment

by:tel2
Comment Utility
Thanks David,

I prefer Perl because it's flexible enough to be used for almost anything (including the coding behind websites, which is what I sometimes use it for), and I know it better than sed, awk, etc.  But the sed solutions might be slightly more efficient for problems like yours.

A pleasure doing business, David.

Please hold while I give myself a pay rise for confusing you with 5 solutions instead of 1...
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Introduction Regular patching is part of a system administrator's tasks. However, many patches require that the system be in single-user mode before they can be installed. A cluster patch in particular can take quite a while to apply if the machine…
Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now