Solved

Parsing a file using ksh

Posted on 2016-07-28
10
64 Views
Last Modified: 2016-07-29
I have a file that looks like this:

uid=name
userpassword={crypt}nJEKEgaHjJZY2
uid=john
userpassword={crypt}nJEKEgaHjJZY2
uid=bill
userpassword={crypt}nJEKEgaHjJZY2

Of course the hash will be different on the userpasswords.  What I want to do is read this file and write it out to another file in this format:

name      nJEKEgaHjJZY2
john        nJEKEgaHjJZY2
bill           nJEKEgaHjJZY2

Any help would be appreciated.  Again, of course the userpasswords are unique, but they all have the {crypt} in front of them.  I got this far, but I can't get it to read through the whole file.

servername-[root]/home/share/david/work> jane=$(cat file | cut -d= -f2 | sed s/{crypt}//g)
servername-[root]/home/share/david/work> echo $jane
oracle nJEKEgaHjJZY2

Thanks!
David
0
Comment
Question by:David Aldridge
  • 6
  • 4
10 Comments
 
LVL 1

Author Comment

by:David Aldridge
ID: 41733777
I figured it out.  If someone has a better way, I'd be glad to award the points for it.  I'll leave it open for a bit.

#/usr/bin/ksh
# set -x
count=1

while read -r line
do
        if [[ $count = 1 ]]
        then
                first=$(echo $line | cut -d= -f2 | sed s/{crypt}//g)
                count=$(( $count + 1))
                continue
        else
                second=$(echo $line | cut -d= -f2 | sed s/{crypt}//g)
                echo "$first\t$second"
                count=1
        fi
done < file
0
 
LVL 12

Expert Comment

by:tel2
ID: 41733991
Hi David,

I don't know whether this is "better", but it's more concise and should be a lot faster:

sed 'N;s/\n//;s/uid=//;s/userpassword={crypt}/\t/' file

Output:
name        nJEKEgaHjJZY2
john        nJEKEgaHjJZY2
bill        nJEKEgaHjJZY2

Put it in a ksh script if you wish.

If you don't like the above, will you accept Perl solutions?

tel2
1
 
LVL 12

Expert Comment

by:tel2
ID: 41734001
P.S. If you decide to stick with your solution, note:
- The /g modifiers in the sed commands are not necessary.  Try: sed s/{crypt}//
- If you want a real tab character to appear between output fields, use the -e switch: echo -e "$first\t$second"
- The "continue" line is not necessary in this case.
- Your "count=$(( $count + 1))" line could be simplified to:
    count=$((count + 1))
   OR:
    let count=$count+1
- Your "while read -r line" line could be just "while read line" unless the input could contain back-slashes.
- I used to use ksh, but I'd recommend use bash if it is available to you.  It's more modern and powerful, and I think most ksh code will run in bash.
1
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 12

Accepted Solution

by:
tel2 earned 500 total points
ID: 41734043
Here's a slightly different alternative sed approach:

sed 'N;s/uid=\(.*\)\nuserpassword={crypt}\(.*\)/\1\t\2/' file
1
 
LVL 12

Expert Comment

by:tel2
ID: 41734048
P.S. If you ever want to trace the execution of an entire script, you don't need to put this at the start:
    set -x
You can just run the script like this:
    ksh -x scriptname
or:
    bash -x scriptname
This means you don't need to waste time editing the script before and after you do your debugging.
But "set -x" still has its place, because it allows you to turn on debugging from a specific point in the code.
1
 
LVL 1

Author Comment

by:David Aldridge
ID: 41734744
Great stuff!  If you have time or care to shoot a Perl solution, that would be great.  If you don't, that's fine.  I understand the time thing!

Thanks again!
David
0
 
LVL 1

Author Closing Comment

by:David Aldridge
ID: 41734746
Excellent!
0
 
LVL 12

Expert Comment

by:tel2
ID: 41735426
Thanks for the points, David.

Here are 3 Perl solutions:

# Read file line by line removing unwanted text
perl -pe 's/^uid=(.*)\n/$1\t/;s/^userpassword={crypt}//' file

# Read entire file (slurp) then remove unwanted text with a global substitution
perl -0pe 's/^uid=(.+)\nuserpassword={crypt}(.+)/$1\t$2/mg' file

# Read entire file (slurp) then print wanted parts a while loop
perl -0ne 'print "$1\t$2\n" while(/^uid=(.+)\nuserpassword={crypt}(.+)/mg)' file

But each of the above could also be written as traditional Perl scripts (rather than one-liners), and could also be embedded in a shell script as a one-liner or an in-line Perl script (here document).

Too bad I don't understand the time thing yet.    8)

Any questions about the above?
0
 
LVL 1

Author Comment

by:David Aldridge
ID: 41735431
Absolutely brilliant!  Thank you so much for your help. I've had a subscription here since Jan 3, 2001.  It's answers like these that make every dollar with it.  Thank you my friend.
0
 
LVL 12

Expert Comment

by:tel2
ID: 41735439
Thanks David,

I prefer Perl because it's flexible enough to be used for almost anything (including the coding behind websites, which is what I sometimes use it for), and I know it better than sed, awk, etc.  But the sed solutions might be slightly more efficient for problems like yours.

A pleasure doing business, David.

Please hold while I give myself a pay rise for confusing you with 5 solutions instead of 1...
0

Featured Post

Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question