Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Parsing a file using ksh

Posted on 2016-07-28
10
Medium Priority
?
107 Views
Last Modified: 2016-07-29
I have a file that looks like this:

uid=name
userpassword={crypt}nJEKEgaHjJZY2
uid=john
userpassword={crypt}nJEKEgaHjJZY2
uid=bill
userpassword={crypt}nJEKEgaHjJZY2

Of course the hash will be different on the userpasswords.  What I want to do is read this file and write it out to another file in this format:

name      nJEKEgaHjJZY2
john        nJEKEgaHjJZY2
bill           nJEKEgaHjJZY2

Any help would be appreciated.  Again, of course the userpasswords are unique, but they all have the {crypt} in front of them.  I got this far, but I can't get it to read through the whole file.

servername-[root]/home/share/david/work> jane=$(cat file | cut -d= -f2 | sed s/{crypt}//g)
servername-[root]/home/share/david/work> echo $jane
oracle nJEKEgaHjJZY2

Thanks!
David
0
Comment
Question by:David Aldridge
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
10 Comments
 
LVL 1

Author Comment

by:David Aldridge
ID: 41733777
I figured it out.  If someone has a better way, I'd be glad to award the points for it.  I'll leave it open for a bit.

#/usr/bin/ksh
# set -x
count=1

while read -r line
do
        if [[ $count = 1 ]]
        then
                first=$(echo $line | cut -d= -f2 | sed s/{crypt}//g)
                count=$(( $count + 1))
                continue
        else
                second=$(echo $line | cut -d= -f2 | sed s/{crypt}//g)
                echo "$first\t$second"
                count=1
        fi
done < file
0
 
LVL 12

Expert Comment

by:tel2
ID: 41733991
Hi David,

I don't know whether this is "better", but it's more concise and should be a lot faster:

sed 'N;s/\n//;s/uid=//;s/userpassword={crypt}/\t/' file

Output:
name        nJEKEgaHjJZY2
john        nJEKEgaHjJZY2
bill        nJEKEgaHjJZY2

Put it in a ksh script if you wish.

If you don't like the above, will you accept Perl solutions?

tel2
1
 
LVL 12

Expert Comment

by:tel2
ID: 41734001
P.S. If you decide to stick with your solution, note:
- The /g modifiers in the sed commands are not necessary.  Try: sed s/{crypt}//
- If you want a real tab character to appear between output fields, use the -e switch: echo -e "$first\t$second"
- The "continue" line is not necessary in this case.
- Your "count=$(( $count + 1))" line could be simplified to:
    count=$((count + 1))
   OR:
    let count=$count+1
- Your "while read -r line" line could be just "while read line" unless the input could contain back-slashes.
- I used to use ksh, but I'd recommend use bash if it is available to you.  It's more modern and powerful, and I think most ksh code will run in bash.
1
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
LVL 12

Accepted Solution

by:
tel2 earned 2000 total points
ID: 41734043
Here's a slightly different alternative sed approach:

sed 'N;s/uid=\(.*\)\nuserpassword={crypt}\(.*\)/\1\t\2/' file
1
 
LVL 12

Expert Comment

by:tel2
ID: 41734048
P.S. If you ever want to trace the execution of an entire script, you don't need to put this at the start:
    set -x
You can just run the script like this:
    ksh -x scriptname
or:
    bash -x scriptname
This means you don't need to waste time editing the script before and after you do your debugging.
But "set -x" still has its place, because it allows you to turn on debugging from a specific point in the code.
1
 
LVL 1

Author Comment

by:David Aldridge
ID: 41734744
Great stuff!  If you have time or care to shoot a Perl solution, that would be great.  If you don't, that's fine.  I understand the time thing!

Thanks again!
David
0
 
LVL 1

Author Closing Comment

by:David Aldridge
ID: 41734746
Excellent!
0
 
LVL 12

Expert Comment

by:tel2
ID: 41735426
Thanks for the points, David.

Here are 3 Perl solutions:

# Read file line by line removing unwanted text
perl -pe 's/^uid=(.*)\n/$1\t/;s/^userpassword={crypt}//' file

# Read entire file (slurp) then remove unwanted text with a global substitution
perl -0pe 's/^uid=(.+)\nuserpassword={crypt}(.+)/$1\t$2/mg' file

# Read entire file (slurp) then print wanted parts a while loop
perl -0ne 'print "$1\t$2\n" while(/^uid=(.+)\nuserpassword={crypt}(.+)/mg)' file

But each of the above could also be written as traditional Perl scripts (rather than one-liners), and could also be embedded in a shell script as a one-liner or an in-line Perl script (here document).

Too bad I don't understand the time thing yet.    8)

Any questions about the above?
0
 
LVL 1

Author Comment

by:David Aldridge
ID: 41735431
Absolutely brilliant!  Thank you so much for your help. I've had a subscription here since Jan 3, 2001.  It's answers like these that make every dollar with it.  Thank you my friend.
0
 
LVL 12

Expert Comment

by:tel2
ID: 41735439
Thanks David,

I prefer Perl because it's flexible enough to be used for almost anything (including the coding behind websites, which is what I sometimes use it for), and I know it better than sed, awk, etc.  But the sed solutions might be slightly more efficient for problems like yours.

A pleasure doing business, David.

Please hold while I give myself a pay rise for confusing you with 5 solutions instead of 1...
0

Featured Post

Important Lessons on Recovering from Petya

In their most recent webinar, Skyport Systems explores ways to isolate and protect critical databases to keep the core of your company safe from harm.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded‚Ķ
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.
Suggested Courses

636 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question