Solved

Simple BASH help. Input from a file.

Posted on 2014-09-04
26
388 Views
Last Modified: 2014-09-04
Hello,
I have a simple question. I'm trying to write a one liner or script to search a file for a name on each line. Then take that line as a variable and run that variable in this sed command I've come up with.
How do I read each line from the name file formatted 1 name per line and take it as a variable in the sed command and sent to some output file?
Thanks
0
Comment
Question by:cipherlogic
  • 17
  • 9
26 Comments
 

Author Comment

by:cipherlogic
ID: 40304291
Note, I tried where line read to try sending the line to the sed command as a variable but haven't succeeded yet.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40304298
This is for searching a file for a particular "search_string":

grep "search_string" filename | while read line
  do
    sed -e "sed_subcommands" <<< $line
  done > some_output_file

This is for processing every line:

while read line
  do
     sed -e "sed_subcommands" <<< $line
  done < filename > some_output_file

wmp
0
 

Author Comment

by:cipherlogic
ID: 40304305
I'll try that in a bit.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40304310
Please note that both solutions require using the bash shell!
0
 

Author Comment

by:cipherlogic
ID: 40304313
;)
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40304321
The second solution is not very different from

sed -e "sed_subcommands" filename > some_output_file
0
 

Author Comment

by:cipherlogic
ID: 40304417
Ok here's what I got. I still don't understand how to run this against the name file I created though...
The input file I want to run this against is names.txt

#!/bin/bash
 while read line;
 do
 sed -e -n '/$line/{N;N;N;N;N;p}' ldap.ldif | grep description;
 done > names.out
0
 

Author Comment

by:cipherlogic
ID: 40304429
Got it in a one liner.
while read line; do echo -e "$line"; done < names.txt | grep description ldap.ldiff
0
 

Author Comment

by:cipherlogic
ID: 40304492
I've requested that this question be closed as follows:

Accepted answer: 0 points for cipherlogic's comment #a40304429
Assisted answer: 250 points for woolmilkporc's comment #a40304298

for the following reason:

My solution was best.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40304461
Sorry, I cant understand what your one-liner should be good for.

while read line; do echo -e "$line"; done < names.txt

is nothing else than

cat names .txt

Further, if you give grep the name of a file  to search in it will ignore stdin. So your code is equivalent to

grep description ldap.ldiff
0
 

Author Comment

by:cipherlogic
ID: 40304473
No, this code runs sed against every line in an input file and outputs to a file with grep being performed only on each line.
0
 

Author Comment

by:cipherlogic
ID: 40304493
It seems you're correct. I examined the output file and found it is just like running grep on the file.
0
 

Author Comment

by:cipherlogic
ID: 40304504
I basically have 3 files. 1 is a master list of users. 2 is the ldap dump. 3 is the output file.

I need this script to:
 take in the input file
 print each line in a file with the results of searching it as the variable in the sed command
   if it isn't found in the ldap dump, then print a blank line instead of results

That's about it.
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 

Author Comment

by:cipherlogic
ID: 40304526
Basically take this:
sed -n '/namevariable/{N;N;N;N;N;p}' ldap.ldif | grep description

and run it each time a new line is encountered from the input file...

while read line; do echo -e "$line"; done < names.txt

...

bonus if I can print the username line from the input with blank lines if its not found with the sed...
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40304534
print each line in a file with the results of searching it as the variable in the sed command

That's the same as

sed 'command' inputfile

sed works by default against each line of its input file. No need for a "line" variable.

You can take the results of the above (maybe once again filtered by grep) as a pattern list for grep:

sed -n '/namevariable/{N;N;N;N;N;p}' ldap.ldif | grep description
| grep -f - ldapdump > outputfile

Remains the thing with the blank lines ...
0
 

Author Comment

by:cipherlogic
ID: 40304557
Yes but I'm getting the namevariable from names.txt as the input file. I don't know how to pull the variable from the input file. Otherwise I have to run this sed by hand per user.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40304562
You want to read a file and use the strings found there in sed?

while read namevariable
 do
   sed -n '/'$namevariable'/{N;N;N;N;N;p}' ldap.ldif | grep description
 done < names.txt

Please note the additional single quotes ('') and the "$" character in front of the variable name.
0
 

Author Comment

by:cipherlogic
ID: 40304568
well....except namevariable isn't any real variable. Its just a username per line...
0
 

Author Comment

by:cipherlogic
ID: 40304578
Hang on - let me try to hash out what you're saying. I think I may get it now.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40304594
... namevariable isn't any real variable ...

Yes it is, because of "while read namevariable"
0
 

Author Comment

by:cipherlogic
ID: 40304611
#!/bin/bash
 while read line;
 do
 sed -n '/'$line'/{N;N;N;N;N;p}' ldap.ldif | grep description;
 done < names.txt >> names.out


This did the trick, except for it didn't print the usernames and if it wasn't present in the comparison file, didn't produce a blank line.

I need it to print the username above what it greps to show this username / username associated info below...and if its not there, I just want to see the username and a blank...Very close.
0
 

Author Comment

by:cipherlogic
ID: 40304628
This did it...

#!/bin/bash
 while read line;
 do
 echo $line;
 sed -n '/'$line'/{N;N;N;N;N;p}' ldap.ldif | grep description;
 done < names.txt >> names.out
0
 

Author Comment

by:cipherlogic
ID: 40304650
I've requested that this question be closed as follows:

Accepted answer: 0 points for cipherlogic's comment #a40304611
Assisted answer: 500 points for woolmilkporc's comment #a40304534

for the following reason:

Most detailed for others to reference.
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 40304640
This will require a bit more code.

#!/bin/bash
 while read line
   do
     result=$(sed -n '/'$line'/{N;N;N;N;N;p}' ldap.ldif | grep description)
      if [[ ! -z "$result" ]]
         then
           echo $line
            echo $result
              else
                  echo $line
                  echo
       fi
    done < names.txt > names.out

I can't test the above right now, so there might be typos/syntax errors. If so, please let me know, I'll try to correct them.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40304644
This did it...

.. except for the blank line, I guess.
0
 

Author Comment

by:cipherlogic
ID: 40304646
Its cool. This is enough to let me refine output. I've suggested to v close with you getting full points.
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
A short article about problems I had with the new location API and permissions in Marshmallow
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Viewers will learn how to properly install Eclipse with the necessary JDK, and will take a look at an introductory Java program. Download Eclipse installation zip file: Extract files from zip file: Download and install JDK 8: Open Eclipse and …

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now