?
Solved

Need a way to loop a input file

Posted on 2009-02-16
50
Medium Priority
?
346 Views
Last Modified: 2013-12-26
Hello all,  I am in need of a way to create a loop effect to enter information from this file 2 lines at a time. I need a way to strip this file and have it run. The way it stands now, the CLI ( command line interface does not understand multiple lines. So I need to break it down by these two variables

SubnetAddress=165.168.31.0
Domain=emcbi.com beer.com is.beer.com fi.beer.com emc.beer.com

The CLI input string is used like to

qip-setsubnet - u username -p password -f <input file name > -o "Organization name" -t <to append these to existing subnets already in  the system>

so

qip-setsubnet - u username -p password -f e:\ 4-5-fix-subnets.txt -o rich -t

I have perl  on my system if this is the way to go, but I dont have the skills yet to do the above. I can except any ideas that will make this work. I need to be able to take this solution and use it on either windows or Sun O/S's

I have attached the file that needs to be stripped and ran with two variables at a time.

Thank you in advance for your prompt replies
4-5-fix-subnets.txt
0
Comment
Question by:richsark
  • 27
  • 16
  • 7
50 Comments
 
LVL 40

Expert Comment

by:omarfarid
ID: 23651091
see the script below which is an example:

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1 $line2"
done

you may replace the echo line by any line of your's
0
 
LVL 1

Author Comment

by:richsark
ID: 23651110
Hi, so how do I call it to run on the next set all the way down the file?
Thanks
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 23651141
it does that in the while loop, it reads one line with the while read and the next line by the 2nd read. this will loop till the input file is read till end of file
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 1

Author Comment

by:richsark
ID: 23651189
Ok. so where do I put my cli string to get them in?
qip-setsubnet - u username -p password -f e:\ 4-5-fix-subnets.txt -o rich -t
also, when I ran a test of the cat  it shows this
SubnetAddress=172.30.80.0 Domain=beer.com is.beer.com fi.beer.com emc.beer.com encorecredit.com
SubnetAddress=172.30.82.0 Domain=beer.com is.beer.com fi.beer.com emc.beer.com encorecredit.com
SubnetAddress=169.93.158.0 Domain=net.uk.jp.net
SubnetAddress=207.162.226.0 Domain=bear.com is.beer.com fi.beer.com emc.beer.com clearco.com
SubnetAddress=206.181.213.0 Domain=disclosure.com private.disclosure.com
I need it to be on two lines like so:
SubnetAddress=172.30.80.0
Domain=bear.com is.beer.com fi.bear.com emc.beer.com encorecredit.com

Thanks
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 23651235

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo $line1
    echo $line2
done

I am not sure what you want to do with

qip-setsubnet - u username -p password -f e:\ 4-5-fix-subnets.txt -o rich -t
0
 
LVL 1

Author Comment

by:richsark
ID: 23651319
Hello Sir,
in order to get this two lines in the system, I have to call a CLI (see above) and reference the txt file.
The issue I have is that the CLI does not understand this massive file and I need to call it by two lines at a time. The two lines are mandatory and must have the header
SubnetAddress=
Domain=
So I need a way to automate this so its takes my input file, strip it down into two headers have them go into the system use the
CLI qip-setsubnet string and run all the way down the files.
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 23651348
can you give example of how the cli should be called with sample data?
0
 
LVL 1

Author Comment

by:richsark
ID: 23651403
Hi,
I need to take the file I have attached "4-5-fix-subnets.txt "
cut it down into two headers

SubnetAddress=
Domain=
and run using the cli to call the file.  
So I guess need to perhaps pipe each of the two headers out into a scratch file that makes an input file that is called by the CLI and the whole process repeats all the way down the master file called 4-5-fix-subnets.txt

Does that help?
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23651458
What is the syntax of the qip-setsubnet command when passing the 2 parameters instead of the file?
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 23651466
try this:

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1
    $line2" | qip-setsubnet - u username -p password -o rich -t
done
0
 
LVL 1

Author Comment

by:richsark
ID: 23651502
Hi,
We need to create a scratch file called perhaps step1.txt which has the two headers from the master file "4-5-fix-subnets.txt"
Then take the file step1.txt and have it standby so the CLI can call it using
qip-setsubnet - u username -p password -f e:\ step1.txt -o rich -t  
Then it will need to repeat to the next set ofheaders create another file called perhaps step2.txt have it standby while the CLI is called agian to reference the step2.txt
qip-setsubnet - u username -p password -f e:\ step2.txt -o rich -t  
and so on.... all the way down the master file "4-5-fix-subnets.txt"
Does that help?
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 23651530
try

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1
    $line2" > step.txt
    qip-setsubnet - u username -p password -f step.txt -o rich -t
done

0
 
LVL 1

Author Comment

by:richsark
ID: 23651563
Hello, so your last commet will run all the way down the file calling the same file step.txt until its done? so it step.txt will be overwritten with the next step right?
Is there a stop or a way to find out if it failed at some stage?
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 23651618
yes it will overwrite the temp file.

you could save output or errors into file:

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1
    $line2" > step.txt
    qip-setsubnet - u username -p password -f step.txt -o rich -t >> log.txt 2>&1
done
0
 
LVL 1

Author Comment

by:richsark
ID: 23651941
Ok, will try it now and let you know in a few minutes. I need to install CYGWIN on my windows to test this as I dont have access today to a sun box
Thanks
0
 
LVL 1

Author Comment

by:richsark
ID: 23653204
HI, I am having a rough time getting this to work on windows, I installed cygwin but I am getting errors

C:\cygwin\home\false>cat  E:\FalseRoot-New\2-9-09\4-5-fix-subnets.txt   | while read line1
'while' is not recognized as an internal or external command,
operable program or batch file.

Any ideas?
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23653329
You need to be in a shell environment.

First, execute bash.exe (which is in the cygwin bin directory), then run the command as omarfarid showed.
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23653391
Instead of executing bash.exe directly, you could run cygwin.bat and it will cd you into C:\cygwin\bin and then execute 'bash --login -i'

When you installed cygwin, it should have created a link in "Start -> Programs -> Cygwin"
0
 
LVL 1

Author Comment

by:richsark
ID: 23653498
Ok, will do that now, also is there a way to make the output of the above script to left justify?

It seems that the Domain= is tabbed over more the SubnetAddress=

See below:

SubnetAddress=10.3.2.0
    Domain=disclosure.com private.disclosure.com
SubnetAddress=10.3.4.0
    Domain=beers.companystore.com

Thanks
0
 
LVL 1

Author Comment

by:richsark
ID: 23653660
HI, back to Fishmoger, yes I have a shortcut on my desk top. After I launched cygwin, I ran

bash --login -i

my file is called setsub.pl  its located in /home/Administrator/

The contents of setsub.pl is

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1
    $line2" > step.txt
    qip-setsubnet - u name-p pass -f /home/Administrator/4-5-fix-subnets.txt -o rich -t >> log.txt 2>&1
done
~

I executed ./setsub.pl

I got an error:

 ./setsub.pl: line 8: syntax error: unexpected end of file

Also, any ideas how to make the output line up to the left as my previous note above. I guess first things first :)

Thanks
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23653877
The .pl extension is normally for Perl scripts.  The code that omarfarid provided would be a shell script.  Which did you want?

If you want, I could show you the Perl script version.
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23653941

#!/usr/bin/perl
 
use strict;
use warnings;
 
open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>', 'step.txt' or die "Can't open 'step.txt' $!";
    print $FH2 $line1,$line2;
    system("qip-setsubnet -u name -p pass -f /home/Administrator/4-5-fix-subnets.txt -o rich -t >> log.txt 2>&1");
}

Open in new window

0
 
LVL 1

Author Comment

by:richsark
ID: 23653946
Hi FishMonger, please show me the perl version as another option.  I like Omarfarid's version but I am against the clock and I need to test this ASAP.

Could I also ask for a way to make the output left justify, I cant have any tabs where Domain=

SubnetAddress=10.3.2.0
    Domain=disclosure.com private.disclosure.com
SubnetAddress=10.3.4.0
    Domain=beers.companystore.com

Need it to look like:

SubnetAddress=10.3.2.0
Domain=disclosure.com private.disclosure.com
SubnetAddress=10.3.4.0
Domain=beers.companystore.com

Thanks all !
 
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23653957
I forgot 1 line.  Add this just after the print statement

close $FH2;
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23653986
If there is leading whitespace on the Domain= line, you can strip it out with a regex (regular expression).
#!/usr/bin/perl
 
use strict;
use warnings;
 
open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    $line2 =~ s/^\s*//;
    open my $FH2, '>', 'step.txt' or die "Can't open 'step.txt' $!";
    print $FH2 $line1,$line2;
    close $FH2;
    system("qip-setsubnet -u name -p pass -f /home/Administrator/4-5-fix-subnets.txt -o rich -t >> log.txt 2>&1");
}

Open in new window

0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23654013
One more thing, it might be better to use single quotes instead of double quotes in the system call.
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23654032
Hmm, the woos of not paying attention when doing copy/paste.  

system("qip-setsubnet -u name -p pass -f /home/Administrator/4-5-fix-subnets.txt -o rich -t >> log.txt 2>&1");

should be:
system('qip-setsubnet -u name -p pass -f /home/Administrator/step.txt -o rich -t >> log.txt 2>&1');
0
 
LVL 1

Author Comment

by:richsark
ID: 23654143
qip-setsubnet.exe :/home/Administrator/4-5-fix-subnets.txt: Invalid File Name?!qip-setsubnet.exe :/home/Administrator/4-5-fix-subnets.txt: Invalid File Name?!

I have the files located in /home/Administrator

Thanks
0
 
LVL 1

Author Comment

by:richsark
ID: 23654195
Hi FishMonger, I modified your above with:

system('qip-setsubnet -u name -p pass -f /home/Administrator/step.txt -o rich -t >> log.txt 2>&1');

When I run t he script and look at the log.txt I see tons of messages:

qip-setsubnet.exe :/home/Administrator/step.txt: Invalid File Name?!qip-setsubnet.exe :/home/Administrator/step.txt: Invalid File Name?!

Thanks
0
 
LVL 1

Author Comment

by:richsark
ID: 23654281
FishMonger, I ran the file manually, I get the same message, but the dir and path are good, see below:

Administrator@mobile6 ~
$ /cygdrive/e/qip-62/cli/qip-setsubnet -u xx -p xx -f /home/Administrator/step.txt -o rich -t
qip-setsubnet.exe :/home/Administrator/step.txt: Invalid File Name?!

Administrator@mobile6 ~
$ more step.txt
SubnetAddress=63.247.112.0
Domain=econfirm.com theice.com

Administrator@mobile6 ~
$ pwd
/home/Administrator

The file is there, I am not understanding why its invalid??
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23654329
I suspect that qip-setsubnet.exe expects a Windows path.  Exit out of the bash shell and run the perl script from a normal Windows command prompt.  You'll also need to change the path in the system call to use the full Windows path to the file.  I don't know if the cygwin install setup the proper file ext associations you may need to specify the full path to perl.exe and pass your script as a parameter.

Like this (but use the proper paths on your system):

c:\perl\bin\perl c:\cygwin\home\Administrator\setsub.pl

Open in new window

0
 
LVL 1

Author Comment

by:richsark
ID: 23654501
Cool, that worked in Windows after I made the path changes!! Awesome !

 I guess when I goto to the office, I will need to modify the script from windows to unix. I Assume it will work the same :)

Thanks guys.


0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23654547
Yes, it will work the same once the path is adjusted.  With a few extra lines we could make it so that it will work on either system without having to adjust paths.
0
 
LVL 1

Author Comment

by:richsark
ID: 23654573
Hi FishMonger, do you want to do that? If not I can modify the file. Its up to you !

Thanks
0
 
LVL 1

Author Comment

by:richsark
ID: 23655416
Hi FishMonger,  in reference to the log.txt file. Does that get over-written on each new step.txt call?

 I like for the logs to not over-write so I know what failed ( if any )

Could you confirm and or modify the perl script please?

Thanks!


0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23655726
There are several ways to check which OS the script is running on and the one I'm using is probably the most common.

I changed the name of the output file from step.txt to subnet.txt because that seemed more accurate.  I also corrected the output to the log file so that it will append instead of overwrite.

This script has the bare minimum amount of error handling.  Normally I'd add more, but this should get you started.
#!/usr/bin/perl
 
use strict;
use warnings;
 
my $subnet;
if ($^O eq 'MSWin32' ) {
    $subnet = 'c:\path\to\subnet.txt';
}
else {
    $subnet = '/path/to/subnet.txt';
}
 
open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>>', 'subnet.txt' or die "Can't open 'subnet.txt' $!";
    print $FH2 $line1,$line2;
    close $FH2;
    system("qip-setsubnet -u name -p pass -f $subnet -o rich -t >> log.txt 2>&1");
}

Open in new window

0
 
LVL 1

Author Comment

by:richsark
ID: 23656258
Hi FishMonger,

It appears that this new script will run on Win and Sun without any modifications besides where the files are?

I appericate your help and everyone else involved. I will try this new script  on a Sun box when I get to the office and will update you during the day.

Cheers !



0
 
LVL 1

Author Comment

by:richsark
ID: 23662989
HI FishMOnger, I ran the perl scripts on my Sun box today in the office, But I got these errors:
 
Can't locate warnings.pm in @INC (@INC contains: /usr/perl5/5.00503/sun4-solaris /usr/perl5/5.00503 /usr/perl5/site_perl/5.005/sun4-solaris /usr/perl5/site_perl/5.005 .) at ./setsubnet-Unix.pl line 24.
BEGIN failed--compilation aborted at ./setsubnet-Unix.pl line 24.
 
I used your old veriosn of the code:
 
use strict;
use warnings;
 
open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>', 'step.log' or die "Can't open 'step.log' $!";
    print $FH2 $line1,$line2;
    system('/opt/qip/usr/bin/qip-setsubnet -u Xxx -p Xxx-f /opt/qip/rich/scripts/step.log -o rich -t >> setsubnet.log 2>&1'
);
}
 
Any clues? or how can we modify it to work with this version of perl?
0
 
LVL 1

Author Comment

by:richsark
ID: 23663060
HI, I also tried to take out the two warnings from the file, but then I got different errors:
 The code looks like:
 open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>', 'step.log' or die "Can't open 'step.log' $!";
    print $FH2 $line1,$line2;
system("/opt/qip/usr/bin/qip-setsubnet -u XxX -p XxX-f /opt/qip/rich/scripts/step.log -o rich-t >> setsubnet.log 2>&1");
The error is :

<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>', 'step.log' or die "Can't open 'step.log' $!";
    print $FH2 $line1,$line2;
system("/opt/qip/usr/bin/qip-setsubnet -u XxX -p XxX -f /opt/qip/rich/scripts/step.log -o rich -t >> setsubnet.log 2>&1");
Please Help !
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23664331
That's a very old/outdated version (circa 1998).  I would highly recommend upgrading perl to at least 5.8.6 or preferably 5.10

That old version doesn't support the warnings or strict pragmas i.e.,
use strict;
use warnings;

It also doesn't support the use of the my keyword when declaring the vars as well as a number of other features.  Those features were introduced in 2000 in either v5.5 or v5.6

See if this version works better.
#!/usr/bin/perl -w
 
if ($^O eq 'MSWin32' ) {
    $subnet = 'c:\path\to\subnet.txt';
}
else {
    $subnet = '/path/to/subnet.txt';
}
 
open FH1, '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( $line1 = <FH1> ) {
    $line2 = <FH1>;
    open FH2, '>>subnet.txt' or die "Can't open 'subnet.txt' $!";
    print FH2 $line1,$line2;
    close FH2;
    system("qip-setsubnet -u name -p pass -f $subnet -o rich -t >> log.txt 2>&1");
}
close FH1;

Open in new window

0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23664514
I just noticed a goof on my part.  I'm opening subnet.txt in apend mode, instead it should overwite the file.

change
open FH2, '>>subnet.txt' or die "Can't open 'subnet.txt' $!";

to
open FH2, '>subnet.txt' or die "Can't open 'subnet.txt' $!";
0
 
LVL 1

Author Comment

by:richsark
ID: 23665546
Hi, Yea it was a surprise for me as well :)

I will try it in the morning.  I am not able to upgrade perl in the limited time I have. So I hope your script will have the work around for the version I have.

Do you have any doubts?

Thanks


0
 
LVL 1

Author Comment

by:richsark
ID: 23670874
Hey FishMonger,

Great work, the new code worked for the perl I have.  I will run this in production this weekend and let you know.

Thanks a bunch !
0
 
LVL 1

Author Comment

by:richsark
ID: 23675617
Hi FishMonger, can I ask you for a slight change in your perl to work with another request that is simular with what we done above?
Here is what I need to do
take a file called 2.1-rr-for-dns.csv using the same string with a different word:
qip-setzoneprof -u name -p pass -f 2.1-rr-for-dns.csv  -o rich
and have it run one section at a time: a section will be:
see file attached ( for format purposes)
you will notice a section is from rows 1 through 8 (and of course retain the colums)
I need to strip out a master file like we did above and run it 1 section at a time. A new section starts at Row 9 just so we are on the same page
I will throw in more points for ya !
Can you help me today :)
Thanks

2.1-rr-for-dns.csv.xls
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 23676132
It would be more appropriate to close this question and start a new one with the new requirements.

The file you've attached is not a plain text csv file, it's a binary excel spreadsheet.  The parsing of that file will be different and a little more complex.  If there aren't too many records in the file, you could add a blank line between each record which will help to simplify the parsing.  Saving/exporting the file to a plain text csv file would also help to simply the parsing.
0
 
LVL 1

Author Comment

by:richsark
ID: 23676623
Hi, Sorry about that, I ment to save it as a text file. I will re-upload.

Since it was my mistake on the file extension, could you take a look at it?

If not, I would be more then happy to open a new thread. But I was hoping that I could retain your services since you know what I need.

Please let me know

2.1-rr-for-dns.txt
0
 
LVL 1

Author Comment

by:richsark
ID: 23676703
HI, Sorry, I must be dumb, I attached the wrong format. It wont not allow me to upload as csv, I changed it to .txt

FishMonger. I would accept any ideas to get this to work. I am desperate !

Thanks


2.1-rr-for-dns.txt
0
 
LVL 28

Accepted Solution

by:
FishMonger earned 1000 total points
ID: 23676723
I'm happy to help you on this next step, but it still would be more appropriate to close this question and start a new one.

I have some other chores that I need to work on right now and I might not be able to do anything on this until tomorrow.  So, it would be to your benefit to open a new question to give others a chance to help in the mean time.
0
 
LVL 1

Author Comment

by:richsark
ID: 23676737
Ok, cool, I was hoping I would have something by mid-afternoon Thurs. Anyway Please do what you can do, and I will open a new thread.

Thanks and I look fwd in your follow up.

Thanks!
0
 
LVL 1

Author Comment

by:richsark
ID: 23676767
Thanks FishMonger !!
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Six Sigma Control Plans
Suggested Courses

850 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question