Need a way to loop a input file

Hello all,  I am in need of a way to create a loop effect to enter information from this file 2 lines at a time. I need a way to strip this file and have it run. The way it stands now, the CLI ( command line interface does not understand multiple lines. So I need to break it down by these two variables

SubnetAddress=165.168.31.0
Domain=emcbi.com beer.com is.beer.com fi.beer.com emc.beer.com

The CLI input string is used like to

qip-setsubnet - u username -p password -f <input file name > -o "Organization name" -t <to append these to existing subnets already in  the system>

so

qip-setsubnet - u username -p password -f e:\ 4-5-fix-subnets.txt -o rich -t

I have perl  on my system if this is the way to go, but I dont have the skills yet to do the above. I can except any ideas that will make this work. I need to be able to take this solution and use it on either windows or Sun O/S's

I have attached the file that needs to be stripped and ran with two variables at a time.

Thank you in advance for your prompt replies
4-5-fix-subnets.txt
LVL 1
richsarkAsked:
Who is Participating?
 
FishMongerConnect With a Mentor Commented:
I'm happy to help you on this next step, but it still would be more appropriate to close this question and start a new one.

I have some other chores that I need to work on right now and I might not be able to do anything on this until tomorrow.  So, it would be to your benefit to open a new question to give others a chance to help in the mean time.
0
 
omarfaridCommented:
see the script below which is an example:

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1 $line2"
done

you may replace the echo line by any line of your's
0
 
richsarkAuthor Commented:
Hi, so how do I call it to run on the next set all the way down the file?
Thanks
0
Upgrade your Question Security!

Your question, your audience. Choose who sees your identity—and your question—with question security.

 
omarfaridCommented:
it does that in the while loop, it reads one line with the while read and the next line by the 2nd read. this will loop till the input file is read till end of file
0
 
richsarkAuthor Commented:
Ok. so where do I put my cli string to get them in?
qip-setsubnet - u username -p password -f e:\ 4-5-fix-subnets.txt -o rich -t
also, when I ran a test of the cat  it shows this
SubnetAddress=172.30.80.0 Domain=beer.com is.beer.com fi.beer.com emc.beer.com encorecredit.com
SubnetAddress=172.30.82.0 Domain=beer.com is.beer.com fi.beer.com emc.beer.com encorecredit.com
SubnetAddress=169.93.158.0 Domain=net.uk.jp.net
SubnetAddress=207.162.226.0 Domain=bear.com is.beer.com fi.beer.com emc.beer.com clearco.com
SubnetAddress=206.181.213.0 Domain=disclosure.com private.disclosure.com
I need it to be on two lines like so:
SubnetAddress=172.30.80.0
Domain=bear.com is.beer.com fi.bear.com emc.beer.com encorecredit.com

Thanks
0
 
omarfaridCommented:

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo $line1
    echo $line2
done

I am not sure what you want to do with

qip-setsubnet - u username -p password -f e:\ 4-5-fix-subnets.txt -o rich -t
0
 
richsarkAuthor Commented:
Hello Sir,
in order to get this two lines in the system, I have to call a CLI (see above) and reference the txt file.
The issue I have is that the CLI does not understand this massive file and I need to call it by two lines at a time. The two lines are mandatory and must have the header
SubnetAddress=
Domain=
So I need a way to automate this so its takes my input file, strip it down into two headers have them go into the system use the
CLI qip-setsubnet string and run all the way down the files.
0
 
omarfaridCommented:
can you give example of how the cli should be called with sample data?
0
 
richsarkAuthor Commented:
Hi,
I need to take the file I have attached "4-5-fix-subnets.txt "
cut it down into two headers

SubnetAddress=
Domain=
and run using the cli to call the file.  
So I guess need to perhaps pipe each of the two headers out into a scratch file that makes an input file that is called by the CLI and the whole process repeats all the way down the master file called 4-5-fix-subnets.txt

Does that help?
0
 
FishMongerCommented:
What is the syntax of the qip-setsubnet command when passing the 2 parameters instead of the file?
0
 
omarfaridCommented:
try this:

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1
    $line2" | qip-setsubnet - u username -p password -o rich -t
done
0
 
richsarkAuthor Commented:
Hi,
We need to create a scratch file called perhaps step1.txt which has the two headers from the master file "4-5-fix-subnets.txt"
Then take the file step1.txt and have it standby so the CLI can call it using
qip-setsubnet - u username -p password -f e:\ step1.txt -o rich -t  
Then it will need to repeat to the next set ofheaders create another file called perhaps step2.txt have it standby while the CLI is called agian to reference the step2.txt
qip-setsubnet - u username -p password -f e:\ step2.txt -o rich -t  
and so on.... all the way down the master file "4-5-fix-subnets.txt"
Does that help?
0
 
omarfaridCommented:
try

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1
    $line2" > step.txt
    qip-setsubnet - u username -p password -f step.txt -o rich -t
done

0
 
richsarkAuthor Commented:
Hello, so your last commet will run all the way down the file calling the same file step.txt until its done? so it step.txt will be overwritten with the next step right?
Is there a stop or a way to find out if it failed at some stage?
0
 
omarfaridCommented:
yes it will overwrite the temp file.

you could save output or errors into file:

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1
    $line2" > step.txt
    qip-setsubnet - u username -p password -f step.txt -o rich -t >> log.txt 2>&1
done
0
 
richsarkAuthor Commented:
Ok, will try it now and let you know in a few minutes. I need to install CYGWIN on my windows to test this as I dont have access today to a sun box
Thanks
0
 
richsarkAuthor Commented:
HI, I am having a rough time getting this to work on windows, I installed cygwin but I am getting errors

C:\cygwin\home\false>cat  E:\FalseRoot-New\2-9-09\4-5-fix-subnets.txt   | while read line1
'while' is not recognized as an internal or external command,
operable program or batch file.

Any ideas?
0
 
FishMongerCommented:
You need to be in a shell environment.

First, execute bash.exe (which is in the cygwin bin directory), then run the command as omarfarid showed.
0
 
FishMongerCommented:
Instead of executing bash.exe directly, you could run cygwin.bat and it will cd you into C:\cygwin\bin and then execute 'bash --login -i'

When you installed cygwin, it should have created a link in "Start -> Programs -> Cygwin"
0
 
richsarkAuthor Commented:
Ok, will do that now, also is there a way to make the output of the above script to left justify?

It seems that the Domain= is tabbed over more the SubnetAddress=

See below:

SubnetAddress=10.3.2.0
    Domain=disclosure.com private.disclosure.com
SubnetAddress=10.3.4.0
    Domain=beers.companystore.com

Thanks
0
 
richsarkAuthor Commented:
HI, back to Fishmoger, yes I have a shortcut on my desk top. After I launched cygwin, I ran

bash --login -i

my file is called setsub.pl  its located in /home/Administrator/

The contents of setsub.pl is

cat 4-5-fix-subnets.txt | while read line1
do
    read line2
    echo "$line1
    $line2" > step.txt
    qip-setsubnet - u name-p pass -f /home/Administrator/4-5-fix-subnets.txt -o rich -t >> log.txt 2>&1
done
~

I executed ./setsub.pl

I got an error:

 ./setsub.pl: line 8: syntax error: unexpected end of file

Also, any ideas how to make the output line up to the left as my previous note above. I guess first things first :)

Thanks
0
 
FishMongerCommented:
The .pl extension is normally for Perl scripts.  The code that omarfarid provided would be a shell script.  Which did you want?

If you want, I could show you the Perl script version.
0
 
FishMongerCommented:

#!/usr/bin/perl
 
use strict;
use warnings;
 
open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>', 'step.txt' or die "Can't open 'step.txt' $!";
    print $FH2 $line1,$line2;
    system("qip-setsubnet -u name -p pass -f /home/Administrator/4-5-fix-subnets.txt -o rich -t >> log.txt 2>&1");
}

Open in new window

0
 
richsarkAuthor Commented:
Hi FishMonger, please show me the perl version as another option.  I like Omarfarid's version but I am against the clock and I need to test this ASAP.

Could I also ask for a way to make the output left justify, I cant have any tabs where Domain=

SubnetAddress=10.3.2.0
    Domain=disclosure.com private.disclosure.com
SubnetAddress=10.3.4.0
    Domain=beers.companystore.com

Need it to look like:

SubnetAddress=10.3.2.0
Domain=disclosure.com private.disclosure.com
SubnetAddress=10.3.4.0
Domain=beers.companystore.com

Thanks all !
 
0
 
FishMongerCommented:
I forgot 1 line.  Add this just after the print statement

close $FH2;
0
 
FishMongerCommented:
If there is leading whitespace on the Domain= line, you can strip it out with a regex (regular expression).
#!/usr/bin/perl
 
use strict;
use warnings;
 
open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    $line2 =~ s/^\s*//;
    open my $FH2, '>', 'step.txt' or die "Can't open 'step.txt' $!";
    print $FH2 $line1,$line2;
    close $FH2;
    system("qip-setsubnet -u name -p pass -f /home/Administrator/4-5-fix-subnets.txt -o rich -t >> log.txt 2>&1");
}

Open in new window

0
 
FishMongerCommented:
One more thing, it might be better to use single quotes instead of double quotes in the system call.
0
 
FishMongerCommented:
Hmm, the woos of not paying attention when doing copy/paste.  

system("qip-setsubnet -u name -p pass -f /home/Administrator/4-5-fix-subnets.txt -o rich -t >> log.txt 2>&1");

should be:
system('qip-setsubnet -u name -p pass -f /home/Administrator/step.txt -o rich -t >> log.txt 2>&1');
0
 
richsarkAuthor Commented:
qip-setsubnet.exe :/home/Administrator/4-5-fix-subnets.txt: Invalid File Name?!qip-setsubnet.exe :/home/Administrator/4-5-fix-subnets.txt: Invalid File Name?!

I have the files located in /home/Administrator

Thanks
0
 
richsarkAuthor Commented:
Hi FishMonger, I modified your above with:

system('qip-setsubnet -u name -p pass -f /home/Administrator/step.txt -o rich -t >> log.txt 2>&1');

When I run t he script and look at the log.txt I see tons of messages:

qip-setsubnet.exe :/home/Administrator/step.txt: Invalid File Name?!qip-setsubnet.exe :/home/Administrator/step.txt: Invalid File Name?!

Thanks
0
 
richsarkAuthor Commented:
FishMonger, I ran the file manually, I get the same message, but the dir and path are good, see below:

Administrator@mobile6 ~
$ /cygdrive/e/qip-62/cli/qip-setsubnet -u xx -p xx -f /home/Administrator/step.txt -o rich -t
qip-setsubnet.exe :/home/Administrator/step.txt: Invalid File Name?!

Administrator@mobile6 ~
$ more step.txt
SubnetAddress=63.247.112.0
Domain=econfirm.com theice.com

Administrator@mobile6 ~
$ pwd
/home/Administrator

The file is there, I am not understanding why its invalid??
0
 
FishMongerCommented:
I suspect that qip-setsubnet.exe expects a Windows path.  Exit out of the bash shell and run the perl script from a normal Windows command prompt.  You'll also need to change the path in the system call to use the full Windows path to the file.  I don't know if the cygwin install setup the proper file ext associations you may need to specify the full path to perl.exe and pass your script as a parameter.

Like this (but use the proper paths on your system):

c:\perl\bin\perl c:\cygwin\home\Administrator\setsub.pl

Open in new window

0
 
richsarkAuthor Commented:
Cool, that worked in Windows after I made the path changes!! Awesome !

 I guess when I goto to the office, I will need to modify the script from windows to unix. I Assume it will work the same :)

Thanks guys.


0
 
FishMongerCommented:
Yes, it will work the same once the path is adjusted.  With a few extra lines we could make it so that it will work on either system without having to adjust paths.
0
 
richsarkAuthor Commented:
Hi FishMonger, do you want to do that? If not I can modify the file. Its up to you !

Thanks
0
 
richsarkAuthor Commented:
Hi FishMonger,  in reference to the log.txt file. Does that get over-written on each new step.txt call?

 I like for the logs to not over-write so I know what failed ( if any )

Could you confirm and or modify the perl script please?

Thanks!


0
 
FishMongerCommented:
There are several ways to check which OS the script is running on and the one I'm using is probably the most common.

I changed the name of the output file from step.txt to subnet.txt because that seemed more accurate.  I also corrected the output to the log file so that it will append instead of overwrite.

This script has the bare minimum amount of error handling.  Normally I'd add more, but this should get you started.
#!/usr/bin/perl
 
use strict;
use warnings;
 
my $subnet;
if ($^O eq 'MSWin32' ) {
    $subnet = 'c:\path\to\subnet.txt';
}
else {
    $subnet = '/path/to/subnet.txt';
}
 
open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>>', 'subnet.txt' or die "Can't open 'subnet.txt' $!";
    print $FH2 $line1,$line2;
    close $FH2;
    system("qip-setsubnet -u name -p pass -f $subnet -o rich -t >> log.txt 2>&1");
}

Open in new window

0
 
richsarkAuthor Commented:
Hi FishMonger,

It appears that this new script will run on Win and Sun without any modifications besides where the files are?

I appericate your help and everyone else involved. I will try this new script  on a Sun box when I get to the office and will update you during the day.

Cheers !



0
 
richsarkAuthor Commented:
HI FishMOnger, I ran the perl scripts on my Sun box today in the office, But I got these errors:
 
Can't locate warnings.pm in @INC (@INC contains: /usr/perl5/5.00503/sun4-solaris /usr/perl5/5.00503 /usr/perl5/site_perl/5.005/sun4-solaris /usr/perl5/site_perl/5.005 .) at ./setsubnet-Unix.pl line 24.
BEGIN failed--compilation aborted at ./setsubnet-Unix.pl line 24.
 
I used your old veriosn of the code:
 
use strict;
use warnings;
 
open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>', 'step.log' or die "Can't open 'step.log' $!";
    print $FH2 $line1,$line2;
    system('/opt/qip/usr/bin/qip-setsubnet -u Xxx -p Xxx-f /opt/qip/rich/scripts/step.log -o rich -t >> setsubnet.log 2>&1'
);
}
 
Any clues? or how can we modify it to work with this version of perl?
0
 
richsarkAuthor Commented:
HI, I also tried to take out the two warnings from the file, but then I got different errors:
 The code looks like:
 open my $FH1, '<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>', 'step.log' or die "Can't open 'step.log' $!";
    print $FH2 $line1,$line2;
system("/opt/qip/usr/bin/qip-setsubnet -u XxX -p XxX-f /opt/qip/rich/scripts/step.log -o rich-t >> setsubnet.log 2>&1");
The error is :

<', '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
while ( my $line1 = <$FH1> ) {
    my $line2 = <$FH1>;
    open my $FH2, '>', 'step.log' or die "Can't open 'step.log' $!";
    print $FH2 $line1,$line2;
system("/opt/qip/usr/bin/qip-setsubnet -u XxX -p XxX -f /opt/qip/rich/scripts/step.log -o rich -t >> setsubnet.log 2>&1");
Please Help !
0
 
FishMongerCommented:
That's a very old/outdated version (circa 1998).  I would highly recommend upgrading perl to at least 5.8.6 or preferably 5.10

That old version doesn't support the warnings or strict pragmas i.e.,
use strict;
use warnings;

It also doesn't support the use of the my keyword when declaring the vars as well as a number of other features.  Those features were introduced in 2000 in either v5.5 or v5.6

See if this version works better.
#!/usr/bin/perl -w
 
if ($^O eq 'MSWin32' ) {
    $subnet = 'c:\path\to\subnet.txt';
}
else {
    $subnet = '/path/to/subnet.txt';
}
 
open FH1, '4-5-fix-subnets.txt' or die "Can't open '4-5-fix-subnets.txt' $!";
 
while ( $line1 = <FH1> ) {
    $line2 = <FH1>;
    open FH2, '>>subnet.txt' or die "Can't open 'subnet.txt' $!";
    print FH2 $line1,$line2;
    close FH2;
    system("qip-setsubnet -u name -p pass -f $subnet -o rich -t >> log.txt 2>&1");
}
close FH1;

Open in new window

0
 
FishMongerCommented:
I just noticed a goof on my part.  I'm opening subnet.txt in apend mode, instead it should overwite the file.

change
open FH2, '>>subnet.txt' or die "Can't open 'subnet.txt' $!";

to
open FH2, '>subnet.txt' or die "Can't open 'subnet.txt' $!";
0
 
richsarkAuthor Commented:
Hi, Yea it was a surprise for me as well :)

I will try it in the morning.  I am not able to upgrade perl in the limited time I have. So I hope your script will have the work around for the version I have.

Do you have any doubts?

Thanks


0
 
richsarkAuthor Commented:
Hey FishMonger,

Great work, the new code worked for the perl I have.  I will run this in production this weekend and let you know.

Thanks a bunch !
0
 
richsarkAuthor Commented:
Hi FishMonger, can I ask you for a slight change in your perl to work with another request that is simular with what we done above?
Here is what I need to do
take a file called 2.1-rr-for-dns.csv using the same string with a different word:
qip-setzoneprof -u name -p pass -f 2.1-rr-for-dns.csv  -o rich
and have it run one section at a time: a section will be:
see file attached ( for format purposes)
you will notice a section is from rows 1 through 8 (and of course retain the colums)
I need to strip out a master file like we did above and run it 1 section at a time. A new section starts at Row 9 just so we are on the same page
I will throw in more points for ya !
Can you help me today :)
Thanks

2.1-rr-for-dns.csv.xls
0
 
FishMongerCommented:
It would be more appropriate to close this question and start a new one with the new requirements.

The file you've attached is not a plain text csv file, it's a binary excel spreadsheet.  The parsing of that file will be different and a little more complex.  If there aren't too many records in the file, you could add a blank line between each record which will help to simplify the parsing.  Saving/exporting the file to a plain text csv file would also help to simply the parsing.
0
 
richsarkAuthor Commented:
Hi, Sorry about that, I ment to save it as a text file. I will re-upload.

Since it was my mistake on the file extension, could you take a look at it?

If not, I would be more then happy to open a new thread. But I was hoping that I could retain your services since you know what I need.

Please let me know

2.1-rr-for-dns.txt
0
 
richsarkAuthor Commented:
HI, Sorry, I must be dumb, I attached the wrong format. It wont not allow me to upload as csv, I changed it to .txt

FishMonger. I would accept any ideas to get this to work. I am desperate !

Thanks


2.1-rr-for-dns.txt
0
 
richsarkAuthor Commented:
Ok, cool, I was hoping I would have something by mid-afternoon Thurs. Anyway Please do what you can do, and I will open a new thread.

Thanks and I look fwd in your follow up.

Thanks!
0
 
richsarkAuthor Commented:
Thanks FishMonger !!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.