Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium


PERL and Text Files -- My Head Hurts

Posted on 1998-10-27
Medium Priority
Last Modified: 2010-03-04
I'd really like someone to explain to me, in words that a dummy like me can understand, how Perl reads or handles text files.

Specifically, I don't understand what that


thing means.

Say I was trying to write a script which would pick a random line from a 100-line text file.

I open the file with an

     or die "can't find file"

error handler, I understand it this far, but then in order to do something with the file I need

     do my stuff
Which I don't really understand -- does while(<INFILE>) mean "read every line of the file"? Does it mean "put every line of the file into $_ one by one"? Does it mean "keep on reading the file until some condition is satisfied, at which time whatever line I'm reading will be placed in $_"?

If I want to generate a random number, then pick that line from my 100 lines, how do I script that? And if I wanted to repeat the "get a random line" process a random number of times, how would I script that?

The trouble is that in my head the process goes like this:
     Generate a random number X
     Open the file
     Get line X of the file
Which ought to be simple, but PERL doesn't seem to do it like that. As far as I can work out from the book I have, it has to read each line of the file into memory and throw them away before it gets to the one it wants.

And if I want to repeat the process, getting a random line from my file Y number of times, I see it as:

     Generate a random number Y
     Repeat Y number of times
          Generate a random number X
          Get line X of the file
     Stop repeating
And again, maybe it's just me, but I'm finding that difficult. DO I have to open the file every time? Do I have to do that while(<INFILE>) thing every time or just once?

I'll gladly give stacks of points to someone who can make me understand how this all works, because it's making my head hurt...
Question by:johnny99
  • 4
  • 2
  • 2

Expert Comment

ID: 1205751
>> while(<INFILE>){
>>   do my stuff
>> }

You're absolutely correct. While there are one or more lines left in the file to read it willl be put in $_ and true is returned. When true is returned it tells the while() statement to run once more. When end of file is reached, false is returned and the while() statement won't be run any more.

srand; // Gives better random numbers
$lineNumber = int(rand 100) + 1; // Get a number between 1 and 100
for ($i=1;$i<=$lineNumber;$i++)
$thatLine = $_;

will do the trick for you. I am not using a while() here. Insted I use a for loop.
It starts by setting $i to 1. Then it is run as long as $i is lower than or equal to (<=) $lineNumber. Every time the loop has run $i is incremented by one (++).

Then, when the lines have been read, the last that was read ($_) will be put in $thatLine.

>> Generate a random number Y
>> Repeat Y number of times
>>   Generate a random number X
>>   Get line X of the file
>> Stop repeating


>> DO I have to open the file every time? Do I have to do that while(<INFILE>) thing every time or just once?

Not really actually. You can put the lines in an array and use it every time which means you won't have to re-read the file:
times = int(rand 10) + 1 // Between 1 and 10
open (INFILE, "<file.txt") or die "open file: $!";
$lineCount = 0;
while (<INFILE>)
  $lines[$lineCount++] = $_; // Add line to array and increment $lineCount
close INFILE;
for ($i=0;$i<$times;$i++)
  $randomedLines[$i] = $lines[int(rand 100)+1];

The lines will now be put in @randomedLines.

LVL 85

Expert Comment

ID: 1205752
perldoc -q 'random line'
Found in perlfaq5.pod
  How do I select a random line from a file?

            Here's an algorithm from the Camel Book:

                rand($.) < 1 && ($line = $_) while <>;

            This has a significant advantage in space over reading
            the whole file in. A simple proof by induction is
            available upon request if you doubt its correctness.

                rand($.) < 1 && ($line = $_) while <>;
LVL 85

Expert Comment

ID: 1205753
@lines = <INFILE>; #grab all lines from the file

# fisher_yates_shuffle( \@array ) :
# generate a random permutation of @array in place
sub fisher_yates_shuffle {
        my $array = shift;
        my $i;
        for( $i = @$array; --$i; ){
            my $j = int rand ($i+1);
            next if $i == $j;
            @$array[$i,$j] = @$array[$j,$i];

fisher_yates_shuffle( \@lines );  # permutes @lines
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.


Author Comment

ID: 1205754
Martinag, you can certainly have the points, for being the only person to even try and answer the question in the way it was asked, i.e. slowly and carefully.

Other people who replied, I'm sure your stuff is very clever, but the information about what all of that stuff means is more valuable than any amount of neat solutions, whether they work or not...

Can you tell me a good place on the web or a good book to help me with questions like this?
LVL 85

Expert Comment

ID: 1205755
perldoc -q book
Found in perlfaq2.pod
  Perl Books

            A number of books on Perl and/or CGI programming are
            available. A few of these are good, some are ok, but
            many aren't worth your money. Tom Christiansen maintains
            a list of these books, some with extensive reviews, at

            The incontestably definitive reference book on Perl,
            written by the creator of Perl, is now in its second

                Programming Perl (the "Camel Book"):
                    Authors: Larry Wall, Tom Christiansen, and Randal Schwartz
                    ISBN 1-56592-149-6      (English)
                    ISBN 4-89052-384-7      (Japanese)
                    URL: http://www.oreilly.com/catalog/pperl2/
                (French, German, Italian, and Hungarian translations also

            The companion volume to the Camel containing thousands
            of real-world examples, mini-tutorials, and complete
            programs (first premiering at the 1998 Perl Conference),

                The Perl Cookbook (the "Ram Book"):
                    Authors: Tom Christiansen and Nathan Torkington,
                                with Foreword by Larry Wall
                    ISBN: 1-56592-243-3
                    URL:  http://perl.oreilly.com/cookbook/

            If you're already a hard-core systems programmer, then
            the Camel Book might suffice for you to learn Perl from.
            But if you're not, check out:

                Learning Perl (the "Llama Book"):
                    Authors: Randal Schwartz and Tom Christiansen
                                with Foreword by Larry Wall
                    ISBN: 1-56592-284-0
                    URL:  http://www.oreilly.com/catalog/lperl2/

            Despite the picture at the URL above, the second edition
            of "Llama Book" really has a blue cover, and is updated
            for the 5.004 release of Perl. Various foreign language
            editions are available, including *Learning Perl on
            Win32 Systems* (the Gecko Book).

            If you're not an accidental programmer, but a more
            serious and possibly even degreed computer scientist who
            doesn't need as much hand-holding as we try to provide
            in the Llama or its defurred cousin the Gecko, please
            check out the delightful book, *Perl: The Programmer's
            Companion*, written by Nigel Chapman.

            You can order O'Reilly books directly from O'Reilly &
            Associates, 1-800-998-9938. Local/overseas is 1-707-829-
            0515. If you can locate an O'Reilly order form, you can
            also fax to 1-707-829-0104. See http://www.ora.com/ on
            the Web.

            What follows is a list of the books that the FAQ authors
            found personally useful. Your mileage may (but, we hope,
            probably won't) vary.

            Recommended books on (or muchly on) Perl follow; those
            marked with a star may be ordered from O'Reilly.

    References  *Programming Perl by Larry Wall, Tom Christiansen, and
                Randal L. Schwartz

                    *Perl 5 Desktop Reference
                        By Johan Vromans

        *Learning Perl [2nd edition]
            by Randal L. Schwartz and Tom Christiansen
                with foreword by Larry Wall
                    *Learning Perl on Win32 Systems
                        by Randal L. Schwartz, Erik Olson, and Tom Christiansen,
                            with foreword by Larry Wall

                    Perl: The Programmer's Companion
                        by Nigel Chapman

                    Cross-Platform Perl
                        by Eric F. Johnson

                    MacPerl: Power and Ease
                        by Vicki Brown and Chris Nandor, foreword by Matthias Neeracher

                    *The Perl Cookbook
                        by Tom Christiansen and Nathan Torkington
                            with foreword by Larry Wall

                    Perl5 Interactive Course [2nd edition]
                        by Jon Orwant

                    *Advanced Perl Programming
                        by Sriram Srinivasan

                    Effective Perl Programming
                        by Joseph Hall

    Special Topics
                    *Mastering Regular Expressions
                        by Jeffrey Friedl

                    How to Set up and Maintain a World Wide Web Site [2nd edition]
                        by Lincoln Stein


Author Comment

ID: 1205756
Thanks Ozo -- it sounds as if I'm a lama, rather than a camel, at this stage.

johnny "that's pronounced 'lay-muh', right?" 99
LVL 85

Expert Comment

ID: 1205757
will also give you access to much of the information in the camel,
as well as answers to many common questions.

  while( <INFILE> ){ }
is actually a magic shorthand for
  while( defined($_=<INFILE>) ){ }
this is covered in
perldoc perlop
under I/O Operators

Accepted Solution

martinag earned 600 total points
ID: 1205758
All right, here's an answer.

I hope you will enjoy your perl programming in the future!


Featured Post


Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I've just discovered very important differences between Windows an Unix formats in Perl,at least 5.xx.. MOST IMPORTANT: Use Unix file format while saving Your script. otherwise it will have ^M s or smth likely weird in the EOL, Then DO NOT use m…
On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans
Suggested Courses

578 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question