Solved

Get and add first line of file to array

Posted on 2002-05-17
6
159 Views
Last Modified: 2010-03-05
I am reading big files.
First I read the totaly into memory and then did with that array what I had to do. This will cause memory problem with big files!

open(DATA,"$datafile")
@r_data = <DATA>;
close(DATA);

Searching the interne I found that it would be better to use the while loop. This will read line after line. It worked and my memory problems where gone.

open(DATA, $r_in->{'datafile'});
while (<DATA>) {
# do something with the data
# select specified lines
# and add them to the @r_data array.
push(@r_data,$_);
}
close(DATA);

The first line of the file (a flat database) contains the headers. How to read the first line of the file to the array @header using the while example?

0
Comment
Question by:mmcw
  • 4
6 Comments
 
LVL 7

Expert Comment

by:yoren
ID: 7015757
Try the following code. I've added "chomp" to remove the new line characters from the ends of the lines. Also note that I've made $header a scalar variable rather than an array, since it has only one line.

open(DATA, $r_in->{'datafile'});
$header = <DATA>;
chomp $header;

while (<DATA>) {
    chomp;
# do something with the data
# select specified lines
# and add them to the @r_data array.
    push(@r_data,$_);
}
close(DATA);


0
 
LVL 19

Accepted Solution

by:
Kim Ryan earned 50 total points
ID: 7015760
Use the special perl variable $. to track the input file line number

while (<DATA>)
{
        if ( $. == 1 )
     {
          push(@h_data,$_);
     }
     else
     {
          push(@r_data,$_);
     }
}
0
 
LVL 12

Expert Comment

by:lexxwern
ID: 7016613

$ctr = 0;

open(DATA, $r_in->{'datafile'});
while (<DATA>)
{
 if($ctr == 1)
 {
   push(@r_data,$_);
 }
 else
 {
   push(@h_data,$_);
 }
 $ctr++;
}
close(DATA);
0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 
LVL 7

Expert Comment

by:yoren
ID: 7016779
mmcw,

What's wrong with the answer I gave? If you tell us what problem you still have, we may be able to solve it.
0
 
LVL 7

Expert Comment

by:yoren
ID: 7016862
mmcw,

What's wrong with the answer I gave? If you tell us what problem you still have, we may be able to solve it.
0
 
LVL 7

Expert Comment

by:yoren
ID: 7016937
For your education, it's more efficient to read the header once rather than check the line number for every line read. With teraplane's code, you're checking every single line to see if it's the header line.
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This video discusses moving either the default database or any database to a new volume.

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now