Spliting a file based on the number of records

I found a script - on this website - that's based on splitting a file based on a pattern, but when I tried to insert an if statement to split the based on the number of occurrences of that pattern it didn't work as I desired (it returned the pattern, but nothing else.) Underneath is what I tried.

Basically I would like the script to split the file if it matches more than n number (in this case 6000) pattern matches of </product>


#!/usr/bin/perl
use strict;

my $i=1;
my $open_product_count=0;
open (DATA,"file.txt") or die;
while(<DATA>){


    while (/<\/product>/ig) {
        $open_product_count++;
    }

if ($open_product_count > 6000) {

  open (FILE,">>file_$i.txt") or die;
  print FILE $_;
  if ($_=~/^<\/product>/){$i++;}
  close (FILE);


}

 }
 close (DATA);
hadronsAsked:
Who is Participating?
 
ozoCommented:
# if I'm understanding what you intended to do, you might have been trying to do something like this:

my $i=1;
my $open_product_count=0;
open (DATA,"file.txt") or die;
open (FILE,">file_$i.txt") or die;
while(<DATA>){
    print FILE;
    while (/<\/product>/ig) {
        if( !(++$open_product_count%6000) ){
           close (FILE);
           $i++;
           open (FILE,">file_$i.txt") or die;
        }
    }
}
close FILE;
close (DATA);
0
 
ozoCommented:
I'm trying to figure out what you intended to do.
Which was the  if statement you inserted, and what do you mean by "it returned the pattern"?
0
 
hadronsAuthor Commented:
I'm sorry for not being clearer in my original question, but basically, if a file has over 6,000 occurrences of </product> (this is the end of a record set) then I want the file to split into smaller files with no more than 6,000 record sets.

I used the code to keep track of the number of times </product> appears and to start splitting at that point:


    while (/<\/product>/ig) {
        $open_product_count++;
    }

if ($open_product_count > 6000) { ...}

I end up with two files that had just </product> in them.
0
 
ozoCommented:
I think you should have ended up with one file for each time <\/product> appears at the beginning of a line in file.txt, after the <\/product> has been seen 6000 times anywhere on each line.
0
 
hadronsAuthor Commented:
Excellent ... thankfully I didn't create 6,000 file with my code, but your works perfectly
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.