?
Solved

Spliting a file based on the number of records

Posted on 2013-11-08
5
Medium Priority
?
397 Views
Last Modified: 2013-11-08
I found a script - on this website - that's based on splitting a file based on a pattern, but when I tried to insert an if statement to split the based on the number of occurrences of that pattern it didn't work as I desired (it returned the pattern, but nothing else.) Underneath is what I tried.

Basically I would like the script to split the file if it matches more than n number (in this case 6000) pattern matches of </product>


#!/usr/bin/perl
use strict;

my $i=1;
my $open_product_count=0;
open (DATA,"file.txt") or die;
while(<DATA>){


    while (/<\/product>/ig) {
        $open_product_count++;
    }

if ($open_product_count > 6000) {

  open (FILE,">>file_$i.txt") or die;
  print FILE $_;
  if ($_=~/^<\/product>/){$i++;}
  close (FILE);


}

 }
 close (DATA);
0
Comment
Question by:hadrons
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
5 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 39633921
I'm trying to figure out what you intended to do.
Which was the  if statement you inserted, and what do you mean by "it returned the pattern"?
0
 

Author Comment

by:hadrons
ID: 39633990
I'm sorry for not being clearer in my original question, but basically, if a file has over 6,000 occurrences of </product> (this is the end of a record set) then I want the file to split into smaller files with no more than 6,000 record sets.

I used the code to keep track of the number of times </product> appears and to start splitting at that point:


    while (/<\/product>/ig) {
        $open_product_count++;
    }

if ($open_product_count > 6000) { ...}

I end up with two files that had just </product> in them.
0
 
LVL 84

Expert Comment

by:ozo
ID: 39634024
I think you should have ended up with one file for each time <\/product> appears at the beginning of a line in file.txt, after the <\/product> has been seen 6000 times anywhere on each line.
0
 
LVL 84

Accepted Solution

by:
ozo earned 2000 total points
ID: 39634077
# if I'm understanding what you intended to do, you might have been trying to do something like this:

my $i=1;
my $open_product_count=0;
open (DATA,"file.txt") or die;
open (FILE,">file_$i.txt") or die;
while(<DATA>){
    print FILE;
    while (/<\/product>/ig) {
        if( !(++$open_product_count%6000) ){
           close (FILE);
           $i++;
           open (FILE,">file_$i.txt") or die;
        }
    }
}
close FILE;
close (DATA);
0
 

Author Closing Comment

by:hadrons
ID: 39634520
Excellent ... thankfully I didn't create 6,000 file with my code, but your works perfectly
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
If you haven’t already, I encourage you to read the first article (http://www.experts-exchange.com/articles/18680/An-Introduction-to-R-Programming-and-R-Studio.html) in my series to gain a basic foundation of R and R Studio.  You will also find the …
Six Sigma Control Plans
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…

719 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question