Solved

Spliting a file based on the number of records

Posted on 2013-11-08
5
393 Views
Last Modified: 2013-11-08
I found a script - on this website - that's based on splitting a file based on a pattern, but when I tried to insert an if statement to split the based on the number of occurrences of that pattern it didn't work as I desired (it returned the pattern, but nothing else.) Underneath is what I tried.

Basically I would like the script to split the file if it matches more than n number (in this case 6000) pattern matches of </product>


#!/usr/bin/perl
use strict;

my $i=1;
my $open_product_count=0;
open (DATA,"file.txt") or die;
while(<DATA>){


    while (/<\/product>/ig) {
        $open_product_count++;
    }

if ($open_product_count > 6000) {

  open (FILE,">>file_$i.txt") or die;
  print FILE $_;
  if ($_=~/^<\/product>/){$i++;}
  close (FILE);


}

 }
 close (DATA);
0
Comment
Question by:hadrons
  • 3
  • 2
5 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 39633921
I'm trying to figure out what you intended to do.
Which was the  if statement you inserted, and what do you mean by "it returned the pattern"?
0
 

Author Comment

by:hadrons
ID: 39633990
I'm sorry for not being clearer in my original question, but basically, if a file has over 6,000 occurrences of </product> (this is the end of a record set) then I want the file to split into smaller files with no more than 6,000 record sets.

I used the code to keep track of the number of times </product> appears and to start splitting at that point:


    while (/<\/product>/ig) {
        $open_product_count++;
    }

if ($open_product_count > 6000) { ...}

I end up with two files that had just </product> in them.
0
 
LVL 84

Expert Comment

by:ozo
ID: 39634024
I think you should have ended up with one file for each time <\/product> appears at the beginning of a line in file.txt, after the <\/product> has been seen 6000 times anywhere on each line.
0
 
LVL 84

Accepted Solution

by:
ozo earned 500 total points
ID: 39634077
# if I'm understanding what you intended to do, you might have been trying to do something like this:

my $i=1;
my $open_product_count=0;
open (DATA,"file.txt") or die;
open (FILE,">file_$i.txt") or die;
while(<DATA>){
    print FILE;
    while (/<\/product>/ig) {
        if( !(++$open_product_count%6000) ){
           close (FILE);
           $i++;
           open (FILE,">file_$i.txt") or die;
        }
    }
}
close FILE;
close (DATA);
0
 

Author Closing Comment

by:hadrons
ID: 39634520
Excellent ... thankfully I didn't create 6,000 file with my code, but your works perfectly
0

Featured Post

Active Directory Webinar

We all know we need to protect and secure our privileges, but where to start? Join Experts Exchange and ManageEngine on Tuesday, April 11, 2017 10:00 AM PDT to learn how to track and secure privileged users in Active Directory.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
This video teaches viewers about errors in exception handling.
The goal of the tutorial is to teach the user how to use functions in C++. The video will cover how to define functions, how to call functions and how to create functions prototypes. Microsoft Visual C++ 2010 Express will be used as a text editor an…

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question