• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 194
  • Last Modified:

Generating multiple files

Hello,

The following (perl -i.bak -pe '$_.="\n\n" unless $.%2000' ) produces an output with two \n after every 2k records.

The files generated have no specific column length and (in some cases 98K records in others 157k).

I'm looking for a way to make an adjustment to the above so the final output created is in batches of 50k records (not inclusive of \n)

Thank you
0
faithless1
Asked:
faithless1
  • 3
  • 2
3 Solutions
 
tel2Commented:
Hi ff1,

How about this:
    perl -i.bak -pe '$len+=length;print "\n\n",$len=0 if $len>51200'
0
 
tel2Commented:
Correction to my post above:
    perl -i.bak -pe '$len+=length;$len=length,print "\n\n" if $len>51200'

This still won't always create exactly 50K (51,200 byte) paragraphs, since it doesn't split lines.  Is that a concern ff1?  If so, do you want lines to be split?

By the way, this script is similar to ozo's one, but neither "generate multiple files", as the subject of this thread suggests.  Are you wanting multiple files?
0
 
faithless1Author Commented:
Hi tel2,

Yes, I'm looking to generate multiple files from a file with 467192 records. Every 2k records will be followed by 2 empty lines.
0
 
faithless1Author Commented:
This command: perl -i.bak -pe '$_.="\n\n" unless $.%2000'

Considering that I will truncate the file to 450K for simplicity, the above script will add 432 empty lines so then I will need to output each batch to a different file.

50048 - 1st

100096 - 2nd etc
0
 
tel2Commented:
Sorry fl1, I still don't understand the requirements, and I doubt I will be putting more time into it.
Your initial description (apart from the heading) says nothing about multiple files.  The code you've supplied doesn't generate multiple files - it generates one file, and puts 2 newlines after each 2000 lines.  And your requested adjustment to that code is: "...so the final output created is in batches of 50k records (not inclusive of \n)".  No mention of multiple files there.

If you want to make this clearer for me or anyone else who might work on this:
- Please explain the whole thing clearly.
- Provide sample input and output, including filenames.  (For conciseness, if you want to abbreviate it, by putting "...etc..." in the middle of the data, feel free.)

By the way, why do you want this?

Thanks.
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

  • 3
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now