Appending some data to a flat text file

Posted on 2009-03-31
Last Modified: 2013-12-25
I have a Programme below like this, which will take a file and create some keys and values and append to a XML File.

I have written functions for all this XML Functions.

while i try to do for big files, XML is creating very big problem to process huge data.  So i am looking to append the genrated key and values to a flat text file. How can i do this. Please help.

a Big Thanks in advance.

use Utils::XMLUtils qw(loadXMLFile














use Digest::SHA1 qw(sha1);

sub getHexData {

    my ($key)= unpack("H*",shift);

    return $key; 


print "Enter the file path :";

my $file = <STDIN>;


print "Enter chunk size :";

my $size = <STDIN>;


open(FILE, $file) || die "Unable to open the file";

my $doc = createXMLDocument();

my $SigData = createNewElement($doc, "SigData");

$len = read(FILE, $data, $size);

while ( $len != 0) {

  $datalen = $size == $len ? $size: $len;

  $digest = sha1($data);

  $keydata = getHexData($digest);

  $data = getHexData($data);

  $kvpair = createNewElement($doc, "KVPair");

  $key = createNewElement($doc, "Key");

  $keytext = createTextElement($doc, $keydata);

  assignChild($key, $keytext);

  $Value = createNewElement($doc, "Value");

  $Valuetext = createTextElement($doc, $data);

  assignChild($Value, $Valuetext);

  assignChild($kvpair, $key);

  assignChild($kvpair, $Value);

  assignChild($SigData, $kvpair);

  $len = read(FILE, $data, $size);




Open in new window

Question by:koti688
LVL 39

Expert Comment

ID: 24032944
What do you want your flat file to look like?  Should it be csv?
LVL 26

Accepted Solution

wilcoxon earned 500 total points
ID: 25953710
Have you considered writing out the XML file as you go?  I can't tell how easy it would be to do this without seeing the code for your Utils::XMLUtils functions.

Also, is there a reason why you wrote your own XML utility functions rather than using XML::Simple (or another standard XML module)?  It's possible you are doing that behind the scenes but it's hard to tell.

In order to append to a file, you could do something as simple as:

# after line 30
open OUT, '>', 'sample.csv' or die "could not write sample.csv: $!";
# after line 38
print OUT "$keydata,$data\n";
# after line 54
close OUT;

If this doesn't answer your question, please provide some more details.

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Powershell variable with -SearchBase cmdlet 2 87
jQuery Generalization 3 64
React or Angular? 6 54
Auto-indent certain lines in Notepad++ 10 35
I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
This article will show, step by step, how to integrate R code into a R Sweave document
The viewer will learn how to count occurrences of each item in an array.
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

864 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now