Appending some data to a flat text file

Posted on 2009-03-31
Last Modified: 2013-12-25
I have a Programme below like this, which will take a file and create some keys and values and append to a XML File.

I have written functions for all this XML Functions.

while i try to do for big files, XML is creating very big problem to process huge data.  So i am looking to append the genrated key and values to a flat text file. How can i do this. Please help.

a Big Thanks in advance.

use Utils::XMLUtils qw(loadXMLFile
use Digest::SHA1 qw(sha1);
sub getHexData {
    my ($key)= unpack("H*",shift);
    return $key; 
print "Enter the file path :";
my $file = <STDIN>;
print "Enter chunk size :";
my $size = <STDIN>;
open(FILE, $file) || die "Unable to open the file";
my $doc = createXMLDocument();
my $SigData = createNewElement($doc, "SigData");
$len = read(FILE, $data, $size);
while ( $len != 0) {
  $datalen = $size == $len ? $size: $len;
  $digest = sha1($data);
  $keydata = getHexData($digest);
  $data = getHexData($data);
  $kvpair = createNewElement($doc, "KVPair");
  $key = createNewElement($doc, "Key");
  $keytext = createTextElement($doc, $keydata);
  assignChild($key, $keytext);
  $Value = createNewElement($doc, "Value");
  $Valuetext = createTextElement($doc, $data);
  assignChild($Value, $Valuetext);
  assignChild($kvpair, $key);
  assignChild($kvpair, $Value);
  assignChild($SigData, $kvpair);
  $len = read(FILE, $data, $size);

Open in new window

Question by:koti688
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 39

Expert Comment

ID: 24032944
What do you want your flat file to look like?  Should it be csv?
LVL 26

Accepted Solution

wilcoxon earned 500 total points
ID: 25953710
Have you considered writing out the XML file as you go?  I can't tell how easy it would be to do this without seeing the code for your Utils::XMLUtils functions.

Also, is there a reason why you wrote your own XML utility functions rather than using XML::Simple (or another standard XML module)?  It's possible you are doing that behind the scenes but it's hard to tell.

In order to append to a file, you could do something as simple as:

# after line 30
open OUT, '>', 'sample.csv' or die "could not write sample.csv: $!";
# after line 38
print OUT "$keydata,$data\n";
# after line 54
close OUT;

If this doesn't answer your question, please provide some more details.

Featured Post

Monthly Recap

May was a big month for new releases from Linux Academy! Take a look at what our team built recently in our blog. You can access the newest releases from our blog.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this tutorial I will show you how to make a simple HTML bar chart with the usage of WhizBase, If you want more information about WhizBase please read my previous articles at (http://www.experts-ex…
This article will show, step by step, how to integrate R code into a R Sweave document
Learn the basics of modules and packages in Python. Every Python file is a module, ending in the suffix: .py: Modules are a collection of functions and variables.: Packages are a collection of modules.: Module functions and variables are accessed us…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question