Appending some data to a flat text file

I have a Programme below like this, which will take a file and create some keys and values and append to a XML File.

I have written functions for all this XML Functions.
                       LoadXMLFile
                       getNodesFromPath
                       getNodeFromPath
                       getChildren
                       getNodeName
                       getValueFromPath
                       getValuesFromPath
                       checkIfExists
                       createXMLDocument
                       createNewElement
                       assignChild
                       createTextElement
                       wrtiteTofile

while i try to do for big files, XML is creating very big problem to process huge data.  So i am looking to append the genrated key and values to a flat text file. How can i do this. Please help.


a Big Thanks in advance.

use Utils::XMLUtils qw(loadXMLFile
                       getNodesFromPath
                       getNodeFromPath
                       getChildren
                       getNodeName
                       getValueFromPath
                       getValuesFromPath
                       checkIfExists
                       createXMLDocument
                       createNewElement
                       assignChild
                       createTextElement
                       wrtiteTofile
                      );
use Digest::SHA1 qw(sha1);
 
 
sub getHexData {
    my ($key)= unpack("H*",shift);
    return $key; 
}
 
print "Enter the file path :";
my $file = <STDIN>;
chomp($file);
print "Enter chunk size :";
my $size = <STDIN>;
chomp($size);
 
open(FILE, $file) || die "Unable to open the file";
my $doc = createXMLDocument();
my $SigData = createNewElement($doc, "SigData");
$len = read(FILE, $data, $size);
while ( $len != 0) {
  $datalen = $size == $len ? $size: $len;
  $digest = sha1($data);
  $keydata = getHexData($digest);
  $data = getHexData($data);
  $kvpair = createNewElement($doc, "KVPair");
  $key = createNewElement($doc, "Key");
  $keytext = createTextElement($doc, $keydata);
  assignChild($key, $keytext);
  $Value = createNewElement($doc, "Value");
  $Valuetext = createTextElement($doc, $data);
  assignChild($Value, $Valuetext);
  assignChild($kvpair, $key);
  assignChild($kvpair, $Value);
  assignChild($SigData, $kvpair);
 
  $len = read(FILE, $data, $size);
 
}
wrtiteTofile($SigData,"sample.xml",0);
close(FILE);

Open in new window

koti688Asked:
Who is Participating?
 
wilcoxonConnect With a Mentor Commented:
Have you considered writing out the XML file as you go?  I can't tell how easy it would be to do this without seeing the code for your Utils::XMLUtils functions.

Also, is there a reason why you wrote your own XML utility functions rather than using XML::Simple (or another standard XML module)?  It's possible you are doing that behind the scenes but it's hard to tell.

In order to append to a file, you could do something as simple as:

# after line 30
open OUT, '>', 'sample.csv' or die "could not write sample.csv: $!";
# after line 38
print OUT "$keydata,$data\n";
# after line 54
close OUT;

If this doesn't answer your question, please provide some more details.
0
 
Adam314Commented:
What do you want your flat file to look like?  Should it be csv?
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.