Solved

Appending some data to a flat text file

Posted on 2009-03-31
3
388 Views
Last Modified: 2013-12-25
I have a Programme below like this, which will take a file and create some keys and values and append to a XML File.

I have written functions for all this XML Functions.
                       LoadXMLFile
                       getNodesFromPath
                       getNodeFromPath
                       getChildren
                       getNodeName
                       getValueFromPath
                       getValuesFromPath
                       checkIfExists
                       createXMLDocument
                       createNewElement
                       assignChild
                       createTextElement
                       wrtiteTofile

while i try to do for big files, XML is creating very big problem to process huge data.  So i am looking to append the genrated key and values to a flat text file. How can i do this. Please help.


a Big Thanks in advance.

use Utils::XMLUtils qw(loadXMLFile
                       getNodesFromPath
                       getNodeFromPath
                       getChildren
                       getNodeName
                       getValueFromPath
                       getValuesFromPath
                       checkIfExists
                       createXMLDocument
                       createNewElement
                       assignChild
                       createTextElement
                       wrtiteTofile
                      );
use Digest::SHA1 qw(sha1);
 
 
sub getHexData {
    my ($key)= unpack("H*",shift);
    return $key; 
}
 
print "Enter the file path :";
my $file = <STDIN>;
chomp($file);
print "Enter chunk size :";
my $size = <STDIN>;
chomp($size);
 
open(FILE, $file) || die "Unable to open the file";
my $doc = createXMLDocument();
my $SigData = createNewElement($doc, "SigData");
$len = read(FILE, $data, $size);
while ( $len != 0) {
  $datalen = $size == $len ? $size: $len;
  $digest = sha1($data);
  $keydata = getHexData($digest);
  $data = getHexData($data);
  $kvpair = createNewElement($doc, "KVPair");
  $key = createNewElement($doc, "Key");
  $keytext = createTextElement($doc, $keydata);
  assignChild($key, $keytext);
  $Value = createNewElement($doc, "Value");
  $Valuetext = createTextElement($doc, $data);
  assignChild($Value, $Valuetext);
  assignChild($kvpair, $key);
  assignChild($kvpair, $Value);
  assignChild($SigData, $kvpair);
 
  $len = read(FILE, $data, $size);
 
}
wrtiteTofile($SigData,"sample.xml",0);
close(FILE);

Open in new window

0
Comment
Question by:koti688
3 Comments
 
LVL 39

Expert Comment

by:Adam314
ID: 24032944
What do you want your flat file to look like?  Should it be csv?
0
 
LVL 26

Accepted Solution

by:
wilcoxon earned 500 total points
ID: 25953710
Have you considered writing out the XML file as you go?  I can't tell how easy it would be to do this without seeing the code for your Utils::XMLUtils functions.

Also, is there a reason why you wrote your own XML utility functions rather than using XML::Simple (or another standard XML module)?  It's possible you are doing that behind the scenes but it's hard to tell.

In order to append to a file, you could do something as simple as:

# after line 30
open OUT, '>', 'sample.csv' or die "could not write sample.csv: $!";
# after line 38
print OUT "$keydata,$data\n";
# after line 54
close OUT;

If this doesn't answer your question, please provide some more details.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
How to get all the API from website? 11 104
AutoHotkey ActiveX Component Error 8 94
copy-item script help 15 81
Using VBScript. How to obtain the recomended paging file size? 8 71
I hope you'll find this tutorial useful and interesting. So let's try to extend Tcl with a new package.  For anyone more deeply interested please check out the book "Practical Programming in Tcl and Tk". It's really one of the best written books abo…
This article will show, step by step, how to integrate R code into a R Sweave document
The viewer will learn how to dynamically set the form action using jQuery.
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question