Appending some data to a flat text file

Posted on 2009-03-31
Medium Priority
Last Modified: 2013-12-25
I have a Programme below like this, which will take a file and create some keys and values and append to a XML File.

I have written functions for all this XML Functions.

while i try to do for big files, XML is creating very big problem to process huge data.  So i am looking to append the genrated key and values to a flat text file. How can i do this. Please help.

a Big Thanks in advance.

use Utils::XMLUtils qw(loadXMLFile
use Digest::SHA1 qw(sha1);
sub getHexData {
    my ($key)= unpack("H*",shift);
    return $key; 
print "Enter the file path :";
my $file = <STDIN>;
print "Enter chunk size :";
my $size = <STDIN>;
open(FILE, $file) || die "Unable to open the file";
my $doc = createXMLDocument();
my $SigData = createNewElement($doc, "SigData");
$len = read(FILE, $data, $size);
while ( $len != 0) {
  $datalen = $size == $len ? $size: $len;
  $digest = sha1($data);
  $keydata = getHexData($digest);
  $data = getHexData($data);
  $kvpair = createNewElement($doc, "KVPair");
  $key = createNewElement($doc, "Key");
  $keytext = createTextElement($doc, $keydata);
  assignChild($key, $keytext);
  $Value = createNewElement($doc, "Value");
  $Valuetext = createTextElement($doc, $data);
  assignChild($Value, $Valuetext);
  assignChild($kvpair, $key);
  assignChild($kvpair, $Value);
  assignChild($SigData, $kvpair);
  $len = read(FILE, $data, $size);

Open in new window

Question by:koti688
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 39

Expert Comment

ID: 24032944
What do you want your flat file to look like?  Should it be csv?
LVL 26

Accepted Solution

wilcoxon earned 2000 total points
ID: 25953710
Have you considered writing out the XML file as you go?  I can't tell how easy it would be to do this without seeing the code for your Utils::XMLUtils functions.

Also, is there a reason why you wrote your own XML utility functions rather than using XML::Simple (or another standard XML module)?  It's possible you are doing that behind the scenes but it's hard to tell.

In order to append to a file, you could do something as simple as:

# after line 30
open OUT, '>', 'sample.csv' or die "could not write sample.csv: $!";
# after line 38
print OUT "$keydata,$data\n";
# after line 54
close OUT;

If this doesn't answer your question, please provide some more details.

Featured Post

Get real performance insights from real users

Key features:
- Total Pages Views and Load times
- Top Pages Viewed and Load Times
- Real Time Site Page Build Performance
- Users’ Browser and Platform Performance
- Geographic User Breakdown
- And more

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In threads here at EE, each comment has a unique Identifier (ID). It is easy to get the full path for an ID via the right-click context menu. However, we often want to post a short link within a thread rather than the full link. This article shows a…
Originally, this post was published on Monitis Blog, you can check it here . In business circles, we sometimes hear that today is the “age of the customer.” And so it is. Thanks to the enormous advances over the past few years in consumer techno…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …
Suggested Courses

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question