November 19, 2008 01:11am pst

1. Adam314 809,084
2. ozo 479,021
3. FishMonger 141,057
4. Tintin 118,861
5. hielo 74,770
6. mjcoyne 69,328
7. nedfine 45,740
8. clockwatc... 30,424
9. ahoffmann 29,519
10. teraplane 29,380
11. kawas 27,030
12. ravenpl 24,800
13. omarfarid 21,868
14. SmartIntel 21,140
15. oleber 18,950
16. marchent 17,990
17. suhasbha... 17,320
18. ddrudik 14,250
19. nognew 13,250
20. MadShiva 12,300
21. nixfreak 12,200
22. caterham... 11,000
23. hernst42 10,080
24. agriesser 10,000
25. sentner 9,648

1. ozo 3,750,786
2. Adam314 1,731,503
3. Tintin 767,155
4. kandura 579,965
5. mjcoyne 521,466
6. jmcg 517,810
7. FishMonger 493,691
8. maneshr 385,976
9. teraplane 373,588
10. manav_m... 338,348
11. clockwatc... 246,712
12. ahoffmann 215,082
13. rj2 211,481
14. davorg 153,165
15. Tim_Utsc... 113,107
16. shlomoy 109,110
17. gripe 98,559
18. suhasbha... 92,720
19. Perl_Diver 82,106
20. hielo 80,720
21. sstouk 74,470
22. nedfine 73,307
23. jhurst 72,319
24. rugdog 70,951
25. ZiaTioN 66,995

09.22.2008 at 11:04AM PDT, ID: 23752294 | Points: 500

	[x] Attachment Details

Convert Multiple .XML .GZ Files to .CSV (Pipe Delimited) Daily [Script Adaptation Help]

Asked by thyros in Perl Programming Language, Extensible Markup Language (XML), Scripting Languages

Hi,

I would appreciate some help with converting filename.xml.gz files to filename.csv using some semi-automated process to convert several files.

I have a small perl code snippet attached which I am currently using to manually convert one xml file at a time to .csv (pipe delimited).

The script checks for a file called 'tags_to_match.txt" which includes a list of xml attribute headers that I want to extract, i.e.

<number>12</number><text>text</text><general>something else</general>

I would write out as follows in the tags_to_match.txt file.

number
text
general

(one per line for each piece of information to extract).

Then in command prompt I would run the following;

shell> perl xml2csv.pl < myinput.xml > myoutput.csv

This would take the existing xml file and convert it to pipe delimited csv and output according to the name above. This works fine but I need to extend the functionality further. Right now, the output would look something like this;

12|text|something else

However, I need to include the field headers at the top of the file so that the output looks like this;

number|text|general
12|text|something else
48|more text|other general stuff
..

Lastly, I need to process a list of files (about 12 of them on a daily basis) so it becomes quite tedious to write out each command individually, so hopefully this perl snippet can be adapted to look in another file which contains a list of .xml.gz filenames to convert (one per line as I have with tags_to_match.txt). From there I could setup a crontab to run this one perl script which would then process those multiple files. The output csv files can be the same name as the input xml files with the difference of the file extension.
Start Free Trial

         
           
             1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:

           
           
             #!/usr/bin/perl -w
use strict;
 
# little script for thyros, Expert-Exchange.com Q_22833778.html
# see http://www.experts-exchange.com/Q_22833778.html
 
open TAGS, "tags_to_match.txt" or die("ERROR: tags file 'tags_to_match.txt' not found");
my $all_tags = do { local $/; <TAGS> };
my @tags = split(/\n/,$all_tags);
 
my $merge_cat_path_count = 0;
$/ = "</item_data>";
while (<>) {
s/\|//g;
 
 
      die("ERROR: pipeline '|' character found in data!") if /\|/;
 
      foreach my $tag (@tags) {
            my $tmpTag = $tag;  # work on a copy
            my $count = 1;
            $count = $1 if $tmpTag =~ s/(\d+)$//;
            print "$1" if m!(?:<$tmpTag>([^<]*)</$tmpTag>.*?){$count}!ms;            # do not set /g modifier!
            print "|";      
      }
      print "\n";
}

           
         

Keywords: Convert Multiple .XML .GZ Files to .…

	Title
1	XML format
2	perl script to transform xml into csv or …
3	loading/locking 200+ xml files: go…
4	Batch Processing to save xml's in UT…
5	semi colon needed
6	Using Perl and XML::Simple to parse…

Loading Advertisement...

20081112-EE-VQP-44 / EE_QW_2_20070628