• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 697
  • Last Modified:

File concat program

Hi There

I'm tryting to do the following:

I have all my webstats in one folder of mt Linux box ( /var/log/httpd)

They are listed as follow:

dom1-access.log
dom1-access.log.1
dom1-access.log.2
dom2-access.log
dom2-access.log.1
dom2-access.log.2
dom3-access.log
dom3-access.log.1
etc.etc

( note, dom1 and dom2 are only examples, there are no usable patterns in the part infront of the -access.log)

I would now like to have a script that can run through the folder to concat all the files for one particular dom into one file
ie, I want to combine dom1-access.log, dom1-access.log.1, dom1-access.log.2 , dom-access.log.3 etc into just dom1-access.log, ie, add the contents of the rolled logs to the "current" log file. The trick would also be to construct the new file as follow:

Since dom1-access.log.x (where x is the largest of whatever is present ( the log roller will go up to .4 I think)) is the "oldest" and it's data needs to be added to the "new" dom1-access.log file, then the dom1-access.log.3, then ...2, then ....1, and finaly the original dom1-access.log contents..

Can someone pls give me some pointers/examples since I am not very familiar with Perl, but Perl ( or Bash) is the only scripting language available to me on the server.


0
psimation
Asked:
psimation
  • 3
  • 2
1 Solution
 
afongCommented:
#!/usr/bin/perl

opendir (INDIR,".");
@allfiles=readdir(INDIR);
closedir INDIR;

#combine files
foreach (sort {$b cmp $a} (@allfiles))
{
  if ( /(.*)-(access.log.*)/ )
  {
        if ($sites{$1}++)
        {       open (OUT, ">>$1.log")}
        else {  open (OUT, ">$1.log")}
        open (IN, "$_");
        while (<IN>) {  print OUT }
        close IN;
        close OUT;
   }
}

#now overwrite original with combined
foreach (keys %sites)
{
        rename "$_.access.log $_.access.log.0\n";
        rename "$_.dat $_.access.log\n";
}

NOTE: I backed up the original log. If you want to erase the files when you are done use "unlink". I also assumed you are running the perl program in your log directory
0
 
afongCommented:
Sorry: the last couple of lines should be...
{
  rename "$_-access.log", "$_-access.log.0";
  rename "$_.log", "$_-access.log";
}
0
 
TintinCommented:
This is a better/easier task for a shell script

#!/bin/bash
cd /var/log/httpd

for log in *.log
do
   domain=${log%%-access.log}
   for domlog in `ls -r $domain-*[0-9]`
   do
       cat $domlog >>$log
    done
 done
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
TintinCommented:
psimation, just curious why you'd went for a more complex Perl solution over the easier shell solution.
0
 
psimationAuthor Commented:
It was the first solution posted to work  ;) By the time you posted, my problem was already solved, so it wouldn't be fair for me to then dismiss afong's solution. I do appreciate your input though!
0
 
TintinCommented:
Which solution did you end up using though?  It's fine to give points to the first working solution, but remember that the first is not always the best.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 3
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now