Link to home
Create AccountLog in
Avatar of George R. Kasica
George R. KasicaFlag for United States of America

asked on

Configure remote logging same as local logging

I have a rsyslogd Ubuntu question. I'm trying to implement a syslog server to collect all my logs in one location. Using the following script on local logging splits stuff out nicely. Adding the last couple of lines to send it off to the remote systems gets me one log per machine, and I think I'm missing a bunch of traffic; if not, then it's all combined in that one file. Not very useful. Can anyone suggest an edit or what to remote log that works so well locally??


Here is what I get remotely - 1 file per system:

 cd /var/log/remote/

/var/log/remote# ll


drwxr-xr-x  2 syslog syslog      4096 Jul  8 16:58 ./

drwxrwxr-x 15 root   syslog      4096 Jul  8 17:25 ../

-rw-r-----  1 syslog adm       964552 Jul  8 23:39 eagle

-rw-r-----  1 syslog adm      3056244 Jul  8 23:39 netwrx1ai

-rw-r-----  1 syslog adm    180647321 Jul  8 23:40 ntp1

-rw-r-----  1 syslog adm    188488996 Jul  8 23:40 ntp2

-rw-r-----  1 syslog adm    186950620 Jul  8 23:40 ntp3

-rw-r-----  1 syslog adm       592244 Jul  8 23:39 saturn2

-rw-r-----  1 syslog adm      2086513 Jul  8 23:39 ubuntu-desktop

-rw-r-----  1 syslog adm       213766 Jul  8 17:08 ubuntu-desktop-notebook



# /etc/rsyslog.conf configuration file for rsyslog

#

# For more information install rsyslog-doc and see

# /usr/share/doc/rsyslog-doc/html/configuration/index.html



#################

#### MODULES ####

#################


module(load="imuxsock") # provides support for local system logging

module(load="imklog")   # provides kernel logging support

#module(load="immark")  # provides --MARK-- message capability


# provides UDP syslog reception

#module(load="imudp")

#input(type="imudp" port="514")


# provides TCP syslog reception

#module(load="imtcp")

#input(type="imtcp" port="514")



###########################

#### GLOBAL DIRECTIVES ####

###########################


#

# Use traditional timestamp format.

# To enable high precision timestamps, comment out the following line.

#

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat


#

# Set the default permissions for all log files.

#

$FileOwner root

$FileGroup adm

$FileCreateMode 0640

$DirCreateMode 0755

$Umask 0022


#

# Where to place spool and state files

#

$WorkDirectory /var/spool/rsyslog


#

# Include all config files in /etc/rsyslog.d/

#

$IncludeConfig /etc/rsyslog.d/*.conf



###############

#### RULES ####

###############


#

# First some standard log files.  Log by facility.

#

auth,authpriv.*                 /var/log/auth.log

*.*;auth,authpriv.none          -/var/log/syslog

#cron.*                         /var/log/cron.log

daemon.*                        -/var/log/daemon.log

kern.*                          -/var/log/kern.log

lpr.*                           -/var/log/lpr.log

mail.*                          -/var/log/mail.log

user.*                          -/var/log/user.log


#

# Logging for the mail system.  Split it up so that

# it is easy to write scripts to parse these files.

#

mail.info                       -/var/log/mail.info

mail.warn                       -/var/log/mail.warn

mail.err                        /var/log/mail.err


#

# Some "catch-all" log files.

#

*.=debug;\

        auth,authpriv.none;\

        news.none;mail.none     -/var/log/debug

*.=info;*.=notice;*.=warn;\

        auth,authpriv.none;\

        cron,daemon.none;\

        mail,news.none          -/var/log/messages


#

# Emergencies are sent to everybody logged in.

#

*.emerg                         :omusrmsg:*

#

#Configure remote logging

#

*.* @24.196.111.53:514

*.* @@24.196.111.53:514



Avatar of madunix
madunix

You want to split your remote logs similarly to your local logs. For this, you need to modify the rsyslog.conf file on the client-side machines sending logs to the syslog server. You will have to specify different facilities for different types of logs on the clients, and then on the server side, you can split these logs based on their facilities.

 

Here's an example of how you might modify your rsyslog.conf on the client machines:

 

auth,authpriv.*   @24.196.111.53:514
daemon.*         @24.196.111.53:514
kern.*           @24.196.111.53:514
...

Open in new window

Then, on the syslog server side, you can handle these different facilities like this:

 

auth,authpriv.*                  /var/log/remote/auth.log
daemon.*                         /var/log/remote/daemon.log
kern.*                           /var/log/remote/kern.log
...

Open in new window


This way, logs from different facilities will be separated into different files.

 

As for the issue of missing some logs, you should look into your network stability.

 

Once you have made the necessary changes to your rsyslog.conf file, you need to restart the rsyslog service for the changes to take effect. You can do this by running the following command:

 

sudo service rsyslog restart

Open in new window

 

Avatar of George R. Kasica

ASKER

Thanks alot! I'll make changes probably Monday. Dealing with a bad migraine today and screen time doesn't help it. One other question. Will they be separated by system like I have now as well like just an example:

/var/log/remote/eagle.auth.log vs. auth.log with all systems combined??

Just bumping this again.

Is there a way to split them by system name as well either everything in one file per system or ideally one ffile per system with one item per file example:

eagle.syslog

saturn2.syslog

etc.

The rsyslogd configuration has templates, which lets you format the log message, the file name, and other outputs. 

 

Templates allow you to specify the format of the logged message. They are also used for dynamic file name generation. They have to be defined before they are used in rules. For more info about templates, see the TEMPLATES section of this manual page.



Check the following links:


https://www.redhat.com/sysadmin/log-aggregation-rsyslog

https://man7.org/linux/man-pages/man5/rsyslog.conf.5.html

https://rsyslog-5-8-6-doc.neocities.org/rsyslog_conf_templates

https://www.rsyslog.com/doc/v8-stable/configuration/templates.html

https://www.thegeekdiary.com/configuring-remote-logging-using-rsyslog-in-centos-rhel/

 

To split logs by hostname, you can use a dynamic file name that includes the hostname.

 

First, you will need to define a new template in your rsyslog configuration file (usually /etc/rsyslog.conf):

 

$template DynamicFile,"/var/log/remote/%HOSTNAME%.log"

Open in new window

 

In the above line, "%HOSTNAME%" will be replaced by the hostname of the machine sending the log message.

 

Then you need to use this template for the actions that write to the log files:

 

*.* -?DynamicFile

Open in new window

 

In the above line, rsyslog is to write all messages to the file specified by the DynamicFile template.

 

Here's how you can modify your configuration to use this:

 

# /etc/rsyslog.conf configuration file for rsyslog
...
 
$template DynamicFile,"/var/log/remote/%HOSTNAME%.log"
 
...
 
*.* -?DynamicFile
 
 

Open in new window


Please ensure to restart the rsyslog service after modifying the configuration file:

 

sudo systemctl restart rsyslog

Open in new window


Doing this means you should have each machine's log data written to a separate file, with the filename being the machine’s hostname.

would it still be possible to split it with one file for each machine and it's various logs. like machine.function example eagle.postfix eagle.auth etc.  ??
ASKER CERTIFIED SOLUTION
Avatar of madunix
madunix

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Thank you very much madunix. Exactly what I need.
Thank you again. Exactly what I'm needing. will make the program edits tomorrow. will let you know how it works.

Working well - Thank you. Sorry, it took so long to reply. I fell, walking my dog and avoiding him on the way down. I broke my right ankle. Now, with hardware in my ankle, I am finally getting back to normal. I just will make the TSA guys at airports a bit crazy. LOL