Solved

log aggregation server linux

Posted on 2014-02-25
5
894 Views
Last Modified: 2014-03-17
hey guys i want to have a central log server, where all of the various logs from my devices go too. My servers, switches, firewalls, send all logs to this point and just collect. What is the best linux distro, or even unix, to faciliate manage this?
0
Comment
Question by:mrbayIt
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 
LVL 20

Expert Comment

by:netcmh
ID: 39886972
While the options of the software itself are plentiful - Zenoss, Logzilla, Splunk, AlienVault, Octopussy etc, the OS or distro choice would boil down to your level of expertise in Linux.

More apps can be found at http://www.infosecisland.com/blogview/12772-Open-Source-Log-Management-Tools-List.html

My preference would be Debian GNU for it's packaging and ease of use for me.

There's a live CD solution that you might want to look into just to get an idea: https://www.wzdftpd.net/redmine/projects/siem-live
0
 

Author Comment

by:mrbayIt
ID: 39887082
i know there were software options, i just recall hearing some Linux distros are optimized for log processing speed wise either via the file system or its structure somehow, but i couldn't remember which one
0
 
LVL 20

Expert Comment

by:netcmh
ID: 39887149
Here's a great writeup on how to select your distros: http://www.linuxinsider.com/story/79717.html

And, here's the distros list:http://distrowatch.com/

If you're looking for enterprise grade, I would recommend Fedora. If it's a small to medium  enough infrastructure, Debian.
0
 
LVL 25

Expert Comment

by:madunix
ID: 39887969
You can direct logs from all systems in your network to one centralized server in linux RHEL via syslogd daemon. Configuration changes are generally required on both client and server to achieve this.
https://access.redhat.com/site/solutions/2725
http://www.howtoforge.com/syslog-better-logging-tutorial
0
 
LVL 27

Accepted Solution

by:
skullnobrains earned 500 total points
ID: 39897040
the os does not matter much. (i like freebsd so i'm gonna advise freebsd, each and every one of us will advise our favorite os/dist)

the tool will be syslog or one of it's derivatives. most linuxes ship with ksyslogd or rsyslogd, freebsd has it's own version of the original syslogd

i know there were software options, i just recall hearing some Linux distros are optimized for log processing speed wise either via the file system or its structure somehow, but i couldn't remember which one

a log filesystem such as hammer might seem to be a good choice performance-wise

i personally would not go in that direction : what kind of volumes do you expect ? would that really exceed what a regular machine can do ?

if you're looking towards high performance writes, a good idea might be using zfs (on bsd or solaris) in simili-raid10 with as many disks as needed and possibly a tiny SSD for write cache (alias ZIL alias zpool log device)

----

also note that configuring remote logs in syslog does not guarantee the logs will be actually logged. your devices may buffer the logs for a little while but when the machine is down, the logs will be lost. likewise if the disks are not fast enough, syslog will throw the data away.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Install XRDP on Ubuntu Server 16.10 x64 3 120
CentOS Backup Options 3 68
NFS v4 7 76
cmake and message 1 27
Introduction We as admins face situation where we need to redirect websites to another. This may be required as a part of an upgrade keeping the old URL but website should be served from new URL. This document would brief you on different ways ca…
The purpose of this article is to demonstrate how we can upgrade Python from version 2.7.6 to Python 2.7.10 on the Linux Mint operating system. I am using an Oracle Virtual Box where I have installed Linux Mint operating system version 17.2. Once yo…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.

710 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question