Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.
Solved
log aggregation server linux
Posted on 2014-02-25
hey guys i want to have a central log server, where all of the various logs from my devices go too. My servers, switches, firewalls, send all logs to this point and just collect. What is the best linux distro, or even unix, to faciliate manage this?
[X]
Welcome to Experts Exchange
Add your voice to the tech community where 5M+ people just like you are talking about what matters.
- Help others & share knowledge
- Earn cash & points
- Learn & ask questions
2
- +1
5 Comments
Active today
While the options of the software itself are plentiful - Zenoss, Logzilla, Splunk, AlienVault, Octopussy etc, the OS or distro choice would boil down to your level of expertise in Linux.
More apps can be found at http://www.infosecisland.com/blogview/12772-Open-Source-Log-Management-Tools-List.html
My preference would be Debian GNU for it's packaging and ease of use for me.
There's a live CD solution that you might want to look into just to get an idea: https://www.wzdftpd.net/redmine/projects/siem-live
More apps can be found at http://www.infosecisland.com/blogview/12772-Open-Source-Log-Management-Tools-List.html
My preference would be Debian GNU for it's packaging and ease of use for me.
There's a live CD solution that you might want to look into just to get an idea: https://www.wzdftpd.net/redmine/projects/siem-live
i know there were software options, i just recall hearing some Linux distros are optimized for log processing speed wise either via the file system or its structure somehow, but i couldn't remember which one
Active today
Here's a great writeup on how to select your distros: http://www.linuxinsider.com/story/79717.html
And, here's the distros list:http://distrowatch.com/
If you're looking for enterprise grade, I would recommend Fedora. If it's a small to medium enough infrastructure, Debian.
And, here's the distros list:http://distrowatch.com/
If you're looking for enterprise grade, I would recommend Fedora. If it's a small to medium enough infrastructure, Debian.
Active today
You can direct logs from all systems in your network to one centralized server in linux RHEL via syslogd daemon. Configuration changes are generally required on both client and server to achieve this.
https://access.redhat.com/site/solutions/2725
http://www.howtoforge.com/syslog-better-logging-tutorial
https://access.redhat.com/site/solutions/2725
http://www.howtoforge.com/syslog-better-logging-tutorial
Active 3 days ago
the os does not matter much. (i like freebsd so i'm gonna advise freebsd, each and every one of us will advise our favorite os/dist)
the tool will be syslog or one of it's derivatives. most linuxes ship with ksyslogd or rsyslogd, freebsd has it's own version of the original syslogd
a log filesystem such as hammer might seem to be a good choice performance-wise
i personally would not go in that direction : what kind of volumes do you expect ? would that really exceed what a regular machine can do ?
if you're looking towards high performance writes, a good idea might be using zfs (on bsd or solaris) in simili-raid10 with as many disks as needed and possibly a tiny SSD for write cache (alias ZIL alias zpool log device)
----
also note that configuring remote logs in syslog does not guarantee the logs will be actually logged. your devices may buffer the logs for a little while but when the machine is down, the logs will be lost. likewise if the disks are not fast enough, syslog will throw the data away.
the tool will be syslog or one of it's derivatives. most linuxes ship with ksyslogd or rsyslogd, freebsd has it's own version of the original syslogd
i know there were software options, i just recall hearing some Linux distros are optimized for log processing speed wise either via the file system or its structure somehow, but i couldn't remember which one
a log filesystem such as hammer might seem to be a good choice performance-wise
i personally would not go in that direction : what kind of volumes do you expect ? would that really exceed what a regular machine can do ?
if you're looking towards high performance writes, a good idea might be using zfs (on bsd or solaris) in simili-raid10 with as many disks as needed and possibly a tiny SSD for write cache (alias ZIL alias zpool log device)
----
also note that configuring remote logs in syslog does not guarantee the logs will be actually logged. your devices may buffer the logs for a little while but when the machine is down, the logs will be lost. likewise if the disks are not fast enough, syslog will throw the data away.
Featured Post
Question has a verified solution.
If you are experiencing a similar issue, please ask a related question
This article will explain how to establish a SSH connection to Ubuntu through the firewall and using a different port other then 22. I have set up a Ubuntu virtual machine in Virtualbox and I am running a Windows 7 workstation.
From the Ubuntu vi…
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn several ways to interact with files and get file information from the bash shell.
ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to navigate the file tree with the shell.
Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Suggested Courses
Course of the Month8 days, 13 hours left to enroll
721 members asked questions and received personalized solutions in the past 7 days.
Join the community of 500,000 technology professionals and ask your questions.