RXGeorge
asked on
Best solution for application-specific centralized logging on a windows network.
My company sells a Windows desktop based software solution and some of our enterprise customers have been (vaguely) asking for a "centralized event logging" feature. They want to have a top down view of who is using the software, what they are using it to do, when they use it and how often. I have a good idea of how to implement this in my software, but i have no idea of the best logging destination. I have heard of server based log engines like syslog which enjoy substantial support. I need advice on how to proceed.
Although the target audience for this feature set is IT in medium to large customers, the reporting / analysis of the log data would not be done by typical data center engineers. Instead, the reporting / analysis tools will be used by IT people closer to the lines of business for the purpose of measuring business effectiveness, training and finding error patterns.
As i see it, i need two bits of advice.
1. What server based logging tool to use?
2. What reporting / analysis tool to use?
Although the target audience for this feature set is IT in medium to large customers, the reporting / analysis of the log data would not be done by typical data center engineers. Instead, the reporting / analysis tools will be used by IT people closer to the lines of business for the purpose of measuring business effectiveness, training and finding error patterns.
As i see it, i need two bits of advice.
1. What server based logging tool to use?
2. What reporting / analysis tool to use?
indeed syslog is generally accepted and recognised as RFC3164 , there are others such as Common Event Format (CEF).
These formatted messages are send over by the designated communications module (can be s/w like yours or network appliance engine, log agent etc) that handles the transmission of the messages to your waiting log collection server (like kiwi syslog, SIEMS - security information event mgmt system like SolarWinds, ArcSight, LogRythm, QRadar etc) using either the UDP, TCP or SSL/TLS protocol. There would be a in log size limit but we can take in range of 1024 to 2048 (max). Windows event log (those you see in event viewer) need to be converted as shared by gheist. Good to check with the team managing your log collection server to see what format they are expecting log messages to arrive in.
For logging tool is recommended via Syslog and if necessary employed agent in the device source. Can consider either Snare or Syslog_ng is well tested and wide platform supported.
https://www.intersectalliance.com/our-product/
https://www.balabit.com/network-security/syslog-ng/opensource-logging-system/features/comparison
For reporting SIEMS can be one with correlated rules and dashboard and adhc report generated/scheduled. They ingest those log sent to them. OSSIM (from AlienVault) is one candidate or solarwind or splunk (free text search engine)
https://www.alienvault.com/products/ossim
http://www.solarwinds.com/log-event-manager.aspx
http://www.splunk.com/en_us/products/splunk-enterprise.html
There is gartner mq on the SIEMS as well for cost effective competitive edge - normally the provider will host such doc like this below.
http://securityintelligence.com/gartner-2014-magic-quadrant-siem-security/
Cost wise can be considerable though as they tend to count by the event per second (EPS)...
These formatted messages are send over by the designated communications module (can be s/w like yours or network appliance engine, log agent etc) that handles the transmission of the messages to your waiting log collection server (like kiwi syslog, SIEMS - security information event mgmt system like SolarWinds, ArcSight, LogRythm, QRadar etc) using either the UDP, TCP or SSL/TLS protocol. There would be a in log size limit but we can take in range of 1024 to 2048 (max). Windows event log (those you see in event viewer) need to be converted as shared by gheist. Good to check with the team managing your log collection server to see what format they are expecting log messages to arrive in.
For logging tool is recommended via Syslog and if necessary employed agent in the device source. Can consider either Snare or Syslog_ng is well tested and wide platform supported.
https://www.intersectalliance.com/our-product/
https://www.balabit.com/network-security/syslog-ng/opensource-logging-system/features/comparison
For reporting SIEMS can be one with correlated rules and dashboard and adhc report generated/scheduled. They ingest those log sent to them. OSSIM (from AlienVault) is one candidate or solarwind or splunk (free text search engine)
https://www.alienvault.com/products/ossim
http://www.solarwinds.com/log-event-manager.aspx
http://www.splunk.com/en_us/products/splunk-enterprise.html
There is gartner mq on the SIEMS as well for cost effective competitive edge - normally the provider will host such doc like this below.
http://securityintelligence.com/gartner-2014-magic-quadrant-siem-security/
Cost wise can be considerable though as they tend to count by the event per second (EPS)...
ASKER
Hi btan;
Thanks for your answer.
1. I don't want to have any 3rd party windows desktop agents to install. I want to just make the logging calls directly from my application code. Since my target development environment is .NET, does that change anything in your analysis?
2. Graylog2? Any idea if this is something that i should be considering?
Thanks for your answer.
1. I don't want to have any 3rd party windows desktop agents to install. I want to just make the logging calls directly from my application code. Since my target development environment is .NET, does that change anything in your analysis?
2. Graylog2? Any idea if this is something that i should be considering?
ASKER
Hi btan (again);
Let me clarify my last question. In my researching the links you sent, it seems like the dominant method of centralized logging is to "collect" event data using s/w agents installed on each of the PCs. I am guessing this would mean that if i log directly to the windows event log - then those logs can be gathered up by whatever logging infrastructure the client uses. Am i correct in this?
Let me clarify my last question. In my researching the links you sent, it seems like the dominant method of centralized logging is to "collect" event data using s/w agents installed on each of the PCs. I am guessing this would mean that if i log directly to the windows event log - then those logs can be gathered up by whatever logging infrastructure the client uses. Am i correct in this?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Syslog works with other systems than windows too. Windows syslog (both event sender and "central" part is syslog-ng or rsyslog agent (preferably you make some mainstream linux distribution like LTS ubuntu or CentOS a central log server)
Now you probably want to add some fancy GUI like kibana or greylog2, or even something called SIEM to central collector.