Best solution for application-specific centralized logging on a windows network.

My company sells a Windows desktop based software solution and some of our enterprise customers have been (vaguely) asking for a "centralized event logging" feature.   They want to have a top down view of who is using the software, what they are using it to do,  when they use it and how often. I have a good idea of how to implement this in my software, but i have no idea of the best logging destination.   I have heard of server based log engines like syslog which enjoy substantial support.  I need advice on how to proceed.

Although the target audience for this feature set is IT in medium to large customers, the reporting / analysis of the log data would not be done by typical data center engineers.  Instead, the reporting / analysis tools will be used by IT people closer to the lines of business for the purpose of measuring business effectiveness, training and finding error patterns.  

As i see it, i need two bits of advice.

1.   What server based logging tool to use?
2.   What reporting / analysis tool to use?
RXGeorgeAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

gheistCommented:
Windows own infrastructure: https://technet.microsoft.com/en-us/library/cc748890.aspx

Syslog works with other systems than windows too. Windows syslog (both event sender and "central" part is syslog-ng or rsyslog agent (preferably you make some mainstream linux distribution like LTS ubuntu or CentOS a central log server)

Now you probably want to add some fancy GUI like kibana or greylog2, or even something called SIEM to central collector.
0
btanExec ConsultantCommented:
indeed syslog is generally accepted and recognised as RFC3164 , there are others such as Common Event Format (CEF).

These formatted  messages are send over by the designated communications module (can be s/w like yours or network appliance engine, log agent etc) that handles the transmission of the messages to your waiting log collection server (like kiwi syslog, SIEMS - security information event mgmt system like SolarWinds, ArcSight, LogRythm, QRadar etc) using either the UDP, TCP or SSL/TLS protocol.  There would be a in log size limit but we can take in range of 1024 to 2048 (max). Windows event log (those you see in event viewer) need to be converted as shared by gheist.  Good to check with the team managing your log collection server to see what format they are expecting log messages to arrive in.

For logging tool is recommended via Syslog and if necessary employed agent in the device source. Can consider either Snare or Syslog_ng is well tested and wide platform supported.  
https://www.intersectalliance.com/our-product/
https://www.balabit.com/network-security/syslog-ng/opensource-logging-system/features/comparison

For reporting SIEMS can be one with correlated rules and dashboard and adhc report generated/scheduled. They ingest those log sent to them. OSSIM (from AlienVault) is one candidate or solarwind or splunk (free text search engine)
https://www.alienvault.com/products/ossim
http://www.solarwinds.com/log-event-manager.aspx
http://www.splunk.com/en_us/products/splunk-enterprise.html

There is gartner mq on the  SIEMS as well for cost effective competitive edge - normally the provider will host such doc like this below.
http://securityintelligence.com/gartner-2014-magic-quadrant-siem-security/

Cost wise can be considerable though as they tend to count by the event per second (EPS)...
0
RXGeorgeAuthor Commented:
Hi btan;

Thanks for your answer.

1. I don't want to have any 3rd party windows desktop agents to install.  I want to just make the logging calls directly from my application code.   Since my target development environment is .NET, does that change anything in your analysis?

2. Graylog2?    Any idea if this is something that i should be considering?
0
RXGeorgeAuthor Commented:
Hi btan (again);

Let me clarify my last question.   In my researching the links you sent, it seems like the dominant method of centralized logging is to "collect" event data using s/w agents installed on each of the PCs.  I am guessing this would mean that if i log directly to the windows event log - then those logs can be gathered up by whatever logging infrastructure the client uses.  Am i correct in this?
0
btanExec ConsultantCommented:
yes, simple see it as windows event log is binary while syslog is test based. There is need for conversion if you need to support syslog which is most of the use cases for all installation. Hence, the snare or syslog_ng agent installed to send syslog to the log server deployed. that is the norm strategy from all hence snare alliance is one candidate which I suggested. SIEMS can ingest syslog as well.

Just some more for info on Snare in particular, Snare operates through the actions of a single component; the SnareCore service based application (snarecore.exe). The SnareCore service interfaces with the Windows event logging sub-system to read, filter and send event logs from the primary Application, System and Security event logs to a remote host. This host can be your syslog server or SIEM log collection point.

Note - Collecting events from Windows Logs is available for open source Snare agents, however collecting logs for Custom Event Logs and Applications and Services Logs is only available with the Enterprise Agents.  

As a whole for Snare, the Custom event log capability, TCP protocol capability, TLS/SSL support and the ability to send events to multiple hosts is only available to users who have purchased the Enterprise agents. https://www.intersectalliance.com/our-product/snare-agent/enterprise-vs-opensource/
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Network Management

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.