Transferring log file from Windows to Linux using tftp or ssh

There is a great utility out there called logparser which you could use to dumop the information from any log then upload it using simple ftp or other command line utility. I'm sure you could write up a batch script very easily and schedule that as a task to run automatically.

great thanks.

Would there be a way to have this script continuously running?

I am trying to accomplish uploading the event log files in near real-time

t

I'm not aware of any way to dump the event continuously with this tool but if you were dumping to a windows system you would be able to use the Windows Event Collector Service to consolidate logs from multiple servers.
You may be better off with a log consolidation/SIEM tool.  SolarWinds has a free log consolidating utility for windows.

It is still possible to script it with logparser and just use a filter to output any events that occured within the last 5 minutes or any interval you are willing to use as a delay.

Thanks for the input.

I want to dump the files on a linux system. Which is why i references SSH or something to transfer the files.

Standard unix log format is syslog
You can install rsyslog or syslog-ng agent on windows and send logs in realtime.

I see.

We are implementing an agentless solution. If we install the agent on the centralized windows collector to then send logs to the linux syslog server, it will assume the source is the collector.

Instead we are querying a sql db (where event logs reside) to pull all events based on a timeframe and then export the list sorted by computer name as the first column.

Then this file will be pushed to the linux syslog server

SQL DB sounds very realtime and agentless solution.

If i new of a way in which linux can connect to the sql db where the event logs reside this would be great.

I've heard of using an OBDC connector.

But that is just the tip of the iceberg. now sure what else needs to be programmed

I created thread below. however, I am not sure if this is the correct approach.

https://www.experts-exchange.com/questions/28551423/Linux-syslog-server-to-connect-to-Windows-Audit-Collections-Services-SQL-Database.html

How could database eventually speak Syslog to send logs to UNIX? Polling logs sounds weird.

I forgot about syslog so I now agree with gheist in the suggestions of an agent. The agent will automatically sync the latest events in case of downtime.

I am not sure. I was just thinking about the linux server connecting to the MS SQL db and extract the required logs (all/filtered list: pull events based on a timeframe and then sort by computer name as the first column)

Then import this information to syslog

ms sql and sybase driver for linux is called FreeTDS, or for java jTDS.

thanks for the great info

the only thing that remains is how to extract the required logs (all/filtered list: I want to pull events based on a timeframe and then sort by computer name as the first column)

Linux logs are files that are appended with new records. So no way to mess up time ordering.
Both syslog-ng and rsyslog allow extracting (like awk) computer name, then create subdirectory based on that.
(See their documentation, I find syslog-ng easier, but you may land in other side)

So this is as simple as installing the ms sql driver for linux.

Then running an awk script to extract the records?

Once the ms sql driver is installed, syslog-ng ( which is actually the syslog server we are using on linux) syslog can simply extract directly from the ms sql database?

Never heard of MS SQL supported on Linux.
TDS protocol driver will work.
Why should it extract from database when you can pipe logs directly?

I see. Thanks again.

linux newb

What exactly is it that you mean by piping directly?

For example, I install freetds on the linux syslog server and configure syslog to pipe logs "locally"

Since syslog-ng have awk capabilities?

Syslog-ng can parse log lines using regular expression matching.
I was suggesting installing syslog-ng or rsyslog on windows and forward logs with TCP and spooling on client side - completely reliable.
You will need to parse them ith syslog-ng afterwards (At least sending IP is not source host, so you need to  extract that field.

Sure it can read SQL responses or log files on the disk, but that is slow and unreliable input. With syslog agent on windows you will get 100% of logs copied over.

i see.

We are trying to avoid having to install anything on the windows side. We simply want to connect to the ms sql from the linux syslog server and parse the data from there.

I understand this can occur with the obdc, but what else needs to be configured. This sounds like a in house brewed solution which generally consist of custom tweaks and hacks.

syslog-ng cannot use SQL as source, only as destination.

I see.

So I will need to a custom program on the linux side to grab the data from ms sql db and dump it into a flat file where syslog-ng can grab it?