Transferring log file from Windows to Linux using tftp or ssh

Hello, I was wondering if I can use an application like WinSCP / openssh  or tftpd can constantly push log files from a windows server to a linux server?

thx
t
tobe1424Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

OriNetworksCommented:
There is a great utility out there called logparser which you could use to dumop the information from any log then upload it using simple ftp or other command line utility. I'm sure you could write up a batch script very easily and schedule that as a task to run automatically.
0
tobe1424Author Commented:
great thanks.

Would there be a way to have this script continuously running?

I am trying to accomplish uploading the event log files in near real-time

t
0
OriNetworksCommented:
I'm not aware of any way to dump the event continuously with this tool but if you were dumping to a windows system you would be able to use the Windows Event Collector Service to consolidate logs from multiple servers.
You may be better off with a log consolidation/SIEM tool.  SolarWinds has a free log consolidating utility for windows.

It is still possible to script it with logparser and just use a filter to output any events that occured within the last 5 minutes or any interval you are willing to use as a delay.
0
The Ultimate Tool Kit for Technolgy Solution Provi

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy for valuable how-to assets including sample agreements, checklists, flowcharts, and more!

tobe1424Author Commented:
Thanks for the input.

I want to dump the files on a linux system. Which is why i references SSH or something to transfer the files.
0
gheistCommented:
Standard unix log format is syslog
You can install rsyslog or syslog-ng agent on windows and send logs in realtime.
0
tobe1424Author Commented:
I see.

We are implementing an agentless solution. If we install the agent on the centralized windows collector to then send logs to the linux syslog server, it will assume the source is the collector.

Instead we are querying a sql db (where event logs reside) to pull all events based on a timeframe and then export the list sorted by computer name as the first column.

Then this file will be pushed to the linux syslog server
0
gheistCommented:
SQL DB sounds very realtime and agentless solution.
0
tobe1424Author Commented:
If i new of a way in which linux can connect to the sql db where the event logs reside this would be great.

I've heard of using an OBDC connector.

But that is just the tip of the iceberg. now sure what else needs to be programmed

I created thread below. however, I am not sure if this is the correct approach.

http://www.experts-exchange.com/Database/MS-SQL-Server/Q_28551423.html
0
gheistCommented:
How could database eventually speak Syslog to send logs to UNIX? Polling logs sounds weird.
0
OriNetworksCommented:
I forgot about syslog so I now agree with gheist in the suggestions of an agent. The agent will automatically sync the latest events in case of downtime.
0
tobe1424Author Commented:
I am not sure. I was just thinking about the linux server connecting to the MS SQL db and extract the required logs (all/filtered list: pull events based on a timeframe and then sort by computer name as the first column)

Then import this information to syslog
0
gheistCommented:
ms sql and sybase driver for linux is called FreeTDS, or for java jTDS.
0
tobe1424Author Commented:
thanks for the great info

the only thing that remains is how to extract the required logs (all/filtered list: I want to pull events based on a timeframe and then sort by computer name as the first column)
0
gheistCommented:
Linux logs are files that are appended with new records. So no way to mess up time ordering.
Both syslog-ng and rsyslog allow extracting (like awk) computer name, then create subdirectory based on that.
(See their documentation, I find syslog-ng easier, but you may land in other side)
0
tobe1424Author Commented:
So this is as simple as installing the ms sql driver for linux.

Then running an awk script to extract the records?

Once the ms sql driver is installed, syslog-ng ( which is actually the syslog server we are using on linux) syslog can simply extract directly from the ms sql database?
0
gheistCommented:
Never heard of MS SQL supported on Linux.
TDS protocol driver will work.
Why should it extract from database when you can pipe logs directly?
0
tobe1424Author Commented:
I see. Thanks again.

linux newb

What exactly is it that you mean by piping directly?

For example, I install freetds on the linux syslog server and configure syslog to pipe logs "locally"

Since syslog-ng have awk capabilities?
0
gheistCommented:
Syslog-ng can parse log lines using regular expression matching.
I was suggesting installing syslog-ng or rsyslog on windows and forward logs with TCP and spooling on client side - completely reliable.
You will need to parse them ith syslog-ng afterwards (At least sending IP is not source host, so you need to  extract that field.

Sure it can read SQL responses or log files on the disk, but that is slow and unreliable input. With syslog agent on windows you will get 100% of logs copied over.
0
tobe1424Author Commented:
i see.

We are trying to avoid having to install anything on the windows side. We simply want to connect to the ms sql from the linux syslog server and parse the data from there.

I understand this can occur with the obdc, but what else needs to be configured. This sounds like a in house brewed solution which generally consist of custom tweaks and hacks.
0
gheistCommented:
syslog-ng cannot use SQL as source, only as destination.
0
tobe1424Author Commented:
I see.

So I will need to a custom program on the linux side to grab the data from ms sql db and dump it into a flat file where syslog-ng can grab it?
0
gheistCommented:
Yes, it can work like that.. Custom program is more like perl script that copies SQL result to CSV, but ok.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2008

From novice to tech pro — start learning today.