Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Syslog is broken

Posted on 2008-10-01
8
Medium Priority
?
856 Views
Last Modified: 2013-12-16
Someone booted the host and now syslog's not working.  Would appreciate trouble-shooting tips.

ps -ef | grep syslog
root      3954     1  0 15:50 ?        00:00:00 syslogd -m 0

/etc/syslog.conf snippet:
local6.warning                                          /var/log/cisco/routers/router
local6.debug                                            /var/log/cisco/routers/router
local6.info                                             /var/log/cisco/routers/router

tcpdump output:
15:59:50.179709 IP c3845-inet1.pcmt.local.52931 > netmon.pooched.local.syslog: UDP, length 142
15:59:50.180685 IP c3845-inet1.pcmt.local.52931 > netmon.pooched.local.syslog: UDP, length 142
15:59:50.469254 IP c3845-inet1.pcmt.local.52931 > netmon.pooched.local.syslog: UDP, length 142
15:59:50.469283 IP netmon.pooched.local > c3845-inet1.pcmt.local: icmp 178: netmon.pooched.local udp port syslog unreachable
0
Comment
Question by:amigan_99
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 4
8 Comments
 
LVL 13

Expert Comment

by:WizRd-Linux
ID: 22620098
Iptables hasn't changed?
If you restart syslog, does it begin working?
Does 'netstat -al' show syslog listening?
0
 
LVL 1

Author Comment

by:amigan_99
ID: 22620162
Update: So syslog itself is working somewhat in that boot.log is working, /var/log/messages, /var/log/mail..

But it's the syslog traffic that can't find its way to facility 6.  But that appears set fine in /etc/syslog.conf.

/etc/syslog.conf snippet:
local6.warning                                          /var/log/cisco/routers/router
local6.debug                                            /var/log/cisco/routers/router
local6.info                                             /var/log/cisco/routers/router

As I showed before - the cisco router syslog traffic arrives at the interface of the linux host.  But for some reason the linux host (netmon)  sends back "udp port syslog unreachable".  

15:59:50.469283 IP netmon.pooched.local > c3845-inet1.pcmt.local: icmp 178: netmon.pooched.local udp port syslog unreachable
0
 
LVL 1

Author Comment

by:amigan_99
ID: 22620276
I don't see syslog port 514 in play..

[root@netmon etc]# netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 *:832                       *:*                         LISTEN
tcp        0      0 *:mysql                     *:*                         LISTEN
tcp        0      0 *:sunrpc                    *:*                         LISTEN
tcp        0      0 netmon.pcmt.local:ipp       *:*                         LISTEN
tcp        0      0 netmon.pcmt.local:smtp      *:*                         LISTEN
tcp        0      0 *:8000                      *:*                         LISTEN
tcp        0      0 *:http                      *:*                         LISTEN
tcp        0      0 *:ssh                       *:*                         LISTEN
tcp        0      0 *:https                     *:*                         LISTEN
udp        0      0 *:826                       *:*
udp        0      0 *:829                       *:*
udp        0      0 *:tftp                      *:*
udp        0      0 *:sunrpc                    *:*
udp        0      0 *:ipp                       *:*
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     6134   /tmp/.font-unix/fs7100
unix  2      [ ACC ]     STREAM     LISTENING     8331   /tmp/.gdm_socket
unix  2      [ ACC ]     STREAM     LISTENING     8450   /tmp/.X11-unix/X0
unix  2      [ ACC ]     STREAM     LISTENING     5937   /dev/gpmctl
unix  2      [ ACC ]     STREAM     LISTENING     5415   /var/run/acpid.socket
unix  2      [ ACC ]     STREAM     LISTENING     5726   /var/lib/mysql/mysql.sock
unix  2      [ ACC ]     STREAM     LISTENING     6211   /var/run/dbus/system_bus_socket
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 
LVL 13

Expert Comment

by:WizRd-Linux
ID: 22620402
Syslog should be listed as listening to port 514.

Restart syslog and check /var/log/messages or /var/log/syslog to see if any errors are reported about binding to the port, or specific interfaces.
0
 
LVL 1

Author Comment

by:amigan_99
ID: 22620477
[root@netmon etc]# /etc/rc.d/init.d/syslog restart
Shutting down kernel logger:                               [  OK  ]
Shutting down system logger:                               [  OK  ]
Starting system logger:                                    [  OK  ]
Starting kernel logger:                                    [  OK  ]

From /sys/log/messages:
Oct  1 17:53:45 netmon kernel: Kernel logging (proc) stopped.
Oct  1 17:53:45 netmon kernel: Kernel log daemon terminating.
Oct  1 17:53:46 netmon syslog: klogd shutdown succeeded
Oct  1 17:53:46 netmon exiting on signal 15
Oct  1 17:53:47 netmon syslogd 1.4.1: restart.
Oct  1 17:53:47 netmon syslog: syslogd startup succeeded
Oct  1 17:53:47 netmon syslog: klogd startup succeeded
Oct  1 17:53:47 netmon kernel: klogd 1.4.1, log source = /proc/kmsg started.
Oct  1 17:53:47 netmon syslog: syslogd shutdown succeeded

[root@netmon log]# ps -ef | grep syslogd
root      4368     1  0 17:53 ?        00:00:00 syslogd -m 0

[root@netmon etc]# netstat -l | grep udp
udp        0      0 *:826                       *:*
udp        0      0 *:829                       *:*
udp        0      0 *:tftp                      *:*
udp        0      0 *:sunrpc                    *:*
udp        0      0 *:ipp                       *:*

Nothing in the router log file:
[root@netmon log]# ls -l /var/log/cisco/routers/router
-rw-rw-rw-  1 root root 0 Oct  1 04:02 /var/log/cisco/routers/router




0
 
LVL 13

Accepted Solution

by:
WizRd-Linux earned 2000 total points
ID: 22620541
Syslog doesn't have the -r option... which means it can't receive messages from remote hosts, including routers.

Edit the file /etc/sysconfig/syslog and modify the line SYSLOGD_OPTIONS="-m 0" to be SYSLOGD_OPTIONS="-m 0 -r"

Sorry I didn't notice this earlier.
0
 
LVL 13

Expert Comment

by:WizRd-Linux
ID: 22620547
and then restart syslog with /etc/init.d/syslog restart (service syslog restart)
0
 
LVL 1

Author Closing Comment

by:amigan_99
ID: 31502214
All kneel to the WizRd!  Woot!

[root@netmon routers]# ls -l
total 311000
-rw-rw-rw-  1 root root   103197 Oct  1 21:01 router
-rw-r--r--  1 root root  7899873 Sep 22 04:02 router.10.gz
-rw-r--r--  1 root root  8161644 Sep 21 04:02 router.11.gz
-rw-r--r--  1 root root 10608622 Sep 20 04:02 router.12.gz
0

Featured Post

Connect further...control easier

With the ATEN CE624, you can now enjoy a high-quality visual experience powered by HDBaseT technology and the convenience of a single Cat6 cable to transmit uncompressed video with zero latency and multi-streaming for dual-view applications where remote access is required.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you have a server on collocation with the super-fast CPU, that doesn't mean that you get it running at full power. Here is a preamble. When doing inventory of Linux servers, that I'm administering, I've found that some of them are running on l…
In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Suggested Courses

721 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question