Setup Remote System Logging with rsyslog on Linux

The rsyslogtool takes care of receiving all the log message from the kernel and operating system applications and distributing them over files in /var/log.

However, rsyslog can do much more than that which includes logging into a remote server. This can be extremely useful for aggregating logs across a large fleet of servers or when it is not possible to write logs on disk.

In this tutorial, we’re going to install rsyslog on a remote machine so we can ship logs to, redirect all logging to that remote server.

Installing rsyslog on Remote Server

You will need a copy of rsyslog running on a remote machine which will be recieving the logs from your existing server. It’s best that you have this in a remote location. The reason being that if this server crashes at the same time as your server crashes, you won’t be able to get any logs to troubleshoot any issues.

Assuming that you’re using Ubuntu on the remote machine, you’ll already be running rsyslog. If not, you can install it by following the instructions provided inside the rsyslog website.

Once it’s installed, you will need to make sure that it listens on a port which we will send logs to. The default port is 514 which we’ll keep. You will need to edit the file /etc/rsyslog.conf

Local Storage Log Path

/var/syslog/hosts/Host Machine Name/ Year/Month/

Rsyslog Server Configuration file:

/etc/rsyslog.conf

$ModLoad imtcp

$ModLoad imudp

$ModLoad imuxsock

$ModLoad imklog

# Templates

# log every host in its own directory

$template RemoteHost,”/var/syslog/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/syslog.log”

### Rulesets

# Local Logging

$RuleSet local

kern.*                                                 /var/log/messages

*.info;mail.none;authpriv.none;cron.none                /var/log/messages

authpriv.*                                              /var/log/secure

mail.*                                                  -/var/log/maillog

cron.*                                                  /var/log/cron

*.emerg                                                 *

uucp,news.crit                                          /var/log/spooler

local7.*                                                /var/log/boot.log

# use the local RuleSet as default if not specified otherwise

$DefaultRuleset local

 

# Remote Logging

$RuleSet remote

*.* ?RemoteHost

# Send messages we receive to Gremlin

#*.* @@remote.server:514

#*.* @@rjilsiem.jio.com:514

 

#*.* @@remote.server2:514

*.* @@remote.server:514

 

### Listeners

# bind ruleset to tcp listener

$InputTCPServerBindRuleset remote

# and activate it:

$InputTCPServerRun 10514

 

$InputUDPServerBindRuleset remote

$UDPServerRun 514

 Log Source Configuration:

 Configuration file: /etc/rsyslog.conf

 $ModLoad imuxsock # provides support for local system logging

$ModLoad imklog   # provides kernel logging support

#$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

 $template MyTemplate, “<%pri%> %timestamp% zabbix.10.135.63.208 %syslogtag% %msg%\n”    ### these two lines will add zabbix.10.135.63.208 in fwded lines to easily identify source servers

$ActionForwardDefaultTemplate MyTemplate

 

$RepeatedMsgReduction on

$FileOwner syslog

$FileGroup adm

$FileCreateMode 0640

$DirCreateMode 0755

$Umask 0022

$PrivDropToUser syslog

$PrivDropToGroup syslog

$WorkDirectory /var/spool/rsyslog

$IncludeConfig /etc/rsyslog.d/*.conf

*.* @remoteserver:514

  

Troubleshooting Step:

  • Check rsyslogd service running
  • Port 514 UDP is open
  • Rsyslog server able to communicate with remote.server port 514

Thanks 🙂

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s