Logging with rsyslog (v13 Infinity)

Software Application Versions

• Rsyslog 5.8

Authentication

Use the SSL Certificate input to establish secure encrypted connections (using Stunnel) between the rsyslog server and its clients by using the SSL certificate and key for authentication purposes. By default, the input is set to use a credential called LOGGING_SSL_CRED. Therefore, you should create a credential called LOGGING_SSL_CRED that contains both the SSL certificate and key.

Security and Firewall Permissions

By default, log data is sent to the logging server using the UDP protocol (Logging Protocol) on port 514. If you are launching the rsyslog server in a cloud that uses security groups (i.e. Amazon EC2), you must create a security group with UDP port 514 open so that the rsyslog server can collect log data from each client server.

RELP Support for Log Data Delivery

Rsyslog includes support for the reliable event logging protocol (RELP), which guarantees delivery of event logging messages. When a connection is lost, you cannot reliably detect whether or not the last messages sent actually reached their destination. Unlike the syslog protocol, RELP works with a backchannel, over which information about received messages is conveyed back to the sender. This enables RELP to know which messages have been properly received when a connection has been lost.

Log Example

Log data for all rsyslog client servers is saved locally on the rsyslog server in /var/log/messages with the client's private IP address (if available) as a prefix for identification purposes. (e.g. ip-10-244-165-15)  See example output below.

# Note: When using newer images (>5.8/13.4), ensure that you have the 'server_superuser' permission to the Rightscale account where the server is running in order to gain root privileges using the sudo command (Settings > Account Settings > Users).
# sudo -i
# tail -f /var/log/messages
...
Feb 13 17:39:26 ip-10-245-20-219 RightLink[1340]: 17:39:26:   Updating iptables rule for IP Address: 10.253.39.203
Feb 13 17:39:26 ip-10-245-20-219 RightLink[1340]: 17:39:26: ruby_block[Adding firewall rule] called
Feb 13 17:39:26 ip-10-245-20-219 RightLink[1340]: 17:39:26: Chef Run complete in 3.178113 seconds
Feb 13 17:39:26 ip-10-245-20-219 RightLink[23782]: [cook] Disconnecting from agent (4 responses pending)
Feb 13 17:39:26 ip-10-245-20-219 RightLink[1340]: SEND b1 [push v20] (394 bytes) <> <1bd563311b78c96f4cbf2bb1ef942d8d> /updater/update_inputs, persistent
Feb 13 17:39:26 ip-10-245-20-219 RightLink[23782]: [cook] Process stopping
Feb 13 17:39:28 ip-10-245-27-18 RightLink[2385]: Converging
Feb 13 17:39:28 ip-10-245-27-18 RightLink[2385]: 17:39:28: *** Chef 0.10.10.2 ***
Feb 13 17:39:29 ip-10-245-27-18 RightLink[18907]: Opening new HTTP connection to 169.254.169.254:80

Log Data Backups

The ServerTemplate does not have built-in support for storing log data on volumes. It also does not contain any scripts that support backups of the log files. All log entries are stored locally on the rsyslog server's ephemeral drive and will be lost when the server is terminated. Therefore, you should only use his ServerTemplate for development and testing purposes only. If log data must persist after the logging server is terminated, you should consider using a third-party logging service or application such as Loggly or Splunk.