HOWTO: Send your logback to Logstash

I recently started playing around with OpenHAB, which is an excellent home automation application. During my "playing" with the system, I decided to try and run it on a Raspberry Pi 3 Model B. I figured that it's a fast enough computer to handle the load of the OpenHAB. One thing that I've read somewhere was that someone managed to kill 3 SD cards due to extensive logging (completely configurable). So, I thought... What if I could completely bypass writing my logs to local filesystem and go straight to ElasticSearch (via Logstash)? I know, the simplest solution would be to smb / nfs mount a drive and write to it, but I like the ability to of viewing my logs through a nice web UI. So, here's how I got this accomplished...

While I was searching how to do this task, I saw several logback modules that convert your log message to a JSON blob and sends it to the receiver to process. I could not get that to work and I did not want to recompile OpenHAB. So, what I ended up using was modifying my logback.xml file to add a new appender of type SyslogAppender. Here's the code block that I added to my logback.xml file:

<appender name="logstash" class="ch.qos.logback.classic.net.SyslogAppender">
    <syslogHost>Logstash server IP</syslogHost>
    <port>6001</port>
    <facility>LOCAL1</facility>
    <suffixPattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5level] [${HOSTNAME}] [logger] - %msg</suffixPattern>
</appender>

After the above appender has been added, remove all instances of <appender-ref ref="appender name" /> with <appender-ref ref="logstash" />

On the receiving side, I've added a new input configuration file to Logstash to handle these logs like so:

input {
  syslog {
    port => 6001
    type => "openhab_rpi"
  }
}

Of course, if you would like to make use of these messages, you'll need to properly parse these logs using Logstash.

Comments