Migration to Syslog-ng v3.8

My Asus AP came with a primitive syslog in its firmware. So I decided to check Entware-ng for a better replacement. Found an ancient version (v2.1) of Syslog-ng. Set it up and everything worked great for years. Recently Entware-ng upgraded Syslog-ng to v3.8. That's a giant leap in functionality and efficiency. Here I'm going to share my experience of the migration.

My v2.1 syslog-ng.conf

options {  
    chain_hostnames(off); 
    sync(0); 

    # The default action of syslog-ng 1.6.0 is to log a STATS line
    # to the file every 10 minutes.  That's pretty ugly after a while.
    # Change it to every 12 hours so you get a nice daily update of
    # how many messages syslog-ng missed (0).
    stats(43200); 
};

source src { unix-stream("/dev/log"); internal(); };

destination messages { file("/opt/var/log/messages"); };  
destination d_iptables { file("/opt/var/log/iptables.log"); };  
destination d_pixelserv { file("/opt/var/log/pixelserv.log"); };

filter f_iptables { facility(kern) and match("DROP IN=" ); };  
filter f_pixelserv { facility(daemon) and match("pixelserv"); };  
filter f_default { not filter(f_pixelserv) and not filter(f_iptables); };

log { source(src); filter(f_pixelserv); destination(d_pixelserv); };  
log { source(src); filter(f_iptables); destination(d_iptables); };  
log { source(src); filter(f_default); destination(messages); };  

I trimmed the config without loss of any essential details. This config creates three log files, each with a filter sifting messages into it. Any dropped TCP/IP packets will log to /opt/var/log/iptables.log using filter f_iptables. Any pixelserv messages will log to /opt/var/log/pixelserv.log with filter f_pixelserv. And last, any other messages will log to default /opt/var/log/messages.

My v3.8 syslog-ng.conf

@version: 3.8

options {  
    chain_hostnames(no); 
    flush_lines(0); 
    stats-freq(43200); 
};

source src { unix-stream("/dev/log"); internal(); };

destination messages { file("/opt/var/log/messages"); };  
destination d_iptables { file("/opt/var/log/iptables.log"); };  
destination d_pixelserv { file("/opt/var/log/pixelserv.log"); };

filter f_iptables { facility(kern) and message("DROP IN=" ); };  
filter f_pixelserv { facility(daemon) and program("pixelserv"); };  
filter f_default { not filter(f_pixelserv) and not filter(f_iptables); };

log { source(src); filter(f_pixelserv); destination(d_pixelserv); };  
log { source(src); filter(f_iptables); destination(d_iptables); };  
log { source(src); filter(f_default); destination(messages); };  

First thing first. At the start of the config I have to add @version: 3.8 to hint the new Syslog-ng that the config is crafted for v3.8. When Syslog-ng starts and does not find this first line, it'll run in compatibility mode which is not optimal performance.

Three minor changes in the section of global options. For keyword chain_hostnames, its value off is replaced by no in v3.8. The purpose of chain_hostnames is described in detail here. Keyword sync is replaced by flush_lines. flush_lines(0) instructs Syslog-ng to process each received line alone and immediately. Keyword stats is replaced by stats-freq in v3.8. It takes a numeric argument indicating seconds of an interval where Syslog-ng dumps its own running statistics.

Sample output of Syslog-ng statistics

May  1 18:48:29 Phaeo syslog-ng[838]: Log statistics; processed='center(queued)=42692', processed='center(received)=42692', processed='destination(messages)=605',  processed='destination(d_pixelserv)=191', processed='destination(d_iptables)=27679', processed='source(src)=42349'  

The above logged message begins with a timestamp. Then hostname where Syslog-ng runs. Next is the program name followed by a colon. The rest of the line is message content that a program sends to syslog.

In Syslog-ng v2.1, keyword match in a filter can match any sub-string of the whole line. In v3.8, keyword program is required to match any sub-string of the program name. Keyword message is required for matching in the message content. Syslog-ng suggests such use increases efficiency in parsing. Unlike stats and sync which are obsolete in v3.8 (perhaps in earlier v3.x too), keyword match is still available. But I haven't checked how to use since all my need is fulfilled.

That sums up all I did to migrate from Syslog-ng v2.1 to v3.8. The new version is running great on my Asus AP. Will report back if I notice any speed increase.

Author

Stephen Yip

Something about you know. Come and share.

comments powered by Disqus