Since there are a million HOWTOs on setting up remote logging with syslog-ng, I won’t bother going over it again. I will however take this moment to go into a little about how you can setup remote logging of your Squid servers. We are going to take advantage of some of the built in regex support of syslog-ng and also some other of the categorizing capabilities of syslog-ng.
Organization
Before we begin, I want to discuss a little about organization. It’s one of the things that I cover because I think it’s important. I won’t step up onto my soapbox as to why right now, but I will cover it some other time and it will relate to security and system administration which is what I know most of you are here for.
Keeping your logs organized allows programs like logrotate to do their job as well as log analysis scripts and even custom rolled scripts to do their jobs properly and efficiently. A part of organization is also syncronization. You should also ensure that NTP is properly setup so that the time’s on all log’s on the server and the client are in sync. Some log analysis programs are finicky and won’t work properly unless everything is in chronological order. Time fluctuations are also somewhat confusing to read if you are trying to do forensics on a server.
Squid Server Setup
Setting up your Squid server to do the loging and send it to a remote server is relatively easy. The first thing you need to do is to modify your squid.conf file to log to your syslog. Your squid.conf is generally located at /etc/squid/squid.conf. Find the line that begins with the access_log directive. It will likely look like this:
access_log /var/log/squid/squid.log squid
I recommend doing the remote logging as an addition to current local logging. Two copies are better than one, especially if you can spare the space and handle the network traffic. Add the following line to your squid.conf:
access_log syslog squid
This tells squid to create another access_log file, log it to the syslog in the standard squid logging format.
We also have to ensure that squid is not logged twice on your machine. This means using syslog-ng’s filtering capabilities to remove squid from being logged locally by the syslog. Edit your syslog-ng.conf file and add the following lines.
# The filter removes all entries that come from the
# program 'squid' from the syslog
filter f_remove { program("squid"); };
# Everything that should be in the 'user' facility
filter f_user { facility(user); };
# The log destination should be the '/var/log/user.log' file
destination df_user { file("/var/log/user.log"); };
# The log destination should be sent via UDP
destination logserver { udp("logserver.mycompany.com"); };
# The actual logging directive
log {
# Standard source of all sources
source(s_all);
# Apply the 'f_user' filter
filter(f_user);
# Apply the 'f_remove' filter to remove all squid entries
filter(f_remove);
# Send whatever is left in the user facility log file to
# to the 'user.log' file
destination(df_user);
# Send it to the logserver
destination(logserver);
};
Without describing all the lines that should be in a syslog-ng.conf file (as one should read the manual to find that out), I will merely say that the s_all has the source for all the syslog possiblities.
Log Server Setup
Although setting up your logserver might be a little more complex then setting up your squid server to log remotely, it is also relatively easy. The first item of interest is to ensure that syslog-ng is listening on the network socket. I prefer to use UDP even though there is no guarantee of message delivery like with TCP. It allows for network traffic latency when transferring data across poor connections. Do this by adding the udp() to your source directive:
# All sources
source src {
internal();
pipe("/proc/kmsg");
unix-stream("/dev/log");
file("/proc/kmsg" log_prefix("kernel: "));
udp();
};
Next you need to setup your destinations. This includes the destinations for all logs received via the UDP socket. As I spoke about organization already, I won’t beat a dead horse too badly, but I will show you how I keep my logs organized.
# Log Server destination
destination logs {
# Location of the log files using syslog-ng internal variables
file("/var/log/$HOST/$YEAR/$MONTH/$FACILITY.$YEAR-$MONTH-$DAY"
# Log files owned by root, group is adm and permissions of 665
owner(root) group(adm) perm(665)
# Create the directories if they don't exist with 775 perms
create_dirs(yes) dir_perm(0775));
};
We haven’t actually done the logging yet. There are still filters that have to be setup so we can see what squid is doing separate from other user level log facilities. We also have to ensure the proper destinations are created. Following along the same lines for squid,
# Anything that's from the program 'squid'
# and the 'user' log facility
filter f_squid { program("squid") and facility(user); };
# This is our squid destination log file
destination d_squid {
# The squid log file with dates
file("/var/log/$HOST/$YEAR/$MONTH/squid.$YEAR-$MONTH-$DAY"
owner(root) group(adm) perm(665)
create_dirs(yes) dir_perm(0775));
};
# This is the actual Squid logging
log { source(src); filter(f_squid); destination(d_squid); };
# Remove the 'squid' log entries from 'user' log facility
filter f_remove { not program("squid"); };
# Log everything else less the categories removed
# by the f_remove period
log {
source(src);
filter(f_remove);
destination(logs);
};
We have just gone over how one should organize basic remove logging and handle squid logging. Speaking as someone who has a lot of squid log analysis to do, centrally locating all my squid logs make log analysis and processing easier. I also don’t have to start transferring logs from machine to machine to do analysis. This is especially useful when logs like squid can be in excess of a few gigs per day.
