Dirk große Osterhues

*[PyPI]: Python Package Index

This is an update to my previous article on how to set up FlexGet in Ubuntu. In this post, FlexGet v.1.0r2733 will be used on Ubuntu 11.10.

The reason for this new post is that the install and update processes have changed from using easy_install to pip, an alternative Python package installer. Both installation as also upgrading can be done with this tool.

Still, there have to be certain packages around:

aptitude install python2.6 python-setuptools python-pip

Now, the installation can easily take place:

pip install flexget

In case of an upgrade, the according line would be:

pip install --upgrade flexget

The following step is optional. In case an interface to transmission-daemon is of use, the according FlexGet-plugin is installed like this:

pip install transmissionrpc

Spamassassin already ships with the nice feature of running automatic updates to its core rules by using sa-update. Customisation of this update process is not well documented though. So it will be described in this article.

At http://wiki.apache.org/spamassassin/CustomRulesets quite some additional channels are listed. As an example the configuration of the German Language Ruleset will be outlined.

The Spamassassin Channel is documented on that website and says:

Available at: SpamAssassin Channel: 70_zmi_german.cf.zmi.sa-update.dostech.net

In your local Spamassassin's setup the update rules reside at /var/lib/spamassassin/{SA_VERSION_NUMBER}. There, you may want to add the new channel definition by introducing the according gpg-key as follows:

wget http://daryl.dostech.ca/sa-update/sare/GPG.KEY
sa-update --import GPG.KEY

Now, a manual channel update for that particular channel can be run:

sa-update --channel 70_zmi_german.cf.zmi.sa-update.dostech.net  --gpgkey 856AA88A

In order to have the cron-triggered nightly sa-update-run also update the new channel, a hook for sa-update has to be defined in /etc/spamassassin/sa-update-hooks.d/. There, create a file called german-rules-updates with the following content:

#!/bin/sh
sa-update --channel 70_zmi_german.cf.zmi.sa-update.dostech.net  --gpgkey 856AA88A

The German rules should now be updated each day and become active thereafter.

*[NCSA]: National Center for Supercomputing Applications

This article is deprecated. For the recent version check the following article: https://www.grosseosterhues.com/2011/12/squid-config-with-ncsa-authentication/

This article describes a quick and easy way to implement a web proxy with most basic user authentication. The authentication mechanism uses an NCSA-style htpasswd file, well known also for other basic authentication purposes. The proxy server in use will be Squid proxy.

To install the necessary software, this line will do:

aptitude install squid

Now, a new file containing usernames and password hashes needs to be created:

htpasswd -c /etc/squid/passwd USERNAME
chmod 400 /etc/squid/passwd && chown proxy /etc/squid/passwd

The file /etc/squid/squid.conf has to be modified. The following lines have to be uncommented while the first of them needs adjustment as to the whereabouts of the passwd file. By default it points to usr/etc/passwd but we want it to be /etc/squid/passwd:

auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/passwd
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic casesensitive off

In the ACL section two lines have to be inserted. The ACL section can be found by searching for "# ACCESS CONTROLS".

acl ncsa_users proxy_auth REQUIRED
http_access allow ncsa_users

In case you want Squid to not reveal your client's real IP address in HTTP requests it forwards, the follow option has to be disabled:

forwarded_for off 

A restart of Squid's daemon will activate the recent changes:

service squid restart 

*[MTA]: Mail Transfer Agent
Although virus scanners are installed on virtually all regular client systems nowadays, it's nice to have the mailserver pre-scanning mails for malware. This article describes the configuration of ClamAV in conjunction with Exim 4 in Ubuntu Linux.

Assuming a working Exim 4 environment, the fist step is to install the necessary ClamAV package:

aptitude install clamav-daemon

To tell Exim which anti-virus scanner to use, the file /etc/exim4/exim4.conf.template needs to be customized. The default config already brings the necessary lines. They just need to be adjusted and uncommented:

av_scanner = clamd:/var/run/clamav/clamd.ctl
[...]
deny
  malware = *
  message = This message was detected as possible malware ($malware_name).
warn
  message = X-Virus-Scanned: (${readsocket{/var/run/clamav/clamd.ctl}{VERSION}{2s}{}{AV Not running}})

In order for ClamAV to access the mail spool it has to become a member of Exim's group:

usermod -a -G Debian-exim clamav && service clamav-daemon restart

Now, sending a testmail like the one from EICAR should give you the appropriate malware rejection.

*[MTA]: Mail Transfer Agent
In this article the approach of greylisting as a protection against spam in Exim 4 is being described. The concepts of greylisting temporarily rejects mails at SMTP time. A regular MTA without spamming intent will try to send mail again after a certain period of time. The average trojan or worm on the other hand usually sends spam in waves. It will try to send its load only once and won't try again later. This concept facilitates spam reduction to a huge extent.

Setting up greylisting in Ubuntu Linux using Exim 4 is a breeze. To work, it simply needs the following lines.

Firstly, the installation of the package greylistd:

aptitude install greylistd

To activate the new package in Exim's configuration, the following command will do some magic:

greylistd-setup-exim4 add

The default configuration of greylistd is a bit too paranoid from my point of view. Therefore I adjust /etc/greylistd/config as follows (usually the last two existing entries in that file):

singlecheck = true
singleupdate = true

This has the effect that a domain that has already been verified won't become greylisted again if a different user sends from that particular domain. Also, it enables for manual manipulation of the whitelist by just adding IPs of "good" MTAs.

The whitelist file that is supposed to be edited by admins resides at /etc/greylistd/whitelist-hosts. The predifined whitelist can be found at /var/lib/greylistd/whitelist-hosts.

*[MTA]: Mail Transfer Agent
This article describes the configuration of the spam scanner called spamassassin in conjunction with the MTA called Exim 4 in Ubuntu Linux (as of writing Ubuntu 11.10 is the most recent). Assuming an otherwise working Exim 4 setup, this document describes just the essential steps to a working spamscanner.

First, the necessary packages need to be installed:

aptitude install exim4-daemon-heavy sa-exim spamassassin

In /etc/default/spamassassin set the following lines:

ENABLED=1
CRON=1

The first option lets the spamassassin daemon start on boot. The second one defines that spamassassin's rules become updated each night using a cron job.

In order to check if the first steps went ok, it's time to start spamassassin the very first time:

service spamassassin start

To check if spamassassin runs smoothly on it's dedicated port, this command helps:

lsof -i :783

Now in /etc/exim4/exim4.conf.template, insert the following lines in MAIN CONFIGURATION SETTINGS:

local_scan_path = /usr/lib/exim4/local_scan/sa-exim.so

To enable Exim's spamassassin connector, the file /etc/exim4/sa-exim.conf has to be modified. Comment the following line in that file:

# SAEximRunCond: 0

In default setups, debugging is enabled. That will lead to huge log files. So for production use, changing the following line to zero is advised:

SAEximDebug: 0

The updated file /etc/exim4/exim4.conf.template becomes active by running the following line:

update-exim4.conf && service exim4 restart

For a basic protection, the MTA is now good to go.

Since Ubuntu 11.10 multiarch support becomes set by default in new systems as well as in upgraded ones. This sounds like a nice feature, but it breaks one of my favorite tools: aptitude. As of now, aptitude can't handle multiple arches of the same package and therefore runs wild when told to do any task.

The solution to this is to disable multiarch for the time being. This is done by the following command:

mv /etc/dpkg/dpkg.cfg.d/multiarch /etc/dpkg/dpkg.cfg.d/multiarch.backup && apt-get update

In case aptitude learns how to handle multiarch some day, it can be re-enabled by running the command the other way round.

In order to migrate Cacti's rrd-tool data from one system to a different one, that data has to be converted to XML-format first. That is also true if you move from x86 to a x86_64.

To to so, first convert all rrd-data in /var/lib/cacti/rrd to XML using the following line in that folder:

for i in find -name "*.rrd"; do rrdtool dump $i > $i.xml; done

Now pack up the resulting XML-files:

tar czf cacti-xml.tar.gz /var/lib/cacti/rrd/*xml

Move the tar.gz-container to the new system and unpack its files in the same directory using:

tar xzf cacti-xml.tar.gz

The re-conversion of those XML-files into rrd format is done using this line:

for i in find -name "*.xml"; do rrdtool restore $i echo $i |sed s/.xml//g; done