Dirk große Osterhues

[Sieve]: Sieve: filtering language
[Openmailadmin]: IMAP management frontend
[IMAP Idle]: Immediate notification of users about any mailbox changes
[IMAP]: IMAP: Internet message access protocol
[Roundcube]: Roundcube: browser-based IMAP client
[SASL]: Simple Authentication and Security Layer
*[MTA]: Mail Transfer Agent

This is a follow-up article on the previous post on how to Migrate IMAP Accounts from one Server to another.

Roundcube is a very decent browser-based IMAP client with an application-like UI.

Retrieving Roundcube package

As Roundcube is under heavy development, the Ubuntu packages are quickly outdated. Therefore, we download the most recent package from the Roundcube project page http://roundcube.net/download.

Download the linked tar.gz-archive into /var/www:

cd /var/www && wget {copied_download_link}
tar xzf {downloaded_file}.tar.gz && mv roundcubemail-0.5.1 roundcubemail

Creating necessary SQL tables

The next step is to create the SQL tables. Do is as follows:

cd /var/www/roundcubemail
mysql -u root -p # enter MySQL's root-user password 
  CREATE DATABASE roundcubemail /*!40101 CHARACTER SET utf8 COLLATE utf8_general_ci */;
  GRANT ALL PRIVILEGES ON roundcubemail.* TO roundcube@localhost \
  IDENTIFIED BY '{choose_your_password}';
  quit
mysql -u root -p roundcubemail < SQL/mysql.initial.sql

Adjusting Roundcube config-files

Now, the Roundcube configuration needs to know about the whereabouts of its database. Therefore, edit the according file config/db.inc.php. First create a local copy of the dist file:

cd /var/www/roundcubemail
cp config/db.inc.php.dist config/db.inc.php

Replace "pass" with Roundcube's database password in the following line:

$rcmail_config['db_dsnw'] = 'mysql://roundcube:pass@localhost/roundcubemail';

Now, create the second relevant configuration file, main.inc.php:

cp config/main.inc.php.dist config/main.inc.php

To use Roundcube for your locally installed server, enter localhost in the following line:

$rcmail_config['default_host'] = '';

There are loads of other configuration options in main.inc.php, giving the possibility to tweak the very last bit of Roundcube. You might want to read the according comments in the config file which describe them in detail.

[Sieve]: Sieve: filtering language
[Openmailadmin]: IMAP management frontend
[IMAP Idle]: Immediate notification of users about any mailbox changes
[IMAP]: IMAP: Internet message access protocol
[Roundcube]: Roundcube: browser-based IMAP client
[SASL]: Simple Authentication and Security Layer
*[MTA]: Mail Transfer Agent

This is a follow-up article on the previous post on how to Migrate IMAP Accounts from one Server to another.

Openmailadmin is an administrative interface to IMAP mailservers including cyrus-imapd. It requires a webserver (we use Apache), a database (MySQL in our case) and PHP of version 5.1 or later. Openmailadmin features a generic administration hierarchy and is fairy easy to use -- especially compared to a rarely used and rather cryptic cyradm cli-command.

Required Ubuntu packages

To setup Openmailadmin, this line needs to be run in order to install the necessities:

aptitude install apache2-mpm-prefork php5-mysql mysql-server-5.1 \
                 libphp-adodb php-log php5-idn

You might want to enable SSL in Apache2. If so follow the Simple activation of SSL for Apache2 in Ubuntu 10.10.

Get Openmailadmin

Now, it's time to get the Openmailadmin package. Create it's directory, download the package and extract it like this:

mkdir -p /var/www/ && cd /var/www/
wget http://static.ossdl.de/openmailadmin/downloads/openmailadmin-1.0.1.tbz2
tar xjf openmailadmin-1.0.1.tbz2
mv openmailadmin-1.0.1 openmailadmin && rm openmailadmin-1.0.1.tbz2
chown -R www-data:www-data /var/www/openmailadmin

Set up Openmailadmin

The next step is to access setup.php by opening https://www.{your_server_name}.tld/openmailadmin. If you installed all dependencies in the first step, you should be good to go to set up the Openmailadmin backend.

The second step in the setup frontend is a bit more complicated, yet quite well inline-documented.

Configuration of PAM

To make saslauthd work with your new databased authentication system, you will need to install the appropiate package:

aptitude install libpam-mysql                   

Also, you neet to adjust /etc/pam.d/imap. Replace these lines:

@include common-auth
@include common-account 

with these (don't forget to set the variables in the curly brackets with your settings):

auth    sufficient pam_mysql.so user={db_user} passwd={db_password} host=localhost db={db_name} table=user usercolumn=mbox passwdcolumn=password crypt=3 sqlLog=0
account required   pam_mysql.so user={db_user} passwd={db_password} host=localhost db={db_name} table=user usercolumn=mbox passwdcolumn=password crypt=3 sqlLog=0

This configuration uses MD5 for password hashes. To use SHA1, you may want to replace crypt=3 with crypt=4 and edit /var/www/openmailadmin/inc/config.local.inc.php accordingly.

After entering all your config data, you may use your new administrative frontend to cyrus-imapd at https://www.{your_server_name}.tld/openmailadmin.

There is a easy to use tool that facilitates the quick'n dirty move of the content of one IMAP account to a new server. Its name is imapsync. In Ubuntu, you may install it as this:

aptitude install imapsync

Now, set two files for the old and the new account in your home, containing the respective passwords of the two accounts. They are called passfile_1 and passfile_2 from now on.

Now run imapsync with the following options:

imapsync --host1 {old_imap_host} --user1 {old_imap_user} --authmech1 LOGIN \
         --passfile1 passfile_1 --port1 993 --ssl1 \
         --host2 {new_imap_host} --user2 {new_imap_user} --authmech2 LOGIN \
         --passfile2 passfile_2 --port2 993 --ssl2

You just need to adjust the ports (143 vs. ssl) and the authentication mechanisms to your needs and you're set.

[Sieve]: Sieve: filtering language
[Openmailadmin]: IMAP management frontend
[IMAP Idle]: Immediate notification of users about any mailbox changes
[IMAP]: IMAP: Internet message access protocol
[Roundcube]: Roundcube: browser-based IMAP client
[SASL]: Simple Authentication and Security Layer
*[MTA]: Mail Transfer Agent

This article describes the installation of Cyrus IMAP-server v2.2. It will use SSL for secure access and SASL for user authentication.

Follow-up articles will explain how to add the Exim 4 as MTA, the IMAP administrative management system Openmailadmin, the spamfilter SpamAssassin and Roundcube as the webfrontend to the IMAP-server for the users.

Necessary packages

In order to set up Cyrus IMAP-server, some necessary packages have to be installed:

aptitude install cyrus-imapd-2.2 cyrus-common-2.2 sasl2-bin cyrus-admin-2.2

Activation of Saslauthd auth-daemon

In order for saslauthd to start, the following line in /etc/default/saslauthd has to be set:

# Should saslauthd run automatically on startup? (default: no)
START=yes

Now, start saslauthd:

service saslauthd start

Configuration of /etc/cyrus.conf

Now, open the file /etc/cyrus.conf in which some changes should be made. It is divided into three sections called START, SERVICES and EVENTS`.

We start with IMAP Idle. To enable it, uncomment the following line in the START section:

idled          cmd="idled"

To activate secure IMAP access and to disable NNTP the SERVICES section has to be set:

# --- Normal cyrus spool, or Murder backends ---
# add or remove based on preferences
#imap            cmd="imapd -U 30" listen="localhost:imap" prefork=0 maxchild=100
imaps           cmd="imapd -s -U 30" listen="imaps" prefork=0 maxchild=100
#pop3           cmd="pop3d -U 30" listen="pop3" prefork=0 maxchild=50
#pop3s          cmd="pop3d -s -U 30" listen="pop3s" prefork=0 maxchild=50
#nntp           cmd="nntpd -U 30" listen="nntp" prefork=0 maxchild=100
#nntps          cmd="nntpd -s -U 30" listen="nntps" prefork=0 maxchild=100

In the EVENTS section, we want to enable the SQUAT indexes for mailboxes, enabling significantly reduced search times:

# reindex changed mailboxes (fulltext) approximately every other hour
squatter_1      cmd="/usr/bin/nice -n 19 /usr/sbin/squatter -s" period=120

Configuration of /etc/imapd.conf

In /etc/imapd.conf you first need to declare the cyrus' admin user:

admins: cyrus

Now, define the authentication mechanism by setting sasl_mech_list and sasl_pwcheck_method:

sasl_mech_list: LOGIN PLAIN
sasl_pwcheck_method: auxprop saslauthd
sasl_auxprop_plugin: sasldb

To activate SSL for secure IMAP access, set the following variables and place your certificates and key accordingly:

tls_cert_file:          /etc/ssl/certs/{your_server_name}.cert.pem
tls_key_file:           /etc/ssl/private/{your_server_name}.key.pem
tls_ca_file:            /etc/ssl/{your_ca_chain}.pem 

Make sure Cyrus can read the SSL files by running this:

chmod 640 /etc/ssl/certs/{your_server_name}.cert.pem \
          /etc/ssl/private/{your_server_name}.key.pem \
          /etc/ssl/{your_ca_chain}.pem

chown :mail /etc/ssl/certs/{your_server_name}.cert.pem \
            /etc/ssl/private/{your_server_name}.key.pem \
            /etc/ssl/{your_ca_chain}.pem

Also in this file, the IMAP Idle functionality has to be set defining idlemethod:

idlemethod: idled

Activate your setting by restarting the Cyrus daemon:

service cyrus2.2 restart

Initial user creation

First, create an inital SASL password in order to enable cyrus' administrative user:

saslpasswd2 -c cyrus

Now, authenticate yourself as the user cyrus to cyradm with the previous password:

cyradm --user cyrus localhost

As user cyrus in cyradm, you may do maintenance tasks like creating a new user:

cm user.{username}

Test your authentication by running:

testsaslauthd -u username -p password

If that worked, you may now connect to your new IMAP account by accessing your server on port 993 and using the set username and password as authentication.

This article explains installation of most recent Transmission versions into Ubuntu 10.10.

Ubuntu's Personal Package Archive (https://launchpad.net/ubuntu/+ppas) allows to install packages from a package repository that is managed by individual package maintainers. It contains software that is not part of the standard Ubuntu repository or that is more recent than packages in regular Ubuntu flavors.

In order to use PPAs, the package python-software-properties has to be available on the system:

aptitude install python-software-properties

Now, the transmission-ppa can be installed using the following command:

add-apt-repository ppa:transmissionbt/ppa

After updating the local package information the new transmission-daemon can be installed:

aptitude update && aptitude install transmission-daemon 

*[LUKS]: Linux Unified Key Setup
This article describes the process of setting up a physical drive with LUKS encrypted filesystem. The process is tested on Ubuntu 10.10.

Prepratations

First, one necessary package needs to be installed:

aptitude install cryptsetup

Setup of the partition

Initialisation of the encrypted partition is done like this:

cryptsetup -h sha256 -c aes-cbc-essiv:sha256 -s 256 luksFormat /dev/{physical_partition}

Now, the newly encrypted partition needs to be introduced to the system i.e. to the device mapper. Do this as follow with cryptfs being the device's mapping name, i.e. name it as you like:

cryptsetup luksOpen /dev/{physical_partition} cryptfs

The next step is to set up a filesystem on the partition. I prefer xfs, but you are free to choose what suits you best:

mkfs.xfs /dev/mapper/cryptfs

Now, mount your new partition:

mount /dev/mapper/cryptfs /{mount_point}

If that worked, you may want to add your encrypted partition to /etc/fstab by adding the following line. Replace {mount_point} with the location where you wish to mount your device:

/dev/mapper/cryptfs /{mount_point} xfs defaults 0 2

Set up mount at boot time

In order to let the device get mounted at boot time, you may want to add these lines to /etc/rc.local. Bear in mind that now you'll have to enter the cryptfs passphrase at boot:

cryptsetup luksOpen /dev/{crypt_partition} cryptfs
mount /dev/mapper/cryptfs

If you wish to easily mount the device from shell after boot, save the following lines as /usr/local/bin/crypt-start:

#!/bin/sh
cryptsetup luksOpen /dev/{crypt_partition} cryptfs
mount /dev/mapper/cryptfs

Make the file executable:

chmod +x /usr/local/bin/crypt-start

... and run the command crypt-start after boot.

Change LUKS passphrase

Changing an existing LUKS passphrase seems a bit odd on first sight. That is, because you first need to add a new passphrase before you can remove the old one. Yet, this helps to only change your passphrase at a point where you can be absolutely certain that you do know the new passphrase. The relevant commands look like this:

cryptsetup luksAddKey /dev/{physical_partition}
cryptsetup luksRemoveKey /dev/{physical_partition}

The qcow2 disk format has some decent features like encryption, compression and -- as its abbreviation indicates -- copy to write support. Yet, its growing size is difficult to predict. In addition, the compression and the copy processes make it quite a bit slower than raw disk images. A simple and non-representative benchmark (bonnie++ -b -u root -d /mnt) shows it:

qcow2:

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
<host>           2G   856  94 33126   7 30392   7  4727  92 739753  70 300.4  20
Latency             51096us    2473ms     972ms    5240us    1516us     382ms
Version  1.96       ------Sequential Create------ --------Random Create--------
<host>              -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16    35   0 +++++ +++    46   0    34   0 +++++ +++    44   0
Latency               338ms     808us    2136ms     341ms     155us    2016ms

Raw:

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
<host>           2G   885  97 18719  15 75344  70  3993  88 1663036  81 351.6 138
Latency             12763us   34358ms     273ms    4526us     520us     310ms
Version  1.96       ------Sequential Create------ --------Random Create--------
<host>              -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16    35   0 +++++ +++    49   0    35   0 +++++ +++    47   0
Latency              1324ms     797us     915ms     452ms     156us     823ms

So if you're like me and still refrain from using LVM as underlying disk layout but prefer images which can be copied and moved in a quick'n dirty way, you might want to migrate your qcow2 images to raw ones. This article describes the necessary steps.

Shutdown of the VM

First, shutdown the vm which disks you wish to convert:

virsh shutdown {vm_name}

Conversion of the image files

Each of the image files of the virtual machine have to be converted by running the following for the respective files:

qemu-img convert {image_name}.qcow2 {image_name}.raw 

Adapt the virtual machine's xml config

Now, two lines of the vm's xml configuration have to be adjusted. Do this by running:

 virsh edit {vm_name}

The lines are:

Change them to match:

This article assumes a KVM guest that runs from raw or qcow2 devices using emulated ide access. Since some time now, full virtualisation for block devices is available, facilitating the benefit of paravirtualisation.

Benchmarked in bonnie++, the difference looks like this, revealing a way better performance of virtio:

IDE:

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
<host>          2G   856  94 33126   7 30392   7  4727  92 739753  70 300.4  20
Latency             51096us    2473ms     972ms    5240us    1516us     382ms
Version  1.96       ------Sequential Create------ --------Random Create--------
<host>              -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16    35   0 +++++ +++    46   0    34   0 +++++ +++    44   0
Latency               338ms     808us    2136ms     341ms     155us    2016ms

Virtio:

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
<host>          2G   732  91 58767   8 72183  11  4026  94 1477698  80 321.6   9
Latency             16751us    1159ms     492ms    4155us     624us     422ms
Version  1.96       ------Sequential Create------ --------Random Create--------
<host>              -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16    34   0 +++++ +++    46   0    34   0 +++++ +++    45   0
Latency               240ms     790us    1502ms     296ms     167us    1450ms

So in order to make the switch from from ide to virtio, the following steps need to be taken:

Run virsh edit <your_vm_name>. From there, edit the config file and adjust all lines of

<target dev='hda' bus='ide'/>

so they look like this

<target dev='vda' bus='virtio'/>

Furthermore, remove all <address type .../> lines so that libvirt can regenerate them appropriately.

Inside the guest, edit /etc/fstab and replace all occurrences of /dev/sdX with /dev/vdX`.

That's it, now shutdown the machine and start it with an virsh start <your_vm_name> (just a reboot inside the started VM won't work).

Introduction

Starting with Firefox 4 it is really easy to keep your different Firefox browser setups at home, on your mobile or at work in sync. The Mozilla Corporation even hosts a central server for you so basically the setup can't be more easy.

Yet, if you're a little paranoid and you don't want to store your precious browser data on someoneelse's servers, you might wish for your own sync server. There are two posibilities to setup your own server: the Weave Minial Server and the full blown one.

Weave Minimal Server's development has been stopped now and is deprecated since shortly after FF4's relase. Therefore, this article documents the setup of the full server.

The chosen install location will be /var/www/weave. The article assumes a Mysql server for authentication installed locally. Further, a functional Apache 2 server is assumed with a fully configured SSL setup. The documentation was tested using an Ubuntu 10.10 setup.

Package Retrieval and Storage

A working installation of a full Weave server consists of of two packages: the Weave Sync Server and the Weave Registration Server. You may download and unpack the archives like this:

mkdir -p /var/www/weave/weave-synchronisation \
         /var/www/weave/weave-registration 
cd /var/www/weave && touch index.html
   wget http://hg.mozilla.org/services/sync-server/archive/tip.tar.gz
   tar xzf tip.tar.gz -C weave-synchronisation --strip=1 && rm tip.tar.gz
   wget http://hg.mozilla.org/services/reg-server/archive/tip.tar.gz
   tar xzf tip.tar.gz -C weave-registration --strip=1 && rm tip.tar.gz

In case the download links are deprecated, go and look for the according packages at their respective root directories: http://hg.mozilla.org/services/sync-server/ and http://hg.mozilla.org/services/reg-server/.

Note as of Oct 29, 2011: the links are deprecated as the whole sync-server gets an overhaul. The old version is still running smooth though, so the updated links are: http://hg.mozilla.org/services/deprecated/sync-server/ and http://hg.mozilla.org/services/deprecated/reg-server/.

Database Setup

Create a new database called weave containing the following tables:

CREATE TABLE collections (
  userid int(11) NOT NULL,
  collectionid smallint(6) NOT NULL,
  name varchar(32) NOT NULL,
  PRIMARY KEY  (userid,collectionid),
  KEY nameindex (userid,name)
) ENGINE=InnoDB;

CREATE TABLE wbo (
  username int(11) NOT NULL,
  collection smallint(6) NOT NULL default '0',
  id varbinary(64) NOT NULL default '',
  parentid varbinary(64) default NULL,
  predecessorid varbinary(64) default NULL,
  sortindex int(11) default NULL,
  modified bigint(20) default NULL,
  payload longtext,
  payload_size int(11) default NULL,
  ttl int(11) default '2100000000',
  PRIMARY KEY  (username,collection,id),
  KEY parentindex (username,collection,parentid),
  KEY modified (username,collection,modified),
  KEY weightindex (username,collection,sortindex),
  KEY predecessorindex (username,collection,predecessorid),
  KEY size_index (username,payload_size)
) ENGINE=InnoDB;

create table users (
 id int(11) NOT NULL PRIMARY KEY auto_increment,
 username varchar(32),
 password_hash varbinary(128),
 email varbinary(64),
 status tinyint(4) default '1',
 alert text,
 reset varbinary(32) default null,
 reset_expiration datetime
) engine=InnoDB;

Now, create a user named weave in mysql with data permissions on the newly created Weave database. Remember the user's password for later.

Config Adjustments on Sync and Reg Server

To customise the configs of reg and sync portion of the full Weave server, two files have to be created before being edited:

cd /var/www/weave/weave-synchronisation/1.1 && \
   cp default_constants.php.dist default_constants.php
cd /var/www/weave/weave-registration/1.0 && \
   cp weave_user_constants.php.dist weave_user_constants.php

Adjustment of the Synchronisation Server

Now, edit /var/www/weave/weave-synchronisation/1.1/default_constants.php. First, adjust the mysql user's database password according to the previous step. Chanve weave to your own password there in the two line that say:

if (!defined('WEAVE_MYSQL_STORE_READ_PASS')) { define('WEAVE_MYSQL_STORE_READ_PASS', 'weave'); }

Next, adjust the size of maximum payload in the following line. I recommend just adding a zero to 262144:

if (!defined('WEAVE_PAYLOAD_MAX_SIZE')) { define('WEAVE_PAYLOAD_MAX_SIZE', 262144); } #256K

Now, uncomment the following line and insert a random character string where its supposed to:

#if (!defined('WEAVE_SHA_SALT')) { define('WEAVE_SHA_SALT', 'salt goes here'); }

You might also want to adjust the default quota from 5 MB to maybe 20 MB. Uncomment this line and adjust it:

#if (!defined('WEAVE_QUOTA')) { define('WEAVE_QUOTA', 5120000); } #5M

Adjustment of the Registration Server

The configuration file for the registration server is located at /var/www/weave/weave-registration/1.0/weave_user_constants.php. The following lines need to be customised. First, insert the mysql password by replacing the string weave:

if (!defined('WEAVE_MYSQL_AUTH_PASS')) { define('WEAVE_MYSQL_AUTH_PASS', 'weave'); }

Next, uncomment the salt line as before and insert the same character string as before for the sync server:

#if (!defined('WEAVE_SHA_SALT')) { define('WEAVE_SHA_SALT', 'salt goes here'); }

If you like to have user registration protected by captcha (I assume you do), get an account at http://recaptcha.net/ and get yourself a global key. Adjust the following three line to suit your needs:

#if (!defined('WEAVE_REGISTER_USE_CAPTCHA')) { define('WEAVE_REGISTER_USE_CAPTCHA', 0); }
#if (!defined('RECAPTCHA_PUBLIC_KEY')) { define('RECAPTCHA_PUBLIC_KEY', '6LfWcwUAAAAAABnmLyhmgddYeJGdiRlo2MWSOpAl'); }
#if (!defined('RECAPTCHA_PRIVATE_KEY')) { define('RECAPTCHA_PRIVATE_KEY', '6LfWcwUAAAAAAHpjpBNSaxwLVQXQIG-S0Y6IG38O '); }

Apache2 Setup

This article assumes that you have your Apache 2 webserver SSL-secured and that you want the Weave server to be located at https://yourdomain.tld/weave/. Therefore, the following lines need to be inserted into your Apache 2 config (in Ubuntu that is /etc/apache2/sites-enabled/default-ssl):

# ----- weave-server -----------------------------------------------------------------

  Options Indexes FollowSymLinks
  AllowOverride none
  Order allow,deny
  Allow from all


Alias /weave/1.0 /var/www/weave/weave-synchronisation/1.1/index.php
Alias /weave/1.1 /var/www/weave/weave-synchronisation/1.1/index.php
Alias /weave/1.2 /var/www/weave/weave-synchronisation/1.1/index.php

Alias /weave/user/1.0 /var/www/weave/weave-registration/1.0/index.php
Alias /weave/user/1 /var/www/weave/weave-registration/1.0/index.php
Alias /weave/user /var/www/weave/weave-registration/1.0/index.php
Alias /weave/misc/1.0/captcha_html /var/www/weave/weave-registration/1.0/captcha.php
Alias /weave/misc/1/captcha_html /var/www/weave/weave-registration/1.0/captcha.php

Alias /weave /var/www/weave
# ----- /weave-server ----------------------------------------------------------------

Run

service apache2 reload

to make the changes work.

Client Setup

To use your brand new Weave server in your shiny Firefox 4, choose Set Up Sync... from the Tools menu. There, you will need to create a new account and enter your server address (https://yourdomain.tld/weave/) in the server field after choosing the use of a custom server over Firefox Sync Server.

*[GnuDIP]: GNU Dynamic IP

Introduction

This article briefly describes the setup of a dynamic dns-server, very similar to those commercial ones from dyndns.org and others. It will need access to the domain's zone file on the service provider's nameserver and will offer to manage dyndns-services using own domain names.

A specialty of the approach here is that a subdomain named ddns is added to yourdomain.tld. This results in dynamic names like home.ddns.yourdomain.tld, so that e.g. www.yourdomain.tld is still being managed by the nameservers of the service provider, as they usually offer a more failsafe service level.

Prerequisites

This article assumes a running web-server for the GnuDIP's webfrontend as well as a mysql-server of version > 5. This documentation won't explain their setup in further detail.

The following Ubuntu-packages need to be installed:

aptitude install bind9 apache2 libapache2-mod-perl2 \
                 mysql-server dnsutils xinetd wget

Initially, the following lines have to be added to the zone file of the domain which supposedly managed by the service provider (not locally):

ddns                     IN NS      ns1.ddns
ns1.ddns                 IN A       {IP_of_the_ddns-server}

GnuDIP setup

This documentation stores the GnuDIP files in /opt/gnudip. Unpacking the GnuDIP sources, the archive actually stores the documentation in its root-folder, which is not very elegant. Therefore, in this article the doc-files go into a subfolder called doc whereas everything else will reside in GnuDIP's root folder /opt/gnudip.

Create a new directory for GnuDIP:

mkdir -p /opt/gnudip

Download the source package:

(cd /opt/gnudip && \
 wget http://gnudip2.sourceforge.net/gnudip-www/src/gnudip-2.3.5.tar.gz)

Unpack the archive in the recently created directory:

(cd /opt/gnudip && \
 tar xzf /opt/gnudip/gnudip-2.3.5.tar.gz -C /opt/gnudip --strip 2)

To me, the directory structure of the source package is a bit weird. That's why I reorder a little:

mkdir /opt/gnudip/doc
tar xzf /opt/gnudip/gnudip-2.3.5.tar.gz -C /opt/gnudip/doc --strip 1
rm -r /opt/gnudip/doc/gnudip

In /opt/gnudip/etc/ the new dns-update-keys have to be created for later use with bind9 nameserver:

(cd /opt/gnudip/etc && \
 dnssec-keygen -r /dev/urandom -a hmac-md5 -b 128 -n HOST gnudip-key)

This results in two files called /opt/gnudip/etc/Kgnudip-key.+157+{random_id}.key and /opt/gnudip/etc/Kgnudip-key.+157+{random_id}.private.

GnuDIP's configuration needs to know about the new private-file, so the following line in /opt/gnudip/etc/gnudip.conf has to be adjusted accordingly:

nsupdate = -k /opt/gnudip/etc/Kgnudip-key.+157+{random_id}.private

Database preparation

Given an adequately configured mysql-server, the file /opt/gnudip/doc/gnudip.mysql needs to be edited in order to become compatible with recent versions (5.*) of mysql-server:

(cd /opt/gnudip/doc && \
 sed -i "s/\(.*\)default '0'\(.*\)auto_increment,/\1\2auto_increment,/" gnudip.mysql)

Now, the password for the GnuDIP database has to be set in /opt/gnudip/doc/gnudip.mysql. The relevant parts are at:

grant select, insert, update, delete on gnudip2.to gnudip           identified by '{your_password}';
grant select, insert, update, delete on gnudip2.to gnudip@localhost identified by '{your_password}';

The same password has to be set in /opt/gnudip/etc/gnudip.conf at the line containing:

gnudipdatabase = {your_password}

Now, the mysql database and tables are created:

(cd /opt/gnudip/doc && mysql -p < gnudip.mysql)

Configuration of bind9 nameserver

The update-keys created earlier in GnuDIP's etc-directory have to introduced to bind9. Therefore, the key portion in /opt/gnudip/etc/Kgnudip-key.+157+{random_id}.key (cryptic string at the end of the file, assuming 8hmy3QYvyfLb8ZB3D5MHsf== here) is extracted and saved as /etc/bind/gnudip-key:

key gnudip-key {
      algorithm hmac-md5;
      // the TSIG key
      secret "8hmy3QYvyfLb8ZB3D5MHsf==";
    };

A new zone-file for the dynamic domain has be created at /etc/bind/db.ddns{yourdomain.tld}. Beware not to mess with the dots:

$TTL 86400 ; default TTL (1 day)
@          IN SOA  ns1.ddns.{yourdomain.tld}. root.{yourdomain.tld}. (
                          0       ; serial
                          3600    ; refresh (1 hour)
                          1800    ; retry (30 minutes)
                          604800  ; expire (1 week)
                          0       ; TTL for NACK-s (0 seconds)
                        )
               IN NS    ns1.ddns.{yourdomain.tld}.
ns1            IN A     {your_ip_address}

Now, the key-file and the zone-file have to be introduced to bind9. Therefore the following lines have to be added to the end of /etc/bind/named.conf:

// include definition of GnuDIP update key
include "/etc/bind/gnudip-key";

// define GnuDIP dynamic DNS zone
zone "ddns.{yourdomain.tld}" in {
      type master;
      file "/etc/bind/db.ddns.{yourdomain.tld}";
      allow-query { any; };
      update-policy { grant gnudip-key subdomain ddns.{yourdomain.tld}; };
      };

In order to create new journal-files for the dynamic domain, bind9 will need write access to its configuration directory:

chown bind /etc/bind

The nameserver now needs a restart:

service bind9 restart

A short test is useful in order to proof the correct implementation until now. First, the connection with the new ddns-nameserver is tested:

$ dig test.ddns.{yourdomain.tld}

; <<>> DiG 9.6.0-APPLE-P2 <<>> test.ddns.{yourdomain.tld}
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 58181
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;test.ddns.{yourdomain.tld}.        IN  A

;; AUTHORITY SECTION:
ddns.{yourdomain.tld}.  0   IN  SOA ns1.ddns.{yourdomain.tld}. root.{yourdomain.tld}. 19 3600 1800 604800 0

;; Query time: 11 msec
;; SERVER: 10.10.10.254#53(10.10.10.254)
;; WHEN: Sun Mar  6 16:38:22 2011
;; MSG SIZE  rcvd: 85

Now, the update function is tested:

$ /usr/bin/nsupdate -v -k /opt/gnudip/etc/Kgnudip-key.+157+{random_id}.private
> update add test.ddns.{yourdomain.tld}. 0 A 127.0.0.1
>
> $

Running the first command again should now resolve test.ddns.{yourdomain.tld} to 127.0.0.1.

Configuration of Apache webserver

In this scenario, Apache 2 is used to serve the webfrontend for the GnuDIP admin as well as for its users. Also, access to web-application is supposed to run over a ssl-secured connection.

Activate ssl in Ubuntu's Apache (load module, activate ssl, restart daemon):

a2enmod ssl
a2ensite default-ssl
service apache2 restart

The following lines have to be added to /etc/apache2/sites-enabled/default-ssl:

RedirectMatch ^/gnudip(\/*)$ https://ns1.ddns.{yourdomain.tld}/gnudip/cgi-bin/gnudip.cgi
Alias /gnudip/html/ /opt/gnudip/html/
<Location /gnudip/html/>
    Options Indexes
    ReadmeName .README
    HeaderName .HEADER
    RemoveHandler .pl
    RemoveType .pl
    AddType text/plain .pl
</Location>
ScriptAlias /gnudip/cgi-bin/ /opt/gnudip/cgi-bin/

Apache has to have access to the files above:

chown -R www-data:www-data /opt/gnudip

The initial admin-user is created using the following script:

/opt/gnudip/sbin/gdipadmin.pl -u {admin_name} {admin_password}

This user can now connect to GnuDIP's webfrontend available at:

https://ns1.ddns.{yourdomain.tld}/gnudip

Certainly, it is useful to reconfigure ns1.ddns.{yourdomain.tld} to something more memorable. The configuration options there are self explanatory and are not further documented at this point.

Apache needs a restart:

service apache2 restart

Configuration of xinetd

To be able to use update clients like e.g. ez-ipupdate a GnuDIP server on port 3495/tcp has be be set up. This is done by following these steps:

Introduction of port 3495 to the system is done by adding this line to /etc/services:

gnudip          3495/tcp

The file /etc/xinetd.d/gnudip has to be created with this content:

service gnudip
    {
            flags       = REUSE
            socket_type = stream
            protocol    = tcp
            wait        = no
            user        = www-data
            server      = /opt/gnudip/sbin/gdipinet.pl
            bind        = 0.0.0.0
    }

xinetd needs a restart now:

service xinetd restart

Client configuration

A valid update by ez-ipupdate may use the following command where [dyndns_name == username]:

ez-ipupdate -i {external_interface} -h {dyndns_name}.ddns.{yourdomain.tld} -S gnudip \
-u {username}:{password} -s {address_of_ddns-server}

Aside from the aforementioned client software GnuDIP comes with command line-based clients of its own in /opt/gnudip/client.

An elegant way of updating the dynamic dns entry is GnuDIP's "Auto URL"-function which offers a web-based update-method with an auth-cookie. Using this, a bookmark in a webbrowser can offer a shortcut to a quick IP-update. Also, commandline based browsers like w3m or elinks can be used to automatically update IPs where no ip-update-client is available.