Run Cyrus IMAPD mailserver from a Docker container with the mailbox data on an external volume

The Objective:

Run an Imap server on the local Linux machine in a way that it is easy to move from one computer to the next.

The use case: I want to run the imap server on my desktop most of the time but when I go away I want to take it along with me on my laptop.

The first solution was to set up a headless virtual machine with VirtualBox and install the cyrus imap server into the virtual machine. This works and the .vdi disk images can be moved from one computer to the next. The downside is that the disk image is around 19GB in size which takes hours to copy. Also running the VM on the Laptop reduces memory and degrades performance.

The Docker solution:

Build a docker container from the latest OpenSuse image and install cyrus imapd into it. Since containers "forget" all changes when the are shut down we use a VOLUME to persist the database and mail data on the host filesystem. The host directory with the cyrus data can then by rsynced to the new machine and the container can be started there. The mail client finds the impad server on localhost:143.

The Dockerfile:

FROM opensuse:42.1

ENV mailboxuser richi
ENV mailboxpassword password

MAINTAINER Richard Eigenmann 

USER root

# add the packages needed for the cyrus server and some to work with the shell
RUN zypper --non-interactive in \
  cyrus-imapd \
  cyradm \
  cyrus-sasl-saslauthd \
  cyrus-sasl-digestmd5 \
  cyrus-sasl-crammd5 \
  sudo less \
  telnet;

# set up the saslauthd accounts (complication: the host name changes all the time!)
# -u cyrus ensures the account is set up for the hostname cyrus
# cyrus is the account we need to run the cyradm commands
RUN echo ${mailboxpassword} | saslpasswd2 -p -u cyrus -c ${mailboxuser}
RUN echo "password" | saslpasswd2 -p -u cyrus -c cyrus
RUN chgrp mail /etc/sasldb2
RUN chsh -s /bin/bash cyrus


# Set up the mailboxes by starting the cyrus imap daemon, calling up cyradm
# and running the create mailbox commands.

# Step 1: set up a sasl password valid under the build hostname (no -u param).
# Since sasl cares about the hostname the validation doesn't work on the above
# passwords with the -u cyrus hostname.

RUN echo "password" | saslpasswd2 -p -c cyrus

# Step 2: We can't use here-documents in docker so we create the instructions
# that cyradm needs to execute in a text file

RUN echo -e "createmailbox user.${mailboxuser}\ncreatemailbox user.${mailboxuser}.Archive\nexit" > /createmailbox.commands

# Step 3: Start the daemon and in the same build container run the cyradm command
# (note the ; \  at the end of the line!)

RUN /sbin/startproc -p /var/run/cyrus-master.pid /usr/lib/cyrus/bin/master -d; \
sudo -u cyrus -i cyradm --user cyrus -w password localhost < /createmailbox.commands; \
mv /createmailbox.commands /createmailbox.commands.completed;


# create a file startup.sh in the root directory
RUN echo -e "#!/bin/bash\n"\
"if [ -e /var/dostart.semaphore ]; then\n"\
"chown -R cyrus:mail /var/spool/imap /var/lib/imap\n"\
"/usr/lib/cyrus/bin/master -d\n"\
"sleep .6\n"\
"ps u --user cyrus\n"\
"fi"\
> /startup.sh; \
chmod +x /startup.sh


# start the cyrus server and a shell
CMD  /startup.sh; /bin/bash

Running the server:

Build the container:

docker build -t richi/cyrus-docker:latest .

Do these steps to set up the mail server and the host directory: Build the container:

# on the host server
mkdir /absolute/path/to/the/exported/directory/var
docker run -it --rm --hostname cyrus -v /absolute/path/to/the/exported/directory/var:/mnt richi/cyrus-docker:latest

# inside the container 
cp -r /var/* /mnt
touch /mnt/dostart.semaphore

All subsequent runs:

docker run -it --rm --hostname cyrus -p 143:143 -v /absolute/path/to/the/exported/directory/var:/var --log-driver=journald richi/cyrus-docker:latest

Testing:

telnet localhost 143

#should result in output like this:

Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE LOGINDISABLED AUTH=DIGEST-MD5 AUTH=CRAM-MD5 SASL-IR] cyrus Cyrus IMAP v2.4.18 server ready

Discussion:

Setting up the basic container and adding the cyrus software is straight forward.

Setting up the mailbox user account with the password and creating the mailbox structure is tricky: cyrus uses saslauthd to check the passwords of the users logging in. Saslauthd has some sort of anti-tamper mechanism that leverages the hostname in the validation. Since the Docker build process changes the hostname at every step this gets problematic. The saslpasswd2 -u cyrus statements set the passwords for the user account and the cyrus admin account for the hostname cyrus (the -u).

To set up a mailbox account cyrus requires the daemon to be running. The user cyrus then needs to run the cyradm command with the instructions to create the mailbox. Here documents don't seem to be supported inside Dockerfiles so we first create a script file "createmailbox.commands". We then use sudo to promote to the cyrus account and then pipe in the instructions from the script file.

This creates a Docker container that can start up and knows the user, his password and has the basic mailbox structure. You can point your mail client at this imap server and things will work fine until you restart the container. The container will forget all changes when it is shut down. Since cyrus impad stores all state in the /var directory a solution is to export the var directory to the host filesystem so that it can be easily transported to other computers as well as backed up. The -v parameter in the docker run command does just this.

The syntax of the -v parameter is the absolute (!) path of the directory on the left gets mounted to the directory on the right of the colon. Annoyingly, if you just use the -v bind-mount parameter the previous contents of the /var directory in the container are hidden and you just see the empty /var directory from the host filesystem. There doesn't appear to be a way to bind-mount the host directory so that all the obfuscated directories and files from the container "shine through" and all new writes go to the bind-mounted directory.

Therefore we must copy all the content in the container's /var to the host directory first. The way I suggest doing this is to start the container and bind-mound the host's directory to /mnt in the container. Then a cp -r can copy all content from /var to the new directory. After shutting down the container and starting it up with the directory mounted to /var we are back to the original view.

But not quite: The important directories for cyrus, /var/lib/imap and /var/spool/imap, used to be owned by cyrus:mail but are owned by root after the volume mount. Since the server feels it can't read the mailbox database if it is root owned we need to correct this before the startup. I have thus created a startup.sh script that fixed the ownership of the mounted host directory and then starts the daemon. To keep everything in one Dockerfile I create the startup stript with an echo statement right inside the Dockerfile.

To facilitate rsyncing from one host to the other I suggest chown -R user:users on the host directory. Docker runs as root and will create all new files as root owned files but can perfectly well read and write to user owned files. Userspace synchronisation tools will find it much easier to deal with user owned files, however.