james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joachim Draeger ...@joachim-draeger.de>
Subject IMAP Draft: Cluster
Date Tue, 04 Jul 2006 14:30:18 GMT


Imap is quite resource intensive. This requires to share the load
between multiple hosts. Another goal is being fault tolerant and to have
a fail-over strategy. This means not having 99.999% availability. This
could be done using a cluster capable RDBMS. At the best this would mean
that you can setup your IMAP servers like accessing one single database.

Partition of Mailboxes

The first option is to distribute the mailboxes among different
repositories. E.g. one DB host for user mailboxes, one for shared
mailboxes and one for news. You could even put the user mailboxes on
different hosts.

One limitation this introduces it that quotas cannot span multiple
repositories. There are performance issues when copying/moving between
different repositories.

Although the setup is quite simple it will always take some time to
administer when e.g. migrating mailboxes from one repository to the

Multiple IMAP Servers

Setting up multiple IMAP severs should also not be too complicated when
using standard RDBMS as backends. Requests could be shared between the
servers randomly using DNS or other techniques that involve load

The only thing the repository implementation on the IMAP server has to
provide is caching uids, flags, and message numbers for the session
lifetime and delivering events when it detects changes on the database.
The client should cope with a failing message retrieval in the case it
has been deleted by another session.

If the mailboxes are distributed in a sufficient way, multiple IMAP
servers accessing multiple RDBMS could be able to deal with a high
volume of traffic and a high volume of stored messages


RDBMS like MySQL support master/slave replication. When the master fails
a slave starts to do his job. IMAP servers could be externally
redirected to the new master by e.g. dynamic DNS. But maybe we should
add a possibility to allow changing target server for a repository

An unanswered question to me is the following situation: The master
crashes but its hard-disks are okay and the slave is a few seconds late
with receiving log data. If I would change the slave to master now, I
would lost statements that are already written to the masters

Write To Master, Read From Slave

This is suggested in the MySQL replication FAQ. An unanswered question
for now is: The slave maybe a few seconds late. When I write to the
master and afterwards read from the slave I may not find what I've just
written there. That would be quite confusing for the client. First it
would require checking how late the slave is. If it is more than x
seconds late I should read from the master directly. Second I would have
to cache what I've written to the master to be able to give the right
responses to the client.

A realistic and not too complicated approach would be the following.
Fortunately IMAP does not allow to change message or header content.
When I want to edit a message e.g. in the Drafts folder, it would be
deleted first and gets a new uid. So it would be no problem to list
messages and retrieve flags from the master and load message and header
content from the slave(s). If retrieving the message from the slave
fails, because it its out of sync (quite unlikely) the IMAP server could
fall back reading it from the master.

Because listing messages and retrieving flags is quite inexpensive and
retrieving message content could be quite expensive, this solution could
scale very well. 

To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

View raw message