/usr/share/doc/cyrus-doc/html/install-murder.html is in cyrus-doc 2.4.18-3.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 | <!-- $Id: install-murder.html,v 1.23 2010/01/06 17:01:29 murch Exp $ -->
<HTML>
<HEAD>
<TITLE>Installing The Cyrus Murder
</title>
</head>
<h1>Installing The Cyrus Murder
</h1>
<body>
<p>The Cyrus Murder provides the ability to horizontally scale your Cyrus
IMAP environment across multiple servers. It is similar in nature to
various proxy solutions such as nginx or perdition with the difference
being that Murder offers a uniform namespace. If you're not currently
using shared mailboxes and you don't intend to use shared mailboxes in the
future, you should probably just consider using a simple proxy solution.</p>
<br>
<h3>Introduction & Assumptions</h3>
This document is intended to be a guide to the configuration of a Cyrus
IMAP Aggregator, aka Cyrus Murder. It is recommended that you review
<A HREF="ag.html">this document</A> to become familliar
with the concepts at work. This document is a work in progress and is at this
point incomplete.<p>
This document assumes that you have successfully been able to setup atleast
one Cyrus IMAP server. This server will become your first backend server.
It also assumes that you are familliar with the administration and day
to day operations of the Cyrus IMAP server and the SASL authentication library.
If you feel uncomfortable with this, please refer to the rest of the
documentation first.<p>
There is a <A HREF=murder.png>diagram</A> that shows the interactions of
the various components of the Cyrus Murder which may be helpful in
understanding the "big picture".<p>
<h2>Installation</h2>
You will need to build Cyrus IMAPd with the <tt>--enable-murder</tt> configure
option. This builds the proxyds and the associated utilities.
<h3>Requirements</h3>
<ul>
<li>Atleast one Cyrus IMAP server. If there are more than one, their name
spaces must not conflict. That is, all the mailbox names must be unique
(or in different namespaces)</li>
<li>Atleast one machine that will become the first Frontend Server.</li>
<li>One machine to become the MUPDATE master server. This can be the
same as one of your frontend servers.</li>
</ul>
<h3>Configuring the MUPDATE Master</h3>
The mupdate master server needs to be running the mupdate service in master
mode. Note that you can have the MUPDATE master be one of your frontend
machines, just do not configure a slave mupdate process on this machine.<P>
To configure an mupdate master, you will want a cyrus.conf that includes a
line similar to the following in the SERVICES section:<p>
<pre>
mupdate cmd="/usr/cyrus/bin/mupdate -m" listen=3905 prefork=1
</pre>
Note the "-m" option to tell mupdate that it should start in master mode.<p>
You will also need to configure atleast a skeleton imapd.conf that defines
the <tt>configdirectory</tt>, a bogus <tt>partition-default</tt> and the
<tt>admins</tt> that can authenticate to the server. Note that slave
mupdate servers as well as the backend servers will need to be able to
authenticate as admins on the master. Here is a very simple imapd.conf for
a master server:<p>
<pre>
configdirectory: /imap/conf
partition-default: /tmp
admins: mupdateslave1 backend1
</pre>
You will also need to configure SASL to properly allow authentication in
your enviornment.<p>
<h3>Setting up the backends to push changes to the MUPDATE Master</h3>
On the backends, configuration to be a part of a murder is easy. You just
need to configure the backend to be a part of the murder. To do this,
set the <tt>mupdate_server</tt> option in imapd.conf. Depending on what
authentication mechanisms you are using, you may also want to set some
or all of the following:
<ul>
<li><tt>mupdate_username</tt></li>
<li><tt>mupdate_authname</tt></li>
<li><tt>mupdate_realm</tt></li>
<li><tt>mupdate_password</tt></li>
</ul>
Once these settings are successfully made, any mailbox operation on the
backend will be sent to the mupdate master for confirmation and entry into
the mupdate database.<p>
You must also configure atleast one user/group using the
<tt>proxyservers</tt> imapd.conf option. This user <i>should not</i>
be an administrator, since this means that anyone who can get ahold
of your proxy servers now has full administrative control on your
backend. Example:<p>
<pre>
admins: cyrus
proxyservers: murder
</pre>
Keep in mind that you will need to create the proxy user(s) and be sure
that they can authenticate to the backend as well.
NOTE: <tt>proxyservers</tt> should not be set on your frontends. It is
also used to determine which servers to create mailboxes on.
<h3>Importing the database from the backend</h3>
Importing the current mailboxes database is easy, as there is a ctl_mboxlist
option to do so. To do the first synchronization, simply change to the
cyrus user, and issue a <tt>ctl_mboxlist -m</tt>.<p>
<p>Note that you may wish to issue a <tt>ctl_mboxlist -mw</tt> first to
be sure you understand all the operations that this command will perform,
since it does require that all mailboxes are unique in the murder namespace.</p>
If everything is configured properly, the
mailbox database of the current host will dump to the mupdate master. If
there are problems, the most likely cause is a misconfiguration of the
authentication settings, or that mupdate might not be running on the master.
Using <tt>mupdatetest</tt> may be helpful in this case (it establishes
an authenticated connection to the mupdate server, if it can).<p>
It is also useful to have the backends automatically resync the state of
their local mailboxes database with the master on start up. You can
configure this by adding the following to the <tt>START</tt> section
of cyrus.conf on the backends:<p>
<pre>
mupdatepush cmd="ctl_mboxlist -m"
</pre>
This will perform synchronization with the mupdate master each time the backend
restarts, bringing the mupdate database up to date with the contents of the
backend (and performing ACTIVATE and DELETES as needed to do so).<p>
<b>Warning</b>: If somehow a mailbox exists on two (or more) backend servers,
each time one of them synchronizes its database that backend server will
become authoritative. Though this should not happen during normal operation
of the murder (because of the consistancy guarantees of the MUPDATE protocol,
and the fact that mailbox operations are denied if the mupdate master is down),
it <b>is</b> possible when first creating the mupdate database or
when bringing a new backend server into the murder.
<h3>Configuring the frontends</h3>
Configuring the frontends is a two step process. First, you want to set
<tt>mupdate_server</tt> (and friends) as you did for the backends above.
However, because the frontends only talk to the mupdate master via a slave
running on the local machine, you will also need to set up a slave on the
same machine, in the <tt>SERVICES</tt> section of cyrus.conf, like so:<p>
<pre>
# mupdate database service - must prefork atleast 1
mupdate cmd="mupdate" listen=3905 prefork=1
</pre>
Note that as this is a threaded service, you must prefork atleast 1 of them
so that the database can be synchronized at startup. Otherwise, the service
will not start running until after you recieve an mupdate client connection
to the slave (which is not a recommended configuration at this point).<p>
You will also want to change all of your <tt>imapd</tt> entries to be
<tt>proxyd</tt>, and all of your <tt>lmtpd</tt> entries to be
<tt>lmtpproxyd</tt>. That is, you will probably have a <tt>SERVICES</tt>
section that looks more like this now:<p>
<pre>
mupdate cmd="/usr/cyrus/bin/mupdate" listen=3905 prefork=1
imap cmd="proxyd" listen="imap" prefork=5
imaps cmd="proxyd -s" listen="imaps" prefork=1
pop3 cmd="pop3d" listen="pop3" prefork=0
pop3s cmd="pop3d -s" listen="pop3s" prefork=0
kpop cmd="pop3d -k" listen="kpop" prefork=0
nntp cmd="nntpd" listen="nntp" prefork=0
nntps cmd="nntpd -s" listen="nntps" prefork=0
sieve cmd="timsieved" listen="sieve" prefork=0
lmtp cmd="lmtpproxyd" listen="/var/imap/socket/lmtp" prefork=0
</pre>
Note that timsieved does not need a proxy daemon, the managesieve protocol
deals with the murder with referrals to the backends internally.<p>
Additionally, you will need entries in imapd.conf to indicate the proxy
auth name and passwords (if you are using a SASL mechanism that requires them)
to the backends, for example, if your backends are
<tt>mail1.andrew.cmu.edu</tt> and <tt>mail2.andrew.cmu.edu</tt> with passwords
of <tt>foo</tt> and <tt>bar</tt>, and an auth name of <tt>murder</tt>:<p>
<pre>
mail1_password: foo
mail2_password: bar
proxy_authname: murder
</pre>
If your SASL mechanism does not require authnames or passwords (e.g.
KERBEROS_V4), then this is not required. Note that we used the same
authname as the configured in the <tt>proxyservers</tt> line in the backend's
<tt>imapd.conf</tt> above.<p>
When you start master on the frontend, a local mailboxes database should
automatically synchronize itself with the contents of the mupdate master,
and you should be ready to go. Your clients should connect to the frontends,
and the frontends will proxy or refer as applicable to the blackend servers.<p>
<h3>Additional backend configuration</h3>
If your authentication system requires usernames, passwords, etc, to
authenticate (e.g. it isn't Kerberos), then you will also need to
specify proxy_authname (and friends) in the backend imapd.confs as well.
This is so that the backends can authenticate to eachother to facilitate
maibox moves. (Backend machines will need to be full admins).
<h3>Delivering mail</h3>
To deliver mail to your Murder, configure your MTA just as you did
before, but instead of connecting directly to <tt>lmtpd</tt>, it
should connect to <tt>lmtpproxyd</tt>. You can connect to the
<tt>lmtpproxyd</tt> running on the frontend machines, or you can
install <tt>master</tt> and <tt>lmtpproxyd</tt> on your SMTP servers.
<h2>Administration</h2>
<h3>Keeping the database synced</h3>
<p>Consistancy in the database is maintained by pushing the current status
of the backends to the master, and having the frontends stay up to date with
the master's database. Since the frontends resync themselves entirely
when they startup, downtime should not at all be a problem. (While they
are up they should be continously recieving database updates, as well as
when they lose connection to the master, they will try to reconnect and
resync their database upon reconnection)</p>
<p>Provided that the namespace of the backend servers is kept discrete
(with no mailboxes existing on the same server), it is not a big deal
to resync the mupdate master using <tt>ctl_mboxlist -m</tt>. If two servers
do have the same mailbox, this will need to be resolved before database
consistancy can be guranteed.</p>
<h3>Moving Mailboxes between backends</h3>
<p>There is currently no 100% foolproof way to do this, however, if you issue
a rename command to a frontend (as you would to move a mailbox between partitions),
and replace the partition name with the name of the new backend, it will move
the mailbox to the indicated backend. You can also use the format
<tt>backend.domain.com!partition</tt> to move to a specific partition
(otherwise the default partition will be used). In cyradm, this looks
like:<p>
<pre>
cyrus.andrew.cmu.edu> rename user.bcyrus user.bcyrus mail2.andrew.cmu.edu!u2
</pre>
Note that since seen state is stored per-user, it is possible that when
moving a shared mailbox users will have strange effects. The general rule
is that moving an INBOX will move the entire user (including all sub-mailboxes
to the INBOX, and seen state, and subscriptions, and sieve scripts, etc).
The seen state is merged with the seen state on the new backend, so that
no data is lost (seen state is also the only part left behind on the source
backend). In the case of any other mailbox, however, only that individual
mailbox is moved. If it is a quota root, the new quota root is instated
on the new server, but otherwise quotas can appear to be violated,
since each backend only takes care of its own quota.<p>
In general, its better to leave trees of mailboxes on the same server, and
not move submailboxes of inboxes between servers.
</p>
<h3>Adding additional backend servers</h3>
<p>This is very easy to do, simply configure an empty backend server
and set its <tt>mupdate_server</tt> parameter to point at the mupdate
master. Then, issue mailbox creates to it as you would any other
backend server.</p>
<h3>Backups</h3>
<p>xxx, need to write stuff. You don't need to really backup the data on
the mupdate master or slaves, since this data can all be generated directly
from the backends quite easily.</p>
<h3>Gotchyas</h3>
<ul>
<li><b>Clients dealing with a pool of frontend servers</b> - Some clients may
not be terribly efficient caching connections to a pool of imap servers, this
isn't a problem, per se, but it may mean that you will see many more
authentications than you are used to.
<ul><li><b>Kerberos issues</b> - If you are using kerberos authentication,
you will want to ensure that all your machines are keyed properly, as we
have seen problems with different clients trying to authenticate to
different services (e.g. imap.imap-pool instead of imap.pool-frontend-1), so
test the clients in use in your enviornment and be sure that they work with
whatever keying scheme you use.</li></ul>
<li><b>Clients dealing with referrals</b> - Some clients (we've had particuar
trouble with pine, though most of these issues have now been resolved and
new versions should be OK (that is, pine > 4.44), but as referrals have not
been extensively used by any IMAP server until now, referrals are very likely
to not work correctly or have surprising effects.
<li><b>Clients dealing with getting a NO on LSUB commands</b> - Some clients
(Outlook, for example) may behave poorly if an LSUB command returns a NO, which
may be the case if the backend server with the user's inbox is down. We
have, for example, seen this result in the deletion of the disconnected
message cache.
<li><b>Behavior of cyradm / some mailbox operations</b> - The behavior
of some administrative commands
might be slightly unexpected. For example, you can only issue a SETQUOTA
to a frontend server if the entire mailbox tree underneath where you are
setting the quota exists on the same backend server, otherwise you will need
to connect directly to the backend servers to perform the needed changes.
Similarally, mailboxes will be created on the same backend server
that their parent is in. In order to create them on a different server
(or to create a new top level mailbox) you will need to connect directly to
the desired backend server.
<li><b>Subscriptions</b> - If users want subscribe to a mailbox other than on
their backend home server, they won't be able to, unless you set
"<tt>allowallsubscribe: t</tt>" in the backend imapd.confs. This essentially
lets any string be subscribed to successfully.
<li><b>Restarting the mupdate master</b> - Because <tt>ctl_cyrusdb -r</tt>
clears reservations on mailbox, if you restart the mupdate master (and
run recovery), then this could (we suspect, very rarely) lead to inconsistancies
in the mupdate database.
</ul>
<h3>Troubleshooting & when things go wrong</h3>
<ul>
<li><b>Mailbox operations are being denied</b> - This is an indication
that the mupdate master may be down. Restart it.</li>
<li><b>Mailbox operations are not being seen by one or more frontends</b> -
This indicates that the mupdate process on a slave may have died,
you may need to restart master. Alternatively, mupdate will retry connections
every 20 seconds or so for about 20 attempts if the master does go down.</li>
<li><b>A frontend's mailboxes.db is corrupt / out of sync</b> - Restart master
on the frontend, and have the mupdate process resynch the local database.
You may need to remove the local mailboxes database if the corruption is
extreme.</li>
<li><b>A mailbox's location keeps switching between two (or more) backend
hosts.</b> - It probably actually exists on both hosts. Delete the
mailbox from all but one of the hosts, and run a ctl_mboxlist -m on the one
where you want it to actually live.</li>
<li><b>Databases are never created on the frontends/slaves</b> - Check
to ensure that the mupdate slave process is started, (is prefork=1)</li>
</ul>
</BODY></HTML>
|