/usr/share/doc/python-pylibmc-doc/html/behaviors.html is in python-pylibmc-doc 1.5.0-4build1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Behaviors — pylibmc 1.5.0 documentation</title>
<link rel="stylesheet" href="_static/main.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '1.5.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="top" title="pylibmc 1.5.0 documentation" href="index.html" />
<link rel="next" title="Miscellaneous information" href="misc.html" />
<link rel="prev" title="Pooling" href="pooling.html" />
</head>
<body role="document">
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="misc.html" title="Miscellaneous information"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="pooling.html" title="Pooling"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">pylibmc 1.5.0 documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="body" role="main">
<div class="section" id="behaviors">
<span id="id1"></span><h1>Behaviors<a class="headerlink" href="#behaviors" title="Permalink to this headline">¶</a></h1>
<p>libmemcached is a lot more flexible than python-memcached, and has provisions
for configuring so-called <em>behaviors</em>. <code class="xref py py-mod docutils literal"><span class="pre">pylibmc</span></code> wraps these in a Python
interface.</p>
<p>Not all of the available behaviors make sense for Python, or are hard to make
use of, and as such some behaviors have been intentionally hidden or exposed in
some other way (UDP and the binary protocol are examples of this.)</p>
<p>Generally, a behavior’s value should be an integer value. The exceptions are
hashing and distribution, which <code class="xref py py-mod docutils literal"><span class="pre">pylibmc</span></code> translates with the C constants’
string equivalents, for readability.</p>
<p>Other than that, the behaviors are more or less one to one mappings of
libmemcached behavior constants.</p>
<dl class="docutils" id="hash">
<dt><code class="docutils literal"><span class="pre">"hash"</span></code></dt>
<dd>Specifies the default hashing algorithm for keys. See <a class="reference internal" href="#hashing">Hashing</a> for more
information and possible values.</dd>
</dl>
<dl class="docutils" id="distribution">
<dt><code class="docutils literal"><span class="pre">"distribution"</span></code></dt>
<dd>Specifies different means of distributing values to servers. See
<a class="reference internal" href="#distribution">Distribution</a> for more information and possible values.</dd>
</dl>
<dl class="docutils" id="ketama">
<dt><code class="docutils literal"><span class="pre">"ketama"</span></code></dt>
<dd>Setting this behavior to <code class="docutils literal"><span class="pre">True</span></code> is a shortcut for setting <code class="docutils literal"><span class="pre">"hash"</span></code> to
<code class="docutils literal"><span class="pre">"md5"</span></code> and <code class="docutils literal"><span class="pre">"distribution"</span></code> to <code class="docutils literal"><span class="pre">"consistent</span> <span class="pre">ketama"</span></code>.</dd>
</dl>
<dl class="docutils" id="ketama-weighted">
<dt><code class="docutils literal"><span class="pre">"ketama_weighted"</span></code></dt>
<dd>Exactly like the <code class="docutils literal"><span class="pre">"ketama"</span></code> behavior, but also enables the weighting
support.</dd>
</dl>
<dl class="docutils" id="ketama-hash">
<dt><code class="docutils literal"><span class="pre">"ketama_hash"</span></code></dt>
<dd>Sets the hashing algorithm for host mapping on continuum. Possible values
include those for the <code class="docutils literal"><span class="pre">"hash"</span></code> behavior.</dd>
</dl>
<dl class="docutils" id="buffer-requests">
<dt><code class="docutils literal"><span class="pre">"buffer_requests"</span></code></dt>
<dd>Enabling buffered I/O causes commands to “buffer” instead of being sent. Any
action that gets data causes this buffer to be be sent to the remote
connection. Quiting the connection or closing down the connection will also
cause the buffered data to be pushed to the remote connection.</dd>
</dl>
<dl class="docutils" id="no-block">
<dt><code class="docutils literal"><span class="pre">"no_block"</span></code></dt>
<dd>Enables asychronous I/O. This is the fastest transport available for storage
functions.</dd>
</dl>
<dl class="docutils" id="tcp-nodelay">
<dt><code class="docutils literal"><span class="pre">"tcp_nodelay"</span></code></dt>
<dd>Setting this behavior will enable the <code class="docutils literal"><span class="pre">TCP_NODELAY</span></code> socket option, which
disables Nagle’s algorithm. This obviously only makes sense for TCP
connections.</dd>
</dl>
<dl class="docutils" id="cas">
<dt><code class="docutils literal"><span class="pre">"cas"</span></code></dt>
<dd>Enables support for CAS operations.</dd>
</dl>
<dl class="docutils" id="verify-keys">
<dt><code class="docutils literal"><span class="pre">"verify_keys"</span></code></dt>
<dd>Setting this behavior will test if the keys for validity before sending to
memcached.</dd>
</dl>
<dl class="docutils" id="connect-timeout">
<dt><code class="docutils literal"><span class="pre">"connect_timeout"</span></code></dt>
<dd>In non-blocking mode, this specifies the timeout of socket connection
in milliseconds.</dd>
</dl>
<dl class="docutils" id="receive-timeout">
<dt><code class="docutils literal"><span class="pre">"receive_timeout"</span></code></dt>
<dd>“This sets the microsecond behavior of the socket against the SO_RCVTIMEO
flag. In cases where you cannot use non-blocking IO this will allow you to
still have timeouts on the reading of data.”</dd>
</dl>
<dl class="docutils" id="send-timeout">
<dt><code class="docutils literal"><span class="pre">"send_timeout"</span></code></dt>
<dd>“This sets the microsecond behavior of the socket against the SO_SNDTIMEO
flag. In cases where you cannot use non-blocking IO this will allow you to
still have timeouts on the sending of data.”</dd>
</dl>
<dl class="docutils" id="num-replicas">
<dt><code class="docutils literal"><span class="pre">"num_replicas"</span></code></dt>
<dd><p class="first">Poor man’s high-availability solution. Specifies numbers of replicas that
should be made for a given item, on different servers.</p>
<p class="last">“[Replication] does not dedicate certain memcached servers to store the
replicas in, but instead it will store the replicas together with all of the
other objects (on the ‘n’ next servers specified in your server list).”</p>
</dd>
</dl>
<dl class="docutils" id="dead-timeout">
<dt><code class="docutils literal"><span class="pre">"dead_timeout"</span></code></dt>
<dd>Once a server has been marked dead, wait this amount of time (in seconds)
before checking to see if the server is alive again.</dd>
</dl>
<dl class="docutils" id="remove-failed">
<dt><code class="docutils literal"><span class="pre">"remove_failed"</span></code></dt>
<dd>Remove a server from the server list after operations on it have failed for
the specified number of times in a row. See the section on <a class="reference internal" href="#failover">Failover</a>.</dd>
</dl>
<dl class="docutils" id="failure-limit">
<dt><code class="docutils literal"><span class="pre">"failure_limit"</span></code> <span class="classifier-delimiter">:</span> <span class="classifier">deprecated</span></dt>
<dd><p class="first">Use <code class="docutils literal"><span class="pre">"remove_failed"</span></code> if at all possible.</p>
<p class="last">Remove a server from the server list after operations on it have failed for
the specified number of times in a row.</p>
</dd>
</dl>
<dl class="docutils" id="auto-eject">
<dt><code class="docutils literal"><span class="pre">"auto_eject"</span></code> <span class="classifier-delimiter">:</span> <span class="classifier">deprecated</span></dt>
<dd><p class="first">Use <code class="docutils literal"><span class="pre">"remove_failed"</span></code> if at all possible.</p>
<p class="last">With this behavior set, hosts which have been disabled will be removed from
the list of servers after <code class="docutils literal"><span class="pre">"failure_limit"</span></code>.</p>
</dd>
</dl>
<div class="section" id="hashing">
<h2>Hashing<a class="headerlink" href="#hashing" title="Permalink to this headline">¶</a></h2>
<p>Basically, the hasher decides how a key is mapped to a specific memcached
server.</p>
<p>The available hashers are:</p>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">"default"</span></code> - libmemcached’s home-grown hasher</li>
<li><code class="docutils literal"><span class="pre">"md5"</span></code> - MD5</li>
<li><code class="docutils literal"><span class="pre">"crc"</span></code> - CRC32</li>
<li><code class="docutils literal"><span class="pre">"fnv1_64"</span></code> - 64-bit <a class="reference external" href="http://en.wikipedia.org/wiki/Fowler_Noll_Vo_hash">FNV-1</a></li>
<li><code class="docutils literal"><span class="pre">"fnv1a_64"</span></code> - 64-bit FNV-1a</li>
<li><code class="docutils literal"><span class="pre">"fnv1_32"</span></code> - 32-bit FNV-1</li>
<li><code class="docutils literal"><span class="pre">"fnv1a_32"</span></code> - 32-bit FNV-1a</li>
<li><code class="docutils literal"><span class="pre">"murmur"</span></code> - <a class="reference external" href="http://en.wikipedia.org/wiki/MurmurHash">MurmurHash</a></li>
</ul>
<p>If <code class="xref py py-mod docutils literal"><span class="pre">pylibmc</span></code> was built against a libmemcached using
<code class="docutils literal"><span class="pre">--enable-hash_hsieh</span></code>, you can also use <code class="docutils literal"><span class="pre">"hsieh"</span></code>.</p>
<div class="section" id="hashing-and-python-memcached">
<span id="id2"></span><h3>Hashing and python-memcached<a class="headerlink" href="#hashing-and-python-memcached" title="Permalink to this headline">¶</a></h3>
<p>python-memcached up until version 1.45 used a CRC32-based hashing algorithm not
reproducible by libmemcached. You can change the hasher for python-memcached
using the <a class="reference external" href="http://pypi.python.org/pypi/cmemcache_hash">cmemcache_hash</a> module, which will make it not only compatible with
<a class="reference external" href="http://gijsbert.org/cmemcache/">cmemcache</a>, but also the <code class="docutils literal"><span class="pre">"crc"</span></code> hasher in libmemcached.</p>
<p>python-memcached 1.45 and later incorporated <code class="docutils literal"><span class="pre">cmemcache_hash</span></code> as its default
hasher, and so will interoperate with libmemcached provided the libmemcached
clients are told to use the CRC32-style hasher. This can be done in
<code class="xref py py-mod docutils literal"><span class="pre">pylibmc</span></code> as follows:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">mc</span><span class="o">.</span><span class="n">behaviors</span><span class="p">[</span><span class="s">"hash"</span><span class="p">]</span> <span class="o">=</span> <span class="s">"crc"</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="id3">
<h2>Distribution<a class="headerlink" href="#id3" title="Permalink to this headline">¶</a></h2>
<p>When using multiple servers, there are a few takes on how to choose a server
from the set of specified servers.</p>
<p>The default method is <code class="docutils literal"><span class="pre">"modula"</span></code>, which is what most implementations use.
You can enable consistent hashing by setting distribution to <code class="docutils literal"><span class="pre">"consistent"</span></code>.</p>
<p>Modula-based distribution is very simple. It works by taking the hash value,
modulo the length of the server list. For example, consider the key <code class="docutils literal"><span class="pre">"foo"</span></code>
under the <code class="docutils literal"><span class="pre">"crc"</span></code> hasher:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">servers</span> <span class="o">=</span> <span class="p">[</span><span class="s">"a"</span><span class="p">,</span> <span class="s">"b"</span><span class="p">,</span> <span class="s">"c"</span><span class="p">]</span>
<span class="gp">>>> </span><span class="n">crc32_hash</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
<span class="go">3187</span>
<span class="gp">>>> </span><span class="mi">3187</span> <span class="o">%</span> <span class="nb">len</span><span class="p">(</span><span class="n">servers</span><span class="p">)</span>
<span class="go">1</span>
<span class="gp">>>> </span><span class="n">servers</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="go">'b'</span>
</pre></div>
</div>
<p>However, if one was to add a server or remove a server, every key would be
displaced by one - in effect, changing your server list would more or less
reset the cache.</p>
<p>Consistent hashing solves this at the price of a more costly key-to-server
lookup function, <a class="reference external" href="http://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-_a_consistent_hashing_algo_for_memcache_clients">last.fm’s RJ explains how it works</a>.</p>
</div>
<div class="section" id="failover">
<h2>Failover<a class="headerlink" href="#failover" title="Permalink to this headline">¶</a></h2>
<p>Most people desire the classical “I don’t really care” type of failover
support: if a server goes down, just use another one. This case is supported,
but not by default. As explained above, the default distribution mechanism is
not very smart, and libmemcached doesn’t support any meaningful failover for
it. If a server goes down, it stays down, and all of its alloted keys will
simply fail. The recommended failover behaviors is for that reason:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">mc</span><span class="o">.</span><span class="n">behaviors</span><span class="p">[</span><span class="s">'ketama'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">True</span>
<span class="n">mc</span><span class="o">.</span><span class="n">behaviors</span><span class="p">[</span><span class="s">'remove_failed'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">mc</span><span class="o">.</span><span class="n">behaviors</span><span class="p">[</span><span class="s">'retry_timeout'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">mc</span><span class="o">.</span><span class="n">behaviors</span><span class="p">[</span><span class="s">'dead_timeout'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">60</span>
</pre></div>
</div>
<p>This will enable ketama hashing, and remove failed servers from rotation on
their first failure, and retry them once every minute. It is the most robust
configuration.</p>
<p>To fully understand the failover state machine, peruse the following graph:</p>
<img src="failover.svg" /><p>While it might seem daunting at first, a closer examination will bring clarity
to this picture. When a server connection fails, the server is marked as
temporarily failed. This state is exited either by <code class="docutils literal"><span class="pre">retry_timeout</span></code> expiring,
in which case the connection is retried, or, if <code class="docutils literal"><span class="pre">remove_failed</span></code> connection
attempts have been made.</p>
<p>When a server runs out of retries, it is marked dead. This removes it from
rotation. However, <strong>only</strong> the <code class="docutils literal"><span class="pre">ketama</span></code> distribution actually removes
servers.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">There used to be two behaviors called <code class="docutils literal"><span class="pre">failure_limit</span></code> and
<code class="docutils literal"><span class="pre">auto_eject</span></code>; these still exist, but their interaction with the
state machine is unclear, and should be avoided. <code class="docutils literal"><span class="pre">remove_failed</span></code>
acts as a combination of the two.</p>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div><!-- I hate crud -->
<div class="footer" role="contentinfo">
© Copyright 2011, Ludvig Ericson.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.4.
</div>
</body>
</html>
|