/usr/share/doc/python-structlog-doc/html/logging-best-practices.html is in python-structlog-doc 15.2.0-1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Logging Best Practices — structlog documentation</title>
<link rel="stylesheet" href="_static/default.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="top" title="structlog documentation" href="index.html" />
<link rel="next" title="Custom Wrappers" href="custom-wrappers.html" />
<link rel="prev" title="Twisted" href="twisted.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="custom-wrappers.html" title="Custom Wrappers"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="twisted.html" title="Twisted"
accesskey="P">previous</a> |</li>
<li><a href="index.html">structlog documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="logging-best-practices">
<h1>Logging Best Practices<a class="headerlink" href="#logging-best-practices" title="Permalink to this headline">¶</a></h1>
<p>The best practice for you depends very much on your context.
To give you some pointers nevertheless, here are a few scenarios that may be applicable to you.</p>
<p>Pull requests for further interesting approaches as well as refinements and more complete examples are very welcome.</p>
<div class="section" id="common-ideas">
<h2>Common Ideas<a class="headerlink" href="#common-ideas" title="Permalink to this headline">¶</a></h2>
<p>Logging is not a new concept and in no way special to Python.
Logfiles have existed for decades and there’s little reason to reinvent the wheel in our little world.</p>
<p>There are several concepts that are very well-solved in general and especially in heterogeneous environments, using special tooling for Python applications does more harm than good and makes the operations staff build dart board with your pictures.</p>
<p>Therefore let’s rely on proven tools as much as possible and do only the absolutely necessary inside of Python<a class="footnote-reference" href="#id2" id="id1">[*]</a>.
A very nice approach is to simply log to <a class="reference external" href="http://en.wikipedia.org/wiki/Standard_out#Standard_output_.28stdout.29">standard out</a> and let other tools take care of the rest.</p>
<div class="section" id="runit">
<h3>runit<a class="headerlink" href="#runit" title="Permalink to this headline">¶</a></h3>
<p>One runner that makes this very easy is the venerable <a class="reference external" href="http://smarden.org/runit/">runit</a> project which made it a part of its design: server processes don’t detach but log to standard out instead.
There it gets processed by other software – potentially by one of its own tools: <a class="reference external" href="http://smarden.org/runit/svlogd.8.html">svlogd</a>.
We use it extensively and it has proven itself extremely robust and capable; check out <a class="reference external" href="https://rubyists.github.io/2011/05/02/runit-for-ruby-and-everything-else.html">this tutorial</a> if you’d like to try it.</p>
<p>If you’re not quite convinced and want an overview on running daemons, have a look at cue’s <a class="reference external" href="https://web.archive.org/web/20130907200323/http://tech.cueup.com/blog/2013/03/08/running-daemons/">daemon showdown</a> that discusses the most common ones.</p>
</div>
</div>
<div class="section" id="local-logging">
<h2>Local Logging<a class="headerlink" href="#local-logging" title="Permalink to this headline">¶</a></h2>
<p>There are basically two common ways to log to local logfiles: writing yourself into files and syslog.</p>
<div class="section" id="syslog">
<h3>Syslog<a class="headerlink" href="#syslog" title="Permalink to this headline">¶</a></h3>
<p>The simplest approach to logging is to forward your entries to the <a class="reference external" href="http://en.wikipedia.org/wiki/Syslogd">syslogd</a>.
Twisted, uwsgi, and runit support it directly.
It will happily add a timestamp and write wherever you tell it in its configuration.
You can also log from multiple processes into a single file and use your system’s <a class="reference external" href="http://manpages.ubuntu.com/manpages/raring/man8/logrotate.8.html">logrotate</a> for log rotation.</p>
<p>The only downside is that syslog has some quirks that show itself under high load like rate limits (<a class="reference external" href="http://blog.abhijeetr.com/2013/01/disable-rate-limiting-in-rsyslog-v5.html">they can be switched off</a>) and lost log entries.</p>
</div>
<div class="section" id="runit-s-svlogd">
<h3>runit’s svlogd<a class="headerlink" href="#runit-s-svlogd" title="Permalink to this headline">¶</a></h3>
<p>If you’ll choose runit for running your daemons, <a class="reference external" href="http://smarden.org/runit/svlogd.8.html">svlogd</a> is a nicer approach.
It receives the log entries via a UNIX pipe and acts on them which includes adding of parse-friendly timestamps in <a class="reference external" href="http://cr.yp.to/daemontools/tai64n.html">tai64n</a> as well as filtering and log rotation.</p>
</div>
</div>
<div class="section" id="centralized-logging">
<h2>Centralized Logging<a class="headerlink" href="#centralized-logging" title="Permalink to this headline">¶</a></h2>
<p>Nowadays you usually don’t want your logfiles in compressed archives distributed over dozens – if not thousands – servers.
You want them at a single location; parsed and easy to query.</p>
<div class="section" id="syslog-again">
<h3>Syslog (Again!)<a class="headerlink" href="#syslog-again" title="Permalink to this headline">¶</a></h3>
<p>The widely deployed syslog implementation <a class="reference external" href="http://www.rsyslog.com">rsyslog</a> supports remote logging out-of-the-box.
Have a look at <a class="reference external" href="http://www.revsys.com/blog/2010/aug/26/centralized-logging-fun-and-profit/">this post</a> by Revolution Systems on the how.</p>
<p>Since syslog is such a widespread solution, there are also ways to use it with basically any centralized product.</p>
</div>
<div class="section" id="logstash-with-logstash-forwarder">
<h3>Logstash with logstash-forwarder<a class="headerlink" href="#logstash-with-logstash-forwarder" title="Permalink to this headline">¶</a></h3>
<p><a class="reference external" href="https://www.elastic.co/products/logstash">Logstash</a> is a great way to parse, save, and search your logs.</p>
<p>The general modus operandi is that you have log shippers that parse your log files and forward the log entries to your Logstash server and store is in <a class="reference external" href="https://www.elastic.co/products/elasticsearch">elasticsearch</a>.
If your log entries consist of a JSON dictionary (and perhaps a <a class="reference external" href="http://cr.yp.to/daemontools/tai64n.html">tai64n</a> timestamp), this is pretty easy and efficient.</p>
<p>If you can’t decide on a log shipper, <a class="reference external" href="https://github.com/elastic/logstash-forwarder">logstash-forwarder</a> (formerly known as Lumberjack) works really well.
When Logstash’s <tt class="docutils literal"><span class="pre">lumberjack</span></tt> input is configured to use <tt class="docutils literal"><span class="pre">codec</span> <span class="pre">=></span> <span class="pre">"json"</span></tt>, having <tt class="docutils literal"><span class="pre">structlog</span></tt> output JSON is all you need.
See the documentation on the <a class="reference internal" href="standard-library.html"><em>Python Standard Library</em></a> for an example configuration.</p>
</div>
<div class="section" id="graylog2">
<h3>Graylog2<a class="headerlink" href="#graylog2" title="Permalink to this headline">¶</a></h3>
<p><a class="reference external" href="http://graylog2.org">Graylog</a> goes one step further.
It not only supports everything those above do (and then some); you can also log directly JSON entries towards it – optionally even through an AMQP server (like <a class="reference external" href="http://www.rabbitmq.com">RabbitMQ</a>) for better reliability.
Additionally, <a class="reference external" href="https://www.graylog.org/resources/gelf-2/">Graylog’s Extended Log Format</a> (GELF) allows for structured data which makes it an obvious choice to use together with structlog.</p>
<table class="docutils footnote" frame="void" id="id2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id1">[*]</a></td><td>This is obviously a privileged UNIX-centric view but even Windows has tools and means for log management although we won’t be able to discuss them here.</td></tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Logging Best Practices</a><ul>
<li><a class="reference internal" href="#common-ideas">Common Ideas</a><ul>
<li><a class="reference internal" href="#runit">runit</a></li>
</ul>
</li>
<li><a class="reference internal" href="#local-logging">Local Logging</a><ul>
<li><a class="reference internal" href="#syslog">Syslog</a></li>
<li><a class="reference internal" href="#runit-s-svlogd">runit’s svlogd</a></li>
</ul>
</li>
<li><a class="reference internal" href="#centralized-logging">Centralized Logging</a><ul>
<li><a class="reference internal" href="#syslog-again">Syslog (Again!)</a></li>
<li><a class="reference internal" href="#logstash-with-logstash-forwarder">Logstash with logstash-forwarder</a></li>
<li><a class="reference internal" href="#graylog2">Graylog2</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="twisted.html"
title="previous chapter">Twisted</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="custom-wrappers.html"
title="next chapter">Custom Wrappers</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/logging-best-practices.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="custom-wrappers.html" title="Custom Wrappers"
>next</a> |</li>
<li class="right" >
<a href="twisted.html" title="Twisted"
>previous</a> |</li>
<li><a href="index.html">structlog documentation</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright 2013-2015, Hynek Schlawack.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.2.3.
</div>
</body>
</html>
|