/usr/share/doc/iptables-optimizer-doc/html/iptables-optimizer.html is in iptables-optimizer-doc 0.9.14-1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>iptables-optimizer - intro — iptables-optimizer 0.9.14 documentation</title>
<link rel="stylesheet" href="_static/classic.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '0.9.14',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="search" title="Search" href="search.html" />
<link rel="top" title="iptables-optimizer 0.9.14 documentation" href="index.html" />
<link rel="next" title="Plausibility" href="plausible.html" />
<link rel="prev" title="Welcome to iptables-optimizer" href="index.html" />
</head>
<body role="document">
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="py-modindex.html" title="Python Module Index"
>modules</a></li>
<li class="right" >
<a href="plausible.html" title="Plausibility"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="index.html" title="Welcome to iptables-optimizer"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">iptables-optimizer 0.9.14 documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="iptables-optimizer-intro">
<h1>iptables-optimizer - intro<a class="headerlink" href="#iptables-optimizer-intro" title="Permalink to this headline">¶</a></h1>
<p>In many SMB environments long term running packet filters are quite
normal. Usually the administrators are often asked to add a rule for
a special purpose, but only accidentally are kept informed about the
end of life for the needs. So the set of rules grows over long time.
Organizational rules may try to minimize this bad behavior, but
nevertheless it is the normal way of doing.</p>
<p>Assume you have a filtering Linux router with some thousands of
iptables rules in its iptables chains. Unfortunately these this will
produce latency for every traversing packet. And of course, this
latency is unwanted behavior. The only useful way of improving is to
reduce the length of the chains. Buts that is not easy if ever
possible. Usually nobody is responsible enough to say, this special
rule is no longer needed. You would like to know somebody for every
rule.</p>
<p>One of the first ideas for an optimization was to use the counters
on each rule to see, if it is needed at all. The set of rules should
run for a month and all those rules showing zero usage could be
deleted. Sounds easy, but this is not as simple doing as said.
Initial step to a solution was to add the beautiful comment module
in every iptables-command and a number from the corresponding source
of the rule. So it was easy to identify the iptables-command within
the source. Nevertheless the finding of useless rules was not done
because some lack of time. The latency grew.</p>
<p>Another idea came up: partitioning of the long chains into parts of same
targets. Within a partition the rules might be sorted on behalf of the
packet counters so that the most often rules are searched first. And all
the unused rules wouldn’t be consulted so much often. Sounds crazy, but
seemed to be <a class="reference external" href="plausible.html">plausible</a>. Tests were done, python was
chosen for the programming part of the job. And a long and stony way
started with the first step.</p>
<div class="section" id="shell-wrapper">
<h2>shell wrapper<a class="headerlink" href="#shell-wrapper" title="Permalink to this headline">¶</a></h2>
<p>As IPv6 is coming soon, a simple modification in the wrappers source led to
ip6tables-optimizer, which behaves exactly the same as iptables-optimizer...
So the shell wrapper exists in two versions now, the second is symlinked to
the first file.</p>
<p>The shell wrapper simply acts in four steps after evaluating the options:</p>
<blockquote>
<div><ol class="arabic simple">
<li>If an executable file <code class="docutils literal"><span class="pre">/var/cache/iptables-optimizer/auto-apply</span></code> exists, restore it into the kernel and rename it</li>
<li>Use <code class="docutils literal"><span class="pre">iptables-save</span> <span class="pre">-t</span> <span class="pre">filter</span> <span class="pre">-c</span></code> to store the kernels tables into a file</li>
<li>Run <code class="docutils literal"><span class="pre">iptables_optimizer.py</span></code>, save stdout and stderr for errorchecking</li>
<li>Use <code class="docutils literal"><span class="pre">iptables-restore</span></code> to push back the rules into the kernel</li>
</ol>
</div></blockquote>
<p>Some error-checking is done, so the code is a little bit more than four lines.
In case of an error, the shell wrapper exits immediately due to runnig with <code class="docutils literal"><span class="pre">bash</span> <span class="pre">-e</span></code>.
The <code class="docutils literal"><span class="pre">pipefail</span></code> option ensures failures to be seen in piped commands as well.
The real tricky things are done in the pythonic part.</p>
<p>Some command line options are provided:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="o">-</span><span class="n">a</span> <span class="n">do</span> <span class="ow">not</span> <span class="n">evaluate</span> <span class="n">auto</span><span class="o">-</span><span class="n">apply</span> <span class="p">(</span><span class="n">auto</span><span class="o">-</span><span class="n">apply6</span><span class="p">)</span>
<span class="o">-</span><span class="n">c</span> <span class="n">keep</span> <span class="n">the</span> <span class="n">packet</span><span class="o">/</span><span class="n">byte</span> <span class="n">counters</span> <span class="n">on</span> <span class="n">restoring</span>
<span class="o">-</span><span class="n">h</span> <span class="n">help</span> <span class="n">message</span> <span class="n">about</span> <span class="n">valid</span> <span class="n">options</span> <span class="ow">and</span> <span class="n">exit</span> <span class="mi">1</span>
<span class="o">-</span><span class="n">v</span> <span class="n">verbose</span> <span class="n">logging</span> <span class="n">about</span> <span class="n">the</span> <span class="n">steps</span><span class="o">.</span> <span class="n">If</span> <span class="n">given</span> <span class="n">twice</span><span class="p">,</span> <span class="n">partitions</span> <span class="ow">and</span> <span class="n">moves</span> <span class="n">are</span> <span class="n">logged</span>
<span class="o">-</span><span class="n">w</span> <span class="n">logging</span> <span class="n">partiontions</span> <span class="ow">and</span> <span class="n">moves</span> <span class="k">for</span> <span class="n">INPUT</span> <span class="ow">and</span> <span class="n">OUTPUT</span> <span class="n">chains</span> <span class="n">only</span><span class="p">,</span> <span class="n">implies</span> <span class="o">-</span><span class="n">vv</span><span class="p">,</span>
</pre></div>
</div>
<p>Usually the iptables-optimizer exits with a return value of zero indicating no error.
The debian packaging produces two files for dpkg:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">iptables</span><span class="o">-</span><span class="n">optimizer_x</span><span class="o">.</span><span class="n">y</span><span class="o">.</span><span class="n">z</span><span class="o">-</span><span class="n">v</span><span class="o">.</span><span class="n">deb</span>
<span class="n">iptables</span><span class="o">-</span><span class="n">optimizer</span><span class="o">-</span><span class="n">doc_x</span><span class="o">.</span><span class="n">y</span><span class="o">.</span><span class="n">z</span><span class="o">-</span><span class="n">v</span><span class="o">.</span><span class="n">deb</span>
</pre></div>
</div>
<p>For production environment the documents are not needed, for your understanding
you do not need the binary package at all.</p>
<p>All used functions within the shell wrapper are sourced from another file,
<code class="docutils literal"><span class="pre">iptables-optimizer-functions</span></code>. This seems to be useful for making them testable
with Karen Wards shunit2, which is available as free software.</p>
</div>
<div class="section" id="python-code">
<h2>python code<a class="headerlink" href="#python-code" title="Permalink to this headline">¶</a></h2>
<p>The only reason to have a shell wrapper for the python script was found in
different python versions, which treated the subprocess module very different
in different Debian stable versions from lenny to jessie.</p>
<p>Python comes with batteries included, they say. The subprocess module
can execute every shell command from within the python code. Sounds well,
worked well until — sometimes you have some different python versions
running because of different operating systems, f.e. in Debian systems
you may find python 2.5, 2.6, 2.7 and 3.2, 3.3, 3.4 and 3.5, just like they
are distributed as standard versions in etch, Lenny, squeeze and wheezy.
Surprisingly the subprocess behavior changed a lot in these. I was very
frustrated about that and therefore decided not to use it. Benefit was
to have a single python script containing the necessary stuff without calling
external commands, but running well in all different python versions.
The external parts were migrated to an external shell script, which
itself calls the python snippet for the complex actions now.</p>
<p>So, what needs to be done? The filter tables are searched for every
traversing packet, all the rules are checked for matching, and if
one matches, its target is applied to the packet and usually the
action for this packet is finished. The less rules must be searched
the quicker the packet is forwarded or dropped. So the rules should
be assorted in a way, more often used ones should be found quicker
than those which are seldom used. But, there is a handicap: Perhaps
the administrator wants some special packets to be dropped and some
other to be forwarded. The old-fashioned handmade rule set respects
this, and the administrators artwork shall never be destroyed by
sorting. Let’s think again, is it possible to sort the rules and to
respect his artwork? Yes, it is possible, but few restrictions apply.</p>
<p>In every chain we have some rules to accept, some to drop and some
others, each intermixed with the others. From a mathematical point
of view (set theory) partitions are the key to solve the puzzle. If
we group consecutive rules having the same targets, inside these groups
we can exchange the rule without changing the policy. Sure.</p>
<p>So we have to find partition borders and then sort within each partition
on behalf of the packet counters.</p>
<p>Two python classes were build: Chain and Filter. An instance of the
Filter class holds at least the predefined chains, perhaps some
user defined chains. On creation of an instance it reads the given
file.</p>
<p>Fortunately the python code is completely independant of the rules
content as it only evaluates the packet/byte counters. So it is used
in the iptables-optimizer as well as in the ip6tables-optimizer.</p>
<div class="section" id="class-filter">
<h3>class Filter<a class="headerlink" href="#class-filter" title="Permalink to this headline">¶</a></h3>
<p>Instanciating the Filter class reads a file, which is an output of
<code class="docutils literal"><span class="pre">iptables-save</span> <span class="pre">-c</span></code>, so we get the chain names at first and then
their content. For each chain an instance of the class Chain is
set up. The packet counters are needed, this is done by the “-c” in the
2nd step of the wrapper. The init method ends up with a full
representation of the kernels filter tables in memory.</p>
<p>The opti method uses the opti method of all chain instances, the show
method is just a wrapper around the many print statements for testing
purposes and for better separating any additional information such
as statistics, which then are printed out to stderr. This kind
separation is fine, especially within the shell wrapper.</p>
</div>
<div class="section" id="class-chain">
<h3>class Chain<a class="headerlink" href="#class-chain" title="Permalink to this headline">¶</a></h3>
<p>On reading the file, an instance of the Chain class is build on the fly.
The appends are done using the corresponding append method, and so
at last a complete picture of the kernels view is in memory. The opti
method on startup uses the make_partitions method prior to the sorting
related to the packet counters.</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">iptables-optimizer - intro</a><ul>
<li><a class="reference internal" href="#shell-wrapper">shell wrapper</a></li>
<li><a class="reference internal" href="#python-code">python code</a><ul>
<li><a class="reference internal" href="#class-filter">class Filter</a></li>
<li><a class="reference internal" href="#class-chain">class Chain</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="index.html"
title="previous chapter">Welcome to iptables-optimizer</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="plausible.html"
title="next chapter">Plausibility</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="py-modindex.html" title="Python Module Index"
>modules</a></li>
<li class="right" >
<a href="plausible.html" title="Plausibility"
>next</a> |</li>
<li class="right" >
<a href="index.html" title="Welcome to iptables-optimizer"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">iptables-optimizer 0.9.14 documentation</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2016, 2015, Johannes Hubertz.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.9.
</div>
</body>
</html>
|