This file is indexed.

/usr/share/doc/clang-3.9-doc/html/PTHInternals.html is in clang-3.9-doc 1:3.9.1-9.

This file is owned by root:root, with mode 0o644.

The actual contents of the file can be viewed below.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">


<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    
    <title>Pretokenized Headers (PTH) &#8212; Clang 3.9 documentation</title>
    
    <link rel="stylesheet" href="_static/haiku.css" type="text/css" />
    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
    
    <script type="text/javascript">
      var DOCUMENTATION_OPTIONS = {
        URL_ROOT:    './',
        VERSION:     '3.9',
        COLLAPSE_INDEX: false,
        FILE_SUFFIX: '.html',
        HAS_SOURCE:  true
      };
    </script>
    <script type="text/javascript" src="_static/jquery.js"></script>
    <script type="text/javascript" src="_static/underscore.js"></script>
    <script type="text/javascript" src="_static/doctools.js"></script>
    <script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" />
    <link rel="top" title="Clang 3.9 documentation" href="index.html" />
    <link rel="next" title="Precompiled Header and Modules Internals" href="PCHInternals.html" />
    <link rel="prev" title="Driver Design &amp; Internals" href="DriverInternals.html" /> 
  </head>
  <body role="document">
      <div class="header" role="banner"><h1 class="heading"><a href="index.html">
          <span>Clang 3.9 documentation</span></a></h1>
        <h2 class="heading"><span>Pretokenized Headers (PTH)</span></h2>
      </div>
      <div class="topnav" role="navigation" aria-label="top navigation">
      
        <p>
        «&#160;&#160;<a href="DriverInternals.html">Driver Design &amp; Internals</a>
        &#160;&#160;::&#160;&#160;
        <a class="uplink" href="index.html">Contents</a>
        &#160;&#160;::&#160;&#160;
        <a href="PCHInternals.html">Precompiled Header and Modules Internals</a>&#160;&#160;»
        </p>

      </div>
      <div class="content">
        
        
  <div class="section" id="pretokenized-headers-pth">
<h1>Pretokenized Headers (PTH)<a class="headerlink" href="#pretokenized-headers-pth" title="Permalink to this headline"></a></h1>
<p>This document first describes the low-level interface for using PTH and
then briefly elaborates on its design and implementation. If you are
interested in the end-user view, please see the <a class="reference internal" href="UsersManual.html#usersmanual-precompiled-headers"><span class="std std-ref">User&#8217;s Manual</span></a>.</p>
<div class="section" id="using-pretokenized-headers-with-clang-low-level-interface">
<h2>Using Pretokenized Headers with <code class="docutils literal"><span class="pre">clang</span></code> (Low-level Interface)<a class="headerlink" href="#using-pretokenized-headers-with-clang-low-level-interface" title="Permalink to this headline"></a></h2>
<p>The Clang compiler frontend, <code class="docutils literal"><span class="pre">clang</span> <span class="pre">-cc1</span></code>, supports three command line
options for generating and using PTH files.</p>
<p>To generate PTH files using <code class="docutils literal"><span class="pre">clang</span> <span class="pre">-cc1</span></code>, use the option <code class="docutils literal"><span class="pre">-emit-pth</span></code>:</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">$</span> clang -cc1 test.h -emit-pth -o test.h.pth
</pre></div>
</div>
<p>This option is transparently used by <code class="docutils literal"><span class="pre">clang</span></code> when generating PTH
files. Similarly, PTH files can be used as prefix headers using the
<code class="docutils literal"><span class="pre">-include-pth</span></code> option:</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">$</span> clang -cc1 -include-pth test.h.pth test.c -o test.s
</pre></div>
</div>
<p>Alternatively, Clang&#8217;s PTH files can be used as a raw &#8220;token-cache&#8221; (or
&#8220;content&#8221; cache) of the source included by the original header file.
This means that the contents of the PTH file are searched as substitutes
for <em>any</em> source files that are used by <code class="docutils literal"><span class="pre">clang</span> <span class="pre">-cc1</span></code> to process a
source file. This is done by specifying the <code class="docutils literal"><span class="pre">-token-cache</span></code> option:</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">$</span> cat test.h
<span class="gp">#</span>include &lt;stdio.h&gt;
<span class="gp">$</span> clang -cc1 -emit-pth test.h -o test.h.pth
<span class="gp">$</span> cat test.c
<span class="gp">#</span>include <span class="s2">&quot;test.h&quot;</span>
<span class="gp">$</span> clang -cc1 test.c -o <span class="nb">test</span> -token-cache test.h.pth
</pre></div>
</div>
<p>In this example the contents of <code class="docutils literal"><span class="pre">stdio.h</span></code> (and the files it includes)
will be retrieved from <code class="docutils literal"><span class="pre">test.h.pth</span></code>, as the PTH file is being used in
this case as a raw cache of the contents of <code class="docutils literal"><span class="pre">test.h</span></code>. This is a
low-level interface used to both implement the high-level PTH interface
as well as to provide alternative means to use PTH-style caching.</p>
</div>
<div class="section" id="pth-design-and-implementation">
<h2>PTH Design and Implementation<a class="headerlink" href="#pth-design-and-implementation" title="Permalink to this headline"></a></h2>
<p>Unlike GCC&#8217;s precompiled headers, which cache the full ASTs and
preprocessor state of a header file, Clang&#8217;s pretokenized header files
mainly cache the raw lexer <em>tokens</em> that are needed to segment the
stream of characters in a source file into keywords, identifiers, and
operators. Consequently, PTH serves to mainly directly speed up the
lexing and preprocessing of a source file, while parsing and
type-checking must be completely redone every time a PTH file is used.</p>
<div class="section" id="basic-design-tradeoffs">
<h3>Basic Design Tradeoffs<a class="headerlink" href="#basic-design-tradeoffs" title="Permalink to this headline"></a></h3>
<p>In the long term there are plans to provide an alternate PCH
implementation for Clang that also caches the work for parsing and type
checking the contents of header files. The current implementation of PCH
in Clang as pretokenized header files was motivated by the following
factors:</p>
<dl class="docutils">
<dt><strong>Language independence</strong></dt>
<dd>PTH files work with any language that
Clang&#8217;s lexer can handle, including C, Objective-C, and (in the early
stages) C++. This means development on language features at the
parsing level or above (which is basically almost all interesting
pieces) does not require PTH to be modified.</dd>
<dt><strong>Simple design</strong></dt>
<dd>Relatively speaking, PTH has a simple design and
implementation, making it easy to test. Further, because the
machinery for PTH resides at the lower-levels of the Clang library
stack it is fairly straightforward to profile and optimize.</dd>
</dl>
<p>Further, compared to GCC&#8217;s PCH implementation (which is the dominate
precompiled header file implementation that Clang can be directly
compared against) the PTH design in Clang yields several attractive
features:</p>
<dl class="docutils">
<dt><strong>Architecture independence</strong></dt>
<dd><p class="first">In contrast to GCC&#8217;s PCH files (and
those of several other compilers), Clang&#8217;s PTH files are architecture
independent, requiring only a single PTH file when building a
program for multiple architectures.</p>
<p class="last">For example, on Mac OS X one may wish to compile a &#8220;universal binary&#8221;
that runs on PowerPC, 32-bit Intel (i386), and 64-bit Intel
architectures. In contrast, GCC requires a PCH file for each
architecture, as the definitions of types in the AST are
architecture-specific. Since a Clang PTH file essentially represents
a lexical cache of header files, a single PTH file can be safely used
when compiling for multiple architectures. This can also reduce
compile times because only a single PTH file needs to be generated
during a build instead of several.</p>
</dd>
<dt><strong>Reduced memory pressure</strong></dt>
<dd>Similar to GCC, Clang reads PTH files
via the use of memory mapping (i.e., <code class="docutils literal"><span class="pre">mmap</span></code>). Clang, however,
memory maps PTH files as read-only, meaning that multiple invocations
of <code class="docutils literal"><span class="pre">clang</span> <span class="pre">-cc1</span></code> can share the same pages in memory from a
memory-mapped PTH file. In comparison, GCC also memory maps its PCH
files but also modifies those pages in memory, incurring the
copy-on-write costs. The read-only nature of PTH can greatly reduce
memory pressure for builds involving multiple cores, thus improving
overall scalability.</dd>
<dt><strong>Fast generation</strong></dt>
<dd>PTH files can be generated in a small fraction
of the time needed to generate GCC&#8217;s PCH files. Since PTH/PCH
generation is a serial operation that typically blocks progress
during a build, faster generation time leads to improved processor
utilization with parallel builds on multicore machines.</dd>
</dl>
<p>Despite these strengths, PTH&#8217;s simple design suffers some algorithmic
handicaps compared to other PCH strategies such as those used by GCC.
While PTH can greatly speed up the processing time of a header file, the
amount of work required to process a header file is still roughly linear
in the size of the header file. In contrast, the amount of work done by
GCC to process a precompiled header is (theoretically) constant (the
ASTs for the header are literally memory mapped into the compiler). This
means that only the pieces of the header file that are referenced by the
source file including the header are the only ones the compiler needs to
process during actual compilation. While GCC&#8217;s particular implementation
of PCH mitigates some of these algorithmic strengths via the use of
copy-on-write pages, the approach itself can fundamentally dominate at
an algorithmic level, especially when one considers header files of
arbitrary size.</p>
<p>There is also a PCH implementation for Clang based on the lazy
deserialization of ASTs. This approach theoretically has the same
constant-time algorithmic advantages just mentioned but also retains some
of the strengths of PTH such as reduced memory pressure (ideal for
multi-core builds).</p>
</div>
<div class="section" id="internal-pth-optimizations">
<h3>Internal PTH Optimizations<a class="headerlink" href="#internal-pth-optimizations" title="Permalink to this headline"></a></h3>
<p>While the main optimization employed by PTH is to reduce lexing time of
header files by caching pre-lexed tokens, PTH also employs several other
optimizations to speed up the processing of header files:</p>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">stat</span></code> caching: PTH files cache information obtained via calls to
<code class="docutils literal"><span class="pre">stat</span></code> that <code class="docutils literal"><span class="pre">clang</span> <span class="pre">-cc1</span></code> uses to resolve which files are included
by <code class="docutils literal"><span class="pre">#include</span></code> directives. This greatly reduces the overhead
involved in context-switching to the kernel to resolve included
files.</li>
<li>Fast skipping of <code class="docutils literal"><span class="pre">#ifdef</span></code> ... <code class="docutils literal"><span class="pre">#endif</span></code> chains: PTH files
record the basic structure of nested preprocessor blocks. When the
condition of the preprocessor block is false, all of its tokens are
immediately skipped instead of requiring them to be handled by
Clang&#8217;s preprocessor.</li>
</ul>
</div>
</div>
</div>


      </div>
      <div class="bottomnav" role="navigation" aria-label="bottom navigation">
      
        <p>
        «&#160;&#160;<a href="DriverInternals.html">Driver Design &amp; Internals</a>
        &#160;&#160;::&#160;&#160;
        <a class="uplink" href="index.html">Contents</a>
        &#160;&#160;::&#160;&#160;
        <a href="PCHInternals.html">Precompiled Header and Modules Internals</a>&#160;&#160;»
        </p>

      </div>

    <div class="footer" role="contentinfo">
        &#169; Copyright 2007-2017, The Clang Team.
      Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.9.
    </div>
  </body>
</html>