/usr/share/doc/m4-doc/Input-processing.html is in m4-doc 1.4.18-1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 | <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!--
This manual (31 December 2016) is for GNU M4 (version
1.4.18), a package containing an implementation of the m4 macro
language.
Copyright (C) 1989-1994, 2004-2014, 2016 Free Software
Foundation, Inc.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License,
Version 1.3 or any later version published by the Free Software
Foundation; with no Invariant Sections, no Front-Cover Texts, and no
Back-Cover Texts. A copy of the license is included in the section
entitled "GNU Free Documentation License." -->
<!-- Created by GNU Texinfo 6.3, http://www.gnu.org/software/texinfo/ -->
<head>
<title>GNU M4 1.4.18 macro processor: Input processing</title>
<meta name="description" content="GNU M4 1.4.18 macro processor: Input processing">
<meta name="keywords" content="GNU M4 1.4.18 macro processor: Input processing">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link href="index.html#Top" rel="start" title="Top">
<link href="Indices.html#Indices" rel="index" title="Indices">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="Syntax.html#Syntax" rel="up" title="Syntax">
<link href="Macros.html#Macros" rel="next" title="Macros">
<link href="Other-tokens.html#Other-tokens" rel="prev" title="Other tokens">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.indentedblock {margin-right: 0em}
blockquote.smallindentedblock {margin-right: 0em; font-size: smaller}
blockquote.smallquotation {font-size: smaller}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
div.lisp {margin-left: 3.2em}
div.smalldisplay {margin-left: 3.2em}
div.smallexample {margin-left: 3.2em}
div.smalllisp {margin-left: 3.2em}
kbd {font-style: oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: inherit; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: inherit; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.nolinebreak {white-space: nowrap}
span.roman {font-family: initial; font-weight: normal}
span.sansserif {font-family: sans-serif; font-weight: normal}
ul.no-bullet {list-style: none}
-->
</style>
</head>
<body lang="en">
<a name="Input-processing"></a>
<div class="header">
<p>
Previous: <a href="Other-tokens.html#Other-tokens" accesskey="p" rel="prev">Other tokens</a>, Up: <a href="Syntax.html#Syntax" accesskey="u" rel="up">Syntax</a> [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Indices.html#Indices" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<a name="How-m4-copies-input-to-output"></a>
<h3 class="section">3.5 How <code>m4</code> copies input to output</h3>
<p>As <code>m4</code> reads the input token by token, it will copy each token
directly to the output immediately.
</p>
<p>The exception is when it finds a word with a macro definition. In that
case <code>m4</code> will calculate the macro’s expansion, possibly reading
more input to get the arguments. It then inserts the expansion in front
of the remaining input. In other words, the resulting text from a macro
call will be read and parsed into tokens again.
</p>
<p><code>m4</code> expands a macro as soon as possible. If it finds a macro call
when collecting the arguments to another, it will expand the second call
first. This process continues until there are no more macro calls to
expand and all the input has been consumed.
</p>
<p>For a running example, examine how <code>m4</code> handles this input:
</p>
<div class="example">
<pre class="example">format(`Result is %d', eval(`2**15'))
</pre></div>
<p>First, <code>m4</code> sees that the token ‘<samp>format</samp>’ is a macro name, so
it collects the tokens ‘<samp>(</samp>’, ‘<samp>`Result is %d'</samp>’, ‘<samp>,</samp>’,
and ‘<samp> <!-- /@w --></samp>’, before encountering another potential macro. Sure
enough, ‘<samp>eval</samp>’ is a macro name, so the nested argument collection
picks up ‘<samp>(</samp>’, ‘<samp>`2**15'</samp>’, and ‘<samp>)</samp>’, invoking the eval macro
with the lone argument of ‘<samp>2**15</samp>’. The expansion of
‘<samp>eval(2**15)</samp>’ is ‘<samp>32768</samp>’, which is then rescanned as the five
tokens ‘<samp>3</samp>’, ‘<samp>2</samp>’, ‘<samp>7</samp>’, ‘<samp>6</samp>’, and ‘<samp>8</samp>’; and
combined with the next ‘<samp>)</samp>’, the format macro now has all its
arguments, as if the user had typed:
</p>
<div class="example">
<pre class="example">format(`Result is %d', 32768)
</pre></div>
<p>The format macro expands to ‘<samp>Result is 32768</samp>’, and we have another
round of scanning for the tokens ‘<samp>Result</samp>’, ‘<samp> <!-- /@w --></samp>’,
‘<samp>is</samp>’, ‘<samp> <!-- /@w --></samp>’, ‘<samp>3</samp>’, ‘<samp>2</samp>’, ‘<samp>7</samp>’, ‘<samp>6</samp>’, and
‘<samp>8</samp>’. None of these are macros, so the final output is
</p>
<div class="example">
<pre class="example">⇒Result is 32768
</pre></div>
<p>As a more complicated example, we will contrast an actual code
example from the Gnulib project<a name="DOCF1" href="#FOOT1"><sup>1</sup></a>,
showing both a buggy approach and the desired results. The user desires
to output a shell assignment statement that takes its argument and turns
it into a shell variable by converting it to uppercase and prepending a
prefix. The original attempt looks like this:
</p>
<div class="example">
<pre class="example">changequote([,])dnl
define([gl_STRING_MODULE_INDICATOR],
[
dnl comment
GNULIB_]translit([$1],[a-z],[A-Z])[=1
])dnl
gl_STRING_MODULE_INDICATOR([strcase])
⇒ <!-- /@w -->
⇒ GNULIB_strcase=1
⇒ <!-- /@w -->
</pre></div>
<p>Oops – the argument did not get capitalized. And although the manual
is not able to easily show it, both lines that appear empty actually
contain two trailing spaces. By stepping through the parse, it is easy
to see what happened. First, <code>m4</code> sees the token
‘<samp>changequote</samp>’, which it recognizes as a macro, followed by
‘<samp>(</samp>’, ‘<samp>[</samp>’, ‘<samp>,</samp>’, ‘<samp>]</samp>’, and ‘<samp>)</samp>’ to form the
argument list. The macro expands to the empty string, but changes the
quoting characters to something more useful for generating shell code
(unbalanced ‘<samp>`</samp>’ and ‘<samp>'</samp>’ appear all the time in shell scripts,
but unbalanced ‘<samp>[]</samp>’ tend to be rare). Also in the first line,
<code>m4</code> sees the token ‘<samp>dnl</samp>’, which it recognizes as a builtin
macro that consumes the rest of the line, resulting in no output for
that line.
</p>
<p>The second line starts a macro definition. <code>m4</code> sees the token
‘<samp>define</samp>’, which it recognizes as a macro, followed by a ‘<samp>(</samp>’,
‘<samp>[gl_STRING_MODULE_INDICATOR]</samp>’, and ‘<samp>,</samp>’. Because an unquoted
comma was encountered, the first argument is known to be the expansion
of the single-quoted string token, or ‘<samp>gl_STRING_MODULE_INDICATOR</samp>’.
Next, <code>m4</code> sees ‘<samp><span class="key">NL</span></samp>’, ‘<samp> </samp>’, and ‘<samp> </samp>’, but this
whitespace is discarded as part of argument collection. Then comes a
rather lengthy single-quoted string token, ‘<samp>[<span class="key">NL</span> dnl
comment<span class="key">NL</span> GNULIB_]</samp>’. This is followed by the token
‘<samp>translit</samp>’, which <code>m4</code> recognizes as a macro name, so a nested
macro expansion has started.
</p>
<p>The arguments to the <code>translit</code> are found by the tokens ‘<samp>(</samp>’,
‘<samp>[$1]</samp>’, ‘<samp>,</samp>’, ‘<samp>[a-z]</samp>’, ‘<samp>,</samp>’, ‘<samp>[A-Z]</samp>’, and finally
‘<samp>)</samp>’. All three string arguments are expanded (or in other words,
the quotes are stripped), and since neither ‘<samp>$</samp>’ nor ‘<samp>1</samp>’ need
capitalization, the result of the macro is ‘<samp>$1</samp>’. This expansion is
rescanned, resulting in the two literal characters ‘<samp>$</samp>’ and
‘<samp>1</samp>’.
</p>
<p>Scanning of the outer macro resumes, and picks up with
‘<samp>[=1<span class="key">NL</span> ]</samp>’, and finally ‘<samp>)</samp>’. The collected pieces of
expanded text are concatenated, with the end result that the macro
‘<samp>gl_STRING_MODULE_INDICATOR</samp>’ is now defined to be the sequence
‘<samp><span class="key">NL</span> dnl comment<span class="key">NL</span> GNULIB_$1=1<span class="key">NL</span> </samp>’.
Once again, ‘<samp>dnl</samp>’ is recognized and avoids a newline in the output.
</p>
<p>The final line is then parsed, beginning with ‘<samp> </samp>’ and ‘<samp> </samp>’
that are output literally. Then ‘<samp>gl_STRING_MODULE_INDICATOR</samp>’ is
recognized as a macro name, with an argument list of ‘<samp>(</samp>’,
‘<samp>[strcase]</samp>’, and ‘<samp>)</samp>’. Since the definition of the macro
contains the sequence ‘<samp>$1</samp>’, that sequence is replaced with the
argument ‘<samp>strcase</samp>’ prior to starting the rescan. The rescan sees
‘<samp><span class="key">NL</span></samp>’ and four spaces, which are output literally, then
‘<samp>dnl</samp>’, which discards the text ‘<samp> comment<span class="key">NL</span></samp>’. Next
comes four more spaces, also output literally, and the token
‘<samp>GNULIB_strcase</samp>’, which resulted from the earlier parameter
substitution. Since that is not a macro name, it is output literally,
followed by the literal tokens ‘<samp>=</samp>’, ‘<samp>1</samp>’, ‘<samp><span class="key">NL</span></samp>’, and
two more spaces. Finally, the original ‘<samp><span class="key">NL</span></samp>’ seen after the
macro invocation is scanned and output literally.
</p>
<p>Now for a corrected approach. This rearranges the use of newlines and
whitespace so that less whitespace is output (which, although harmless
to shell scripts, can be visually unappealing), and fixes the quoting
issues so that the capitalization occurs when the macro
‘<samp>gl_STRING_MODULE_INDICATOR</samp>’ is invoked, rather then when it is
defined. It also adds another layer of quoting to the first argument of
<code>translit</code>, to ensure that the output will be rescanned as a string
rather than a potential uppercase macro name needing further expansion.
</p>
<div class="example">
<pre class="example">changequote([,])dnl
define([gl_STRING_MODULE_INDICATOR],
[dnl comment
GNULIB_[]translit([[$1]], [a-z], [A-Z])=1dnl
])dnl
gl_STRING_MODULE_INDICATOR([strcase])
⇒ GNULIB_STRCASE=1
</pre></div>
<p>The parsing of the first line is unchanged. The second line sees the
name of the macro to define, then sees the discarded ‘<samp><span class="key">NL</span></samp>’
and two spaces, as before. But this time, the next token is
‘<samp>[dnl comment<span class="key">NL</span> GNULIB_[]translit([[$1]], [a-z],
[A-Z])=1dnl<span class="key">NL</span>]</samp>’, which includes nested quotes, followed by
‘<samp>)</samp>’ to end the macro definition and ‘<samp>dnl</samp>’ to skip the
newline. No early expansion of <code>translit</code> occurs, so the entire
string becomes the definition of the macro.
</p>
<p>The final line is then parsed, beginning with two spaces that are
output literally, and an invocation of
<code>gl_STRING_MODULE_INDICATOR</code> with the argument ‘<samp>strcase</samp>’.
Again, the ‘<samp>$1</samp>’ in the macro definition is substituted prior to
rescanning. Rescanning first encounters ‘<samp>dnl</samp>’, and discards
‘<samp> comment<span class="key">NL</span></samp>’. Then two spaces are output literally. Next
comes the token ‘<samp>GNULIB_</samp>’, but that is not a macro, so it is
output literally. The token ‘<samp>[]</samp>’ is an empty string, so it does
not affect output. Then the token ‘<samp>translit</samp>’ is encountered.
</p>
<p>This time, the arguments to <code>translit</code> are parsed as ‘<samp>(</samp>’,
‘<samp>[[strcase]]</samp>’, ‘<samp>,</samp>’, ‘<samp> </samp>’, ‘<samp>[a-z]</samp>’, ‘<samp>,</samp>’, ‘<samp> </samp>’,
‘<samp>[A-Z]</samp>’, and ‘<samp>)</samp>’. The two spaces are discarded, and the
translit results in the desired result ‘<samp>[STRCASE]</samp>’. This is
rescanned, but since it is a string, the quotes are stripped and the
only output is a literal ‘<samp>STRCASE</samp>’.
Then the scanner sees ‘<samp>=</samp>’ and ‘<samp>1</samp>’, which are output
literally, followed by ‘<samp>dnl</samp>’ which discards the rest of the
definition of <code>gl_STRING_MODULE_INDICATOR</code>. The newline at the
end of output is the literal ‘<samp><span class="key">NL</span></samp>’ that appeared after the
invocation of the macro.
</p>
<p>The order in which <code>m4</code> expands the macros can be further explored
using the trace facilities of GNU <code>m4</code> (see <a href="Trace.html#Trace">Trace</a>).
</p>
<div class="footnote">
<hr>
<h4 class="footnotes-heading">Footnotes</h4>
<h3><a name="FOOT1" href="#DOCF1">(1)</a></h3>
<p>Derived from a patch in
<a href="http://lists.gnu.org/archive/html/bug-gnulib/2007-01/msg00389.html">http://lists.gnu.org/archive/html/bug-gnulib/2007-01/msg00389.html</a>,
and a followup patch in
<a href="http://lists.gnu.org/archive/html/bug-gnulib/2007-02/msg00000.html">http://lists.gnu.org/archive/html/bug-gnulib/2007-02/msg00000.html</a></p>
</div>
<hr>
<div class="header">
<p>
Previous: <a href="Other-tokens.html#Other-tokens" accesskey="p" rel="prev">Other tokens</a>, Up: <a href="Syntax.html#Syntax" accesskey="u" rel="up">Syntax</a> [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Indices.html#Indices" title="Index" rel="index">Index</a>]</p>
</div>
</body>
</html>
|