/usr/share/doc/flex-old-doc/flex

html2
<html>
<!-- Created on May 24, 2011 by texi2html 1.82
texi2html was written by: 
            Lionel Cons <Lionel.Cons@cern.ch> (original author)
            Karl Berry  <karl@freefriends.org>
            Olaf Bachmann <obachman@mathematik.uni-kl.de>
            and many others.
Maintained by: Many creative people.
Send bugs and suggestions to <texi2html-bug@nongnu.org>
-->
<head>
<title>Flex - a scanner generator: 11. Start conditions</title>

<meta name="description" content="Flex - a scanner generator: 11. Start conditions">
<meta name="keywords" content="Flex - a scanner generator: 11. Start conditions">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="texi2html 1.82">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.smallquotation {font-size: smaller}
pre.display {font-family: serif}
pre.format {font-family: serif}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: serif; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: serif; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.roman {font-family:serif; font-weight:normal;}
span.sansserif {font-family:sans-serif; font-weight:normal;}
ul.toc {list-style: none}
-->
</style>


</head>

<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">

<a name="Start-conditions"></a>
<table cellpadding="1" cellspacing="1" border="0">
<tr><td valign="middle" align="left">[<a href="flex_10.html#Generated-scanner" title="Previous section in reading order"> &lt; </a>]</td>
<td valign="middle" align="left">[<a href="flex_12.html#Multiple-buffers" title="Next section in reading order"> &gt; </a>]</td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left">[<a href="flex_10.html#Generated-scanner" title="Beginning of this chapter or previous chapter"> &lt;&lt; </a>]</td>
<td valign="middle" align="left">[<a href="flex.html#Top" title="Up section"> Up </a>]</td>
<td valign="middle" align="left">[<a href="flex_12.html#Multiple-buffers" title="Next chapter"> &gt;&gt; </a>]</td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left">[<a href="flex.html#Top" title="Cover (top) of document">Top</a>]</td>
<td valign="middle" align="left">[<a href="flex_toc.html#SEC_Contents" title="Table of contents">Contents</a>]</td>
<td valign="middle" align="left">[Index]</td>
<td valign="middle" align="left">[<a href="flex_abt.html#SEC_About" title="About (help)"> ? </a>]</td>
</tr></table>
<a name="Start-conditions-1"></a>
<h1 class="chapter">11. Start conditions</h1>

<p><code>flex</code> provides a mechanism for conditionally activating
rules.  Any rule whose pattern is prefixed with &quot;&lt;sc&gt;&quot;
will only be active when the scanner is in the start
condition named &quot;sc&quot;.  For example,
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">&lt;STRING&gt;[^&quot;]*        { /* eat up the string body ... */
            &hellip;
            }
</pre></td></tr></table>

<p>will be active only when the scanner is in the &quot;STRING&quot;
start condition, and
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">&lt;INITIAL,STRING,QUOTE&gt;\.        { /* handle an escape ... */
            &hellip;
            }
</pre></td></tr></table>

<p>will be active only when the current start condition is
either &quot;INITIAL&quot;, &quot;STRING&quot;, or &quot;QUOTE&quot;.
</p>
<p>Start conditions are declared in the definitions (first)
section of the input using unindented lines beginning with
either &lsquo;<samp>%s</samp>&rsquo; or &lsquo;<samp>%x</samp>&rsquo; followed by a list of names.  The former
declares <em>inclusive</em> start conditions, the latter <em>exclusive</em>
start conditions.  A start condition is activated using
the <code>BEGIN</code> action.  Until the next <code>BEGIN</code> action is
executed, rules with the given start condition will be active
and rules with other start conditions will be inactive.
If the start condition is <em>inclusive</em>, then rules with no
start conditions at all will also be active.  If it is
<em>exclusive</em>, then <em>only</em> rules qualified with the start
condition will be active.  A set of rules contingent on the
same exclusive start condition describe a scanner which is
independent of any of the other rules in the <code>flex</code> input.
Because of this, exclusive start conditions make it easy
to specify &quot;mini-scanners&quot; which scan portions of the
input that are syntactically different from the rest
(e.g., comments).
</p>
<p>If the distinction between inclusive and exclusive start
conditions is still a little vague, here&rsquo;s a simple
example illustrating the connection between the two.  The set
of rules:
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">%s example
%%

&lt;example&gt;foo   do_something();

bar            something_else();
</pre></td></tr></table>

<p>is equivalent to
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">%x example
%%

&lt;example&gt;foo   do_something();

&lt;INITIAL,example&gt;bar    something_else();
</pre></td></tr></table>

<p>Without the &lsquo;<samp>&lt;INITIAL,example&gt;</samp>&rsquo; qualifier, the &lsquo;<samp>bar</samp>&rsquo; pattern
in the second example wouldn&rsquo;t be active (i.e., couldn&rsquo;t match) when
in start condition &lsquo;<samp>example</samp>&rsquo;.  If we just used &lsquo;<samp>&lt;example&gt;</samp>&rsquo;
to qualify &lsquo;<samp>bar</samp>&rsquo;, though, then it would only be active in
&lsquo;<samp>example</samp>&rsquo; and not in <code>INITIAL</code>, while in the first example
it&rsquo;s active in both, because in the first example the &lsquo;<samp>example</samp>&rsquo;
starting condition is an <em>inclusive</em> (&lsquo;<samp>%s</samp>&rsquo;) start condition.
</p>
<p>Also note that the special start-condition specifier &lsquo;<samp>&lt;*&gt;</samp>&rsquo;
matches every start condition.  Thus, the above example
could also have been written;
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">%x example
%%

&lt;example&gt;foo   do_something();

&lt;*&gt;bar    something_else();
</pre></td></tr></table>

<p>The default rule (to &lsquo;<samp>ECHO</samp>&rsquo; any unmatched character) remains
active in start conditions.  It is equivalent to:
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">&lt;*&gt;.|\\n     ECHO;
</pre></td></tr></table>

<p>&lsquo;<samp>BEGIN(0)</samp>&rsquo; returns to the original state where only the
rules with no start conditions are active.  This state can
also be referred to as the start-condition &quot;INITIAL&quot;, so
&lsquo;<samp>BEGIN(INITIAL)</samp>&rsquo; is equivalent to &lsquo;<samp>BEGIN(0)</samp>&rsquo;.  (The
parentheses around the start condition name are not required but
are considered good style.)
</p>
<p><code>BEGIN</code> actions can also be given as indented code at the
beginning of the rules section.  For example, the
following will cause the scanner to enter the &quot;SPECIAL&quot; start
condition whenever &lsquo;<samp>yylex()</samp>&rsquo; is called and the global
variable <code>enter_special</code> is true:
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">        int enter_special;

%x SPECIAL
%%
        if ( enter_special )
            BEGIN(SPECIAL);

&lt;SPECIAL&gt;blahblahblah
&hellip;more rules follow&hellip;
</pre></td></tr></table>

<p>To illustrate the uses of start conditions, here is a
scanner which provides two different interpretations of a
string like &quot;123.456&quot;.  By default it will treat it as as
three tokens, the integer &quot;123&quot;, a dot (&rsquo;.&rsquo;), and the
integer &quot;456&quot;.  But if the string is preceded earlier in
the line by the string &quot;expect-floats&quot; it will treat it as
a single token, the floating-point number 123.456:
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">%{
#include &lt;math.h&gt;
%}
%s expect

%%
expect-floats        BEGIN(expect);

&lt;expect&gt;[0-9]+&quot;.&quot;[0-9]+      {
            printf( &quot;found a float, = %f\n&quot;,
                    atof( yytext ) );
            }
&lt;expect&gt;\n           {
            /* that's the end of the line, so
             * we need another &quot;expect-number&quot;
             * before we'll recognize any more
             * numbers
             */
            BEGIN(INITIAL);
            }

[0-9]+      {

Version 2.5               December 1994                        18

            printf( &quot;found an integer, = %d\n&quot;,
                    atoi( yytext ) );
            }

&quot;.&quot;         printf( &quot;found a dot\n&quot; );
</pre></td></tr></table>

<p>Here is a scanner which recognizes (and discards) C
comments while maintaining a count of the current input line.
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">%x comment
%%
        int line_num = 1;

&quot;/*&quot;         BEGIN(comment);

&lt;comment&gt;[^*\n]*        /* eat anything that's not a '*' */
&lt;comment&gt;&quot;*&quot;+[^*/\n]*   /* eat up '*'s not followed by '/'s */
&lt;comment&gt;\n             ++line_num;
&lt;comment&gt;&quot;*&quot;+&quot;/&quot;        BEGIN(INITIAL);
</pre></td></tr></table>

<p>This scanner goes to a bit of trouble to match as much
text as possible with each rule.  In general, when
attempting to write a high-speed scanner try to match as
much possible in each rule, as it&rsquo;s a big win.
</p>
<p>Note that start-conditions names are really integer values
and can be stored as such.  Thus, the above could be
extended in the following fashion:
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">%x comment foo
%%
        int line_num = 1;
        int comment_caller;

&quot;/*&quot;         {
             comment_caller = INITIAL;
             BEGIN(comment);
             }

&hellip;

&lt;foo&gt;&quot;/*&quot;    {
             comment_caller = foo;
             BEGIN(comment);
             }

&lt;comment&gt;[^*\n]*        /* eat anything that's not a '*' */
&lt;comment&gt;&quot;*&quot;+[^*/\n]*   /* eat up '*'s not followed by '/'s */
&lt;comment&gt;\n             ++line_num;
&lt;comment&gt;&quot;*&quot;+&quot;/&quot;        BEGIN(comment_caller);
</pre></td></tr></table>

<p>Furthermore, you can access the current start condition
using the integer-valued <code>YY_START</code> macro.  For example, the
above assignments to <code>comment_caller</code> could instead be
written
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">comment_caller = YY_START;
</pre></td></tr></table>

<p>Flex provides <code>YYSTATE</code> as an alias for <code>YY_START</code> (since that
is what&rsquo;s used by AT&amp;T <code>lex</code>).
</p>
<p>Note that start conditions do not have their own
name-space; %s&rsquo;s and %x&rsquo;s declare names in the same fashion as
#define&rsquo;s.
</p>
<p>Finally, here&rsquo;s an example of how to match C-style quoted
strings using exclusive start conditions, including
expanded escape sequences (but not including checking for
a string that&rsquo;s too long):
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">%x str

%%
        char string_buf[MAX_STR_CONST];
        char *string_buf_ptr;

\&quot;      string_buf_ptr = string_buf; BEGIN(str);

&lt;str&gt;\&quot;        { /* saw closing quote - all done */
        BEGIN(INITIAL);
        *string_buf_ptr = '\0';
        /* return string constant token type and
         * value to parser
         */
        }

&lt;str&gt;\n        {
        /* error - unterminated string constant */
        /* generate error message */
        }

&lt;str&gt;\\[0-7]{1,3} {
        /* octal escape sequence */
        int result;

        (void) sscanf( yytext + 1, &quot;%o&quot;, &amp;result );

        if ( result &gt; 0xff )
                /* error, constant is out-of-bounds */

        *string_buf_ptr++ = result;
        }

&lt;str&gt;\\[0-9]+ {
        /* generate error - bad escape sequence; something
         * like '\48' or '\0777777'
         */
        }

&lt;str&gt;\\n  *string_buf_ptr++ = '\n';
&lt;str&gt;\\t  *string_buf_ptr++ = '\t';
&lt;str&gt;\\r  *string_buf_ptr++ = '\r';
&lt;str&gt;\\b  *string_buf_ptr++ = '\b';
&lt;str&gt;\\f  *string_buf_ptr++ = '\f';

&lt;str&gt;\\(.|\n)  *string_buf_ptr++ = yytext[1];

&lt;str&gt;[^\\\n\&quot;]+        {
        char *yptr = yytext;

        while ( *yptr )
                *string_buf_ptr++ = *yptr++;
        }
</pre></td></tr></table>

<p>Often, such as in some of the examples above, you wind up
writing a whole bunch of rules all preceded by the same
start condition(s).  Flex makes this a little easier and
cleaner by introducing a notion of start condition <em>scope</em>.
A start condition scope is begun with:
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">&lt;SCs&gt;{
</pre></td></tr></table>

<p>where SCs is a list of one or more start conditions.
Inside the start condition scope, every rule automatically
has the prefix &lsquo;<samp>&lt;SCs&gt;</samp>&rsquo; applied to it, until a &lsquo;<samp>}</samp>&rsquo; which
matches the initial &lsquo;<samp>{</samp>&rsquo;.  So, for example,
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">&lt;ESC&gt;{
    &quot;\\n&quot;   return '\n';
    &quot;\\r&quot;   return '\r';
    &quot;\\f&quot;   return '\f';
    &quot;\\0&quot;   return '\0';
}
</pre></td></tr></table>

<p>is equivalent to:
</p>
<table><tr><td>&nbsp;</td><td><pre class="example">&lt;ESC&gt;&quot;\\n&quot;  return '\n';
&lt;ESC&gt;&quot;\\r&quot;  return '\r';
&lt;ESC&gt;&quot;\\f&quot;  return '\f';
&lt;ESC&gt;&quot;\\0&quot;  return '\0';
</pre></td></tr></table>

<p>Start condition scopes may be nested.
</p>
<p>Three routines are available for manipulating stacks of
start conditions:
</p>
<dl compact="compact">
<dt> &lsquo;<samp>void yy_push_state(int new_state)</samp>&rsquo;</dt>
<dd><p>pushes the current start condition onto the top of
the start condition stack and switches to <var>new_state</var>
as though you had used &lsquo;<samp>BEGIN new_state</samp>&rsquo; (recall that
start condition names are also integers).
</p>
</dd>
<dt> &lsquo;<samp>void yy_pop_state()</samp>&rsquo;</dt>
<dd><p>pops the top of the stack and switches to it via
<code>BEGIN</code>.
</p>
</dd>
<dt> &lsquo;<samp>int yy_top_state()</samp>&rsquo;</dt>
<dd><p>returns the top of the stack without altering the
stack&rsquo;s contents.
</p></dd>
</dl>

<p>The start condition stack grows dynamically and so has no
built-in size limitation.  If memory is exhausted, program
execution aborts.
</p>
<p>To use start condition stacks, your scanner must include a
&lsquo;<samp>%option stack</samp>&rsquo; directive (see Options below).
</p>
<hr size="6">
<table cellpadding="1" cellspacing="1" border="0">
<tr><td valign="middle" align="left">[<a href="flex_10.html#Generated-scanner" title="Beginning of this chapter or previous chapter"> &lt;&lt; </a>]</td>
<td valign="middle" align="left">[<a href="flex_12.html#Multiple-buffers" title="Next chapter"> &gt;&gt; </a>]</td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left"> &nbsp; </td>
<td valign="middle" align="left">[<a href="flex.html#Top" title="Cover (top) of document">Top</a>]</td>
<td valign="middle" align="left">[<a href="flex_toc.html#SEC_Contents" title="Table of contents">Contents</a>]</td>
<td valign="middle" align="left">[Index]</td>
<td valign="middle" align="left">[<a href="flex_abt.html#SEC_About" title="About (help)"> ? </a>]</td>
</tr></table>
<p>
 <font size="-1">
  This document was generated on <i>May 24, 2011</i> using <a href="http://www.nongnu.org/texi2html/"><i>texi2html 1.82</i></a>.
 </font>
 <br>

</p>
</body>
</html>
flex-old-doc 2.5.4a-10 / usr / share / doc / flex-old-doc / flex_11.html
