Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2
diff docs/lexer-algpseudocode.rst @ 286:051c8877ee22
Implement lexer option "strict_tokentype".
It allows the \ttX command to synthesize not yet existing token types.
| author | Franz Glasner <fzglas.hg@dom66.de> |
|---|---|
| date | Thu, 21 May 2026 09:32:35 +0200 |
| parents | afbca50b7dc1 |
| children | 298841bc4dee |
line wrap: on
line diff
--- a/docs/lexer-algpseudocode.rst Wed May 20 20:35:37 2026 +0200 +++ b/docs/lexer-algpseudocode.rst Thu May 21 09:32:35 2026 +0200 @@ -123,6 +123,19 @@ If :py:obj:`True` all the ``\ENDxxx`` commands will be skipped and yield no output. + .. describe:: strict_tokentype + + **Type::** :py:class:`bool` + + **Default:** :py:obj:`True` + + Control whether `Explicit Token Types`_ yield + :py:class:`pygments.token.Token.Generic.Error` tokens + (when ``True``, this is the default) + or a token type that is synthesized on the fly by + :py:func:`pygments.token.string_to_tokentype` + (when ``False``). + .. describe:: gets **Type:** :py:class:`str` or :py:obj:`None` @@ -476,6 +489,21 @@ Its corresponding token type (the associated `key` in this dictionary) will be used as token type for the token. + If a corresponding token type is not found the lexer's behaviour depends + on the lexer option ``strict_tokentype`` (see `Lexer Options`_): + + If ``True`` (the default) the command yields a + :py:class:`pygments.token.Token.Generic.Error` token type for the + given command's content. + + If ``False`` then the `Pygments`_ function + :py:func:`pygments.token.string_to_tokentype` will be called. + This function yields either an existing token type or synthesizes + a new one on the fly. + The associated highlighting in the output may not be well defined. + + For this argument escaping is neither needed nor supported. + #. The content of the second argument will given the token type of the first parameter. @@ -517,11 +545,20 @@ */ \text{• \\ttX{s\}{Escaping brace \\\} and backslash \\\\!\}} \ttX{s}{Escaping brace \} and backslash \\!} /* - * This is a non-existing token type: you get some generic error markup - * with a Generic.Error token and no expansion. + * This is a non-existing token type: + * by default you get some generic error markup with a Generic.Error + * token and no expansion. + * See also `Lexer Options` and `strict_tokentype`. */ \text{• \\ttX{NON-EXISTING\}{∈_∌\}(p1, p2)} \ttX{NON_EXISTING}{∈_∌}(p1, p2) +An example with a lexer and ``strict_tokentype=False`` +(highlighting obviously is like standard text with the templates used): + +.. code-block:: nonstrict-algpseudocode + + \text{• \\ttX{Generic.Not.Yet.Existing\}{∈_∌\}(p1, p2)} \ttX{Generic.Not.Yet.Existing}{∈_∌}(p1, p2) + Old Syntax (Deprecated)
