diff docs/lexer-algpseudocode.rst @ 286:051c8877ee22

Implement lexer option "strict_tokentype". It allows the \ttX command to synthesize not yet existing token types.
author Franz Glasner <fzglas.hg@dom66.de>
date Thu, 21 May 2026 09:32:35 +0200
parents afbca50b7dc1
children 298841bc4dee
line wrap: on
line diff
--- a/docs/lexer-algpseudocode.rst	Wed May 20 20:35:37 2026 +0200
+++ b/docs/lexer-algpseudocode.rst	Thu May 21 09:32:35 2026 +0200
@@ -123,6 +123,19 @@
      If :py:obj:`True` all the ``\ENDxxx`` commands will be skipped and yield
      no output.
 
+  .. describe:: strict_tokentype
+
+     **Type::** :py:class:`bool`
+
+     **Default:** :py:obj:`True`
+
+     Control whether `Explicit Token Types`_ yield
+     :py:class:`pygments.token.Token.Generic.Error` tokens
+     (when ``True``, this is the default)
+     or a token type that is synthesized on the fly by
+     :py:func:`pygments.token.string_to_tokentype`
+     (when ``False``).
+
   .. describe:: gets
 
      **Type:** :py:class:`str` or :py:obj:`None`
@@ -476,6 +489,21 @@
    Its corresponding token type (the associated `key` in this dictionary)
    will be used as token type for the token.
 
+   If a corresponding token type is not found the lexer's behaviour depends
+   on the lexer option ``strict_tokentype`` (see `Lexer Options`_):
+
+     If ``True`` (the default) the command yields a
+     :py:class:`pygments.token.Token.Generic.Error` token type for the
+     given command's content.
+
+     If ``False`` then the `Pygments`_ function
+     :py:func:`pygments.token.string_to_tokentype` will be called.
+     This function yields either an existing token type or synthesizes
+     a new one on the fly.
+     The associated highlighting in the output may not be well defined.
+
+   For this argument escaping is neither needed nor supported.
+
 #. The content of the second argument will given the token type of
    the first parameter.
 
@@ -517,11 +545,20 @@
       */
    \text{• \\ttX{s\}{Escaping brace \\\} and backslash \\\\!\}}   \ttX{s}{Escaping brace \} and backslash \\!}
      /*
-      * This is a non-existing token type: you get some generic error markup
-      * with a Generic.Error token and no expansion.
+      * This is a non-existing token type:
+      * by default you get some generic error markup with a Generic.Error
+      * token and no expansion.
+      * See also `Lexer Options` and `strict_tokentype`.
       */
    \text{• \\ttX{NON-EXISTING\}{∈_∌\}(p1, p2)}   \ttX{NON_EXISTING}{∈_∌}(p1, p2)
 
+An example with a lexer and ``strict_tokentype=False``
+(highlighting obviously is like standard text with the templates used):
+
+.. code-block:: nonstrict-algpseudocode
+
+   \text{• \\ttX{Generic.Not.Yet.Existing\}{∈_∌\}(p1, p2)}      \ttX{Generic.Not.Yet.Existing}{∈_∌}(p1, p2)
+
 
 
 Old Syntax (Deprecated)