diff docs/lexer-algpseudocode.rst @ 285:afbca50b7dc1

Implement an alternate syntax for "Explicit Token Types". Uses a generic two-argument syntax and allows escaping of characters using the common excaping rules. For this to work the AlgPseudocodeLexer is now based on Pygment's ExtendedRegexLexer instead of RegexLexer.
author Franz Glasner <fzglas.hg@dom66.de>
date Wed, 20 May 2026 20:35:37 +0200
parents 1683a10eabb2
children 051c8877ee22
line wrap: on
line diff
--- a/docs/lexer-algpseudocode.rst	Wed May 20 20:32:42 2026 +0200
+++ b/docs/lexer-algpseudocode.rst	Wed May 20 20:35:37 2026 +0200
@@ -457,6 +457,81 @@
 They allow to handle keywords and operators that are not recognized by default.
 And they allow the user to explicitely highlight some input text at low-level.
 
+.. note:: Explicit token types are **case-sensitive**.
+
+.. note:: Explicit token types work in all `expression` and `text` contexts.
+
+.. note:: Nested explicit token types are *not supported*.
+
+
+Current and Recommended Syntax
+------------------------------
+
+The current and recommended use is the ``\ttX{ARG1}{ARG2}`` command.
+
+This command has two required parameters:
+
+#. The content of the first argument `ARG1` must be a `value` in the
+   :py:data:`pygments.token.STANDARD_TYPES` dict.
+   Its corresponding token type (the associated `key` in this dictionary)
+   will be used as token type for the token.
+
+#. The content of the second argument will given the token type of
+   the first parameter.
+
+   Standard `Escaping Rules`_ apply to this argument!
+
+Examples:
+
+.. code-block:: algpseudocode
+
+   \text{• \\ttX{kc\}{C\}}      \ttX{kc}{C}         \rem C as Keyword.Constant
+   \text{• \\ttX{ow\}{∈\}}      \ttX{ow}{∈}         \rem ∈ as Operator.Word
+   \text{• \\ttX{kc\}{A Constant Keyword\}}  \ttX{kc}{A Constant Keyword}  \rem An explicit Keyword.Constant
+   \text{• \\ttX{nv\}{A Variable Name\}}     \ttX{nv}{A Variable Name}     \rem An explicit Name.Variable
+   \text{• \\ttX{ni\}{An Entity*Name\}}      \ttX{ni}{An Entity*Name}      \rem An explicit Name.Entity
+   \text{• \\ttX{k\}{∈ ∌\}}     \ttX{k}{∈ ∌}       \rem ∈ and ∌ as (ordinary) Keywords
+   \text{• \\ttX{o\}{∈ ∌\}}     \ttX{o}{∈ ∌}       \rem ∈ and ∌ as (ordinary) Operators
+     /*
+      * The line below has ∈_∌ as (peculiar) function name.
+      * Their params are automatic (i.e. a normal expression).
+      */
+   \text{• \\ttX{nf\}{∈_∌\}(p1, p2)}             \ttX{nf}{∈_∌}(p1, p2)
+     /*
+      * The line below has ∈_∌ as (peculiar) decorator name (as used in Python).
+      * Their params are automatic (i.e. a normal expression).
+      */
+   \text{• \\ttX{nd\}{∈_∌\}(p1, p2)}             \ttX{nd}{∈_∌}(p1, p2)
+     /*
+      * Normal emphasis ("strong")
+      */
+   \text{• \\ttX{gs\}{this is strong\}}          \ttX{gs}{this is strong}
+     /*
+      * A strong emphasis.
+      * Note that the backslash is a valid delimiter!
+      */
+   \text{• \\ttX{ges\}{A Strong Emphasis!\}}     \ttX{ges}{A Strong Emphasis!}
+     /*
+      * Escaping is allowed and needed for the closing brace!
+      * The example token type is a "String".
+      */
+   \text{• \\ttX{s\}{Escaping brace \\\} and backslash \\\\!\}}   \ttX{s}{Escaping brace \} and backslash \\!}
+     /*
+      * This is a non-existing token type: you get some generic error markup
+      * with a Generic.Error token and no expansion.
+      */
+   \text{• \\ttX{NON-EXISTING\}{∈_∌\}(p1, p2)}   \ttX{NON_EXISTING}{∈_∌}(p1, p2)
+
+
+
+Old Syntax (Deprecated)
+-----------------------
+
+.. deprecated:: 3.0
+   Use `Current and Recommended Syntax`_ instead.
+
+.. note:: The lower-case ``x`` in ``\ttx-``!
+
 `XX` represents a `value` in the :py:data:`pygments.token.STANDARD_TYPES`
 dict.
 Its corresponding token type (the associated `key` in this `dict`) is
@@ -520,10 +595,6 @@
       */
    \text{• \\ttx-NON-EXISTING?∈_∌?(p1, p2)}      \ttx-NON_EXISTING?∈_∌?(p1, p2)
 
-.. note:: Explicit token types are **case-sensitive**.
-
-.. note:: Explicit token types work in all `expression` and `text` contexts.
-
 
 .. _escaping-rules:
 
@@ -540,7 +611,8 @@
 argument content.
 
 A single backslash yields a :py:class:`pygments.token.Token.Generic.Error`
-token when in `default` and `expression` states.
+token when in `default` and `expression` states
+(and also in `Explicit Token Types`_).
 Contrary---in `text` contexts a single backslash character that does not
 introduce a command yields a normal text token.