Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2

annotate docs/filters.rst @ 274:77cfab93a60e

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
FIX: Example when escaping string delimiters
author Franz Glasner <fzglas.hg@dom66.de>
date Wed, 20 May 2026 09:51:29 +0200
parents cb66de44a7b9
children 315e94bac8a7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 .. -*- coding: utf-8; indent-tabs-mode: nil; -*-
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2
269
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
3 .. _filters:
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
4
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 *********
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 Filters
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7 *********
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8
265
e5ea2f955986 Optimize documentation layout: use a local ToC where appropriate (lexers, filters)
Franz Glasner <fzglas.hg@dom66.de>
parents: 258
diff changeset
9 .. only:: html
e5ea2f955986 Optimize documentation layout: use a local ToC where appropriate (lexers, filters)
Franz Glasner <fzglas.hg@dom66.de>
parents: 258
diff changeset
10
269
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
11 .. contents::
265
e5ea2f955986 Optimize documentation layout: use a local ToC where appropriate (lexers, filters)
Franz Glasner <fzglas.hg@dom66.de>
parents: 258
diff changeset
12
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
13 The package contains the following filters:
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14
190
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
15 .. raw:: latex
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
16
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
17 {\footnotesize
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
18
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
19 .. include:: filterlist.rst
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
20
190
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
21 .. raw:: latex
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
22
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
23 }
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
24
269
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
25 The "normal" AlgPseudocode lexer yields an error token for the code block
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
26 below;
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
27 `Sphinx`_ warns about this and---as a consequence---suppresses highlighting
267
683c4b35ed26 Wording in docs
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
28 for this code block completely:
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
29
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
30 .. code-block:: none
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
31
206
4e4c83a87387 Change the error token filter examples: use a non-existing command
Franz Glasner <fzglas.hg@dom66.de>
parents: 190
diff changeset
32 \nonexisting{TEST}
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
33
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
34 With a custom AlgPseudocode lexer that has ``prohibit_raiseonerror_filter``
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
35 activated the output in `Sphinx`_ is as:
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
36
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
37 .. code-block:: no-raiseonerror-algpseudocode
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
38
206
4e4c83a87387 Change the error token filter examples: use a non-existing command
Franz Glasner <fzglas.hg@dom66.de>
parents: 190
diff changeset
39 \nonexisting{TEST}
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
40
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
41
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
42 With the "errortogenericerror" filter the very same block is highlighted
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
43 as:
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
44
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
45 .. code-block:: genericerror-algpseudocode
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
46
206
4e4c83a87387 Change the error token filter examples: use a non-existing command
Franz Glasner <fzglas.hg@dom66.de>
parents: 190
diff changeset
47 \nonexisting{TEST}
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
48
269
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
49 The above custom lexer has been defined for `Sphinx`_ using:
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
50
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
51 .. code-block:: python
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
52
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
53 import functools
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
54 from pygments_lexer_pseudocode2.filters import ErrorToGenericErrorTokenFilter
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
55 from pygments_lexer_pseudocode2.lexers.algpseudocode import AlgPseudocodeLexer
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
56
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
57 def setup(app):
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
58 app.add_lexer(
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
59 "genericerror-algpseudocode",
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
60 functools.partial(
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
61 AlgPseudocodeLexer,
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
62 filters=[ErrorToGenericErrorTokenFilter]))
9790553d7225 More details on custom filters in the documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
63
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
64
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
65 TokenReplaceFilter
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
66 ==================
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
67
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
68 :Name: tokenreplace
258
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
69 :Filter Options:
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
70 ``replacements``
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
71 **Type:** :py:class:`dict[str | pygments.token.Token, str | pygments.token.Token]`
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
72
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
73 A map from tokens to their replacements.
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
74
180
63b52570795f FIX: Consistent rendering of lexer or filter options
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
75 ``token_from``
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
76 **Type:** :py:class:`str` or :py:class:`pygments.token.Token`
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
77
172
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
78 The name of a token type (like ``Error``) or a token object
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
79 (like :py:class:`pygments.token.Token.Error`).
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
80
258
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
81 If given the `token_to` options is required and `replacements` will be
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
82 augmented with their respective values.
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
83
180
63b52570795f FIX: Consistent rendering of lexer or filter options
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
84 ``token_to``
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
85 **Type:** :py:class:`str` or :py:class:`pygments.token.Token`
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
86
172
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
87 The name of a token type (like ``Generic.Error``) or a token object
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
88 (like :py:class:`pygments.token.Token.Generic.Error`).
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
89
258
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
90 This option is required if `token_from` is given.
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
91
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
92 Replace all token types given as `replacements` keys or in `token_from`
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
93 with the token types given in `replacements` values or in `token_to`.
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
94
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
95 The values in the token stream are retained.
270
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
96
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
97
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
98 ErrorToGenericErrorTokenFilter
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
99 ==============================
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
100
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
101 :Name: errortogenericerror
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
102 :Filter Options: none
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
103
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
104 Replace all :py:class:`pygments.token.Token.Error` tokens in a stream by
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
105 :py:class:`pygments.token.Token.Generic.Error` tokens.
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
106
cb66de44a7b9 Make the filter order more logical.
Franz Glasner <fzglas.hg@dom66.de>
parents: 269
diff changeset
107 The filter is implemented as an application of the `TokenReplaceFilter`_.