annotate docs/filters.rst @ 267:683c4b35ed26

Wording in docs
author Franz Glasner <fzglas.hg@dom66.de>
date Tue, 19 May 2026 21:09:20 +0200
parents e5ea2f955986
children 9790553d7225
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 .. -*- coding: utf-8; indent-tabs-mode: nil; -*-
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 *********
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 Filters
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 *********
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6
265
e5ea2f955986 Optimize documentation layout: use a local ToC where appropriate (lexers, filters)
Franz Glasner <fzglas.hg@dom66.de>
parents: 258
diff changeset
7 .. only:: html
e5ea2f955986 Optimize documentation layout: use a local ToC where appropriate (lexers, filters)
Franz Glasner <fzglas.hg@dom66.de>
parents: 258
diff changeset
8
e5ea2f955986 Optimize documentation layout: use a local ToC where appropriate (lexers, filters)
Franz Glasner <fzglas.hg@dom66.de>
parents: 258
diff changeset
9 .. contents::
e5ea2f955986 Optimize documentation layout: use a local ToC where appropriate (lexers, filters)
Franz Glasner <fzglas.hg@dom66.de>
parents: 258
diff changeset
10
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
11 The package contains the following filters:
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
12
190
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
13 .. raw:: latex
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
14
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
15 {\footnotesize
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
16
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
17 .. include:: filterlist.rst
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
18
190
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
19 .. raw:: latex
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
20
8825282f6900 Select a smaller font size for all lexer and filter tables in the PDF output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
21 }
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
22
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
23 The AlgPseudocode lexer yields an error token for the following code block.
267
683c4b35ed26 Wording in docs
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
24 `Sphinx`_ warns about this and---as a consequence--- suppresses highlighting
683c4b35ed26 Wording in docs
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
25 for this code block completely:
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
26
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
27 .. code-block:: none
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
28
206
4e4c83a87387 Change the error token filter examples: use a non-existing command
Franz Glasner <fzglas.hg@dom66.de>
parents: 190
diff changeset
29 \nonexisting{TEST}
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
30
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
31 With a custom AlgPseudocode lexer that has ``prohibit_raiseonerror_filter``
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
32 activated the output in `Sphinx`_ is as:
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
33
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
34 .. code-block:: no-raiseonerror-algpseudocode
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
35
206
4e4c83a87387 Change the error token filter examples: use a non-existing command
Franz Glasner <fzglas.hg@dom66.de>
parents: 190
diff changeset
36 \nonexisting{TEST}
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
37
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
38
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
39 With the "errortogenericerror" filter the very same block is highlighted
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
40 as:
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
41
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
42 .. code-block:: genericerror-algpseudocode
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
43
206
4e4c83a87387 Change the error token filter examples: use a non-existing command
Franz Glasner <fzglas.hg@dom66.de>
parents: 190
diff changeset
44 \nonexisting{TEST}
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
45
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
46
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
47 ErrorToGenericErrorTokenFilter
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
48 ==============================
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
49
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
50 :Name: errortogenericerror
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
51 :Filter Options: none
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
52
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
53 Replace all :py:class:`pygments.token.Token.Error` tokens in a stream by
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
54 :py:class:`pygments.token.Token.Generic.Error` tokens.
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
55
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
56
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
57 TokenReplaceFilter
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
58 ==================
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
59
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
60 :Name: tokenreplace
258
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
61 :Filter Options:
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
62 ``replacements``
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
63 **Type:** :py:class:`dict[str | pygments.token.Token, str | pygments.token.Token]`
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
64
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
65 A map from tokens to their replacements.
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
66
180
63b52570795f FIX: Consistent rendering of lexer or filter options
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
67 ``token_from``
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
68 **Type:** :py:class:`str` or :py:class:`pygments.token.Token`
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
69
172
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
70 The name of a token type (like ``Error``) or a token object
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
71 (like :py:class:`pygments.token.Token.Error`).
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
72
258
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
73 If given the `token_to` options is required and `replacements` will be
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
74 augmented with their respective values.
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
75
180
63b52570795f FIX: Consistent rendering of lexer or filter options
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
76 ``token_to``
168
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
77 **Type:** :py:class:`str` or :py:class:`pygments.token.Token`
bff8b900713a REFACTOR: All documentation pages refactored: merge intro and details for lexers and filters
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
78
172
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
79 The name of a token type (like ``Generic.Error``) or a token object
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
80 (like :py:class:`pygments.token.Token.Generic.Error`).
ad80fcbf7b47 Docs formatting
Franz Glasner <fzglas.hg@dom66.de>
parents: 168
diff changeset
81
258
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
82 This option is required if `token_from` is given.
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
83
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
84 Replace all token types given as `replacements` keys or in `token_from`
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
85 with the token types given in `replacements` values or in `token_to`.
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
86
e03e8cfdcece Make the TokenReplaceFilter more flexible by allowing a replacement map
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
87 The values in the token stream are retained.