Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2

annotate pygments_lexer_pseudocode2/lexers/algpseudocode.py @ 285:afbca50b7dc1

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Implement an alternate syntax for "Explicit Token Types". Uses a generic two-argument syntax and allows escaping of characters using the common excaping rules. For this to work the AlgPseudocodeLexer is now based on Pygment's ExtendedRegexLexer instead of RegexLexer.
author Franz Glasner <fzglas.hg@dom66.de>
date Wed, 20 May 2026 20:35:37 +0200
parents 5eba722df93e
children 051c8877ee22
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 # SPDX-FileCopyrightText: © 2026 Franz Glasner
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # SPDX-License-Identifier: MIT
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 r"""A pseudocode lexer along the lines of CTAN's algpseudocode or
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7 algpseudocodex.
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9 """
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
11 __all__ = ["AlgPseudocodeLexer",
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
12 "AlgPseudocodeLexer_DE", "AlgPseudocodeLexer_FR"]
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
13
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14
89
7414eed7b275 Introduce logging for unhandled explicit token types
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
15 import logging
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
16 import re
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
17
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
18 import pygments.util
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
19 from pygments.lexer import bygroups, include, words
160
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
20 from pygments.token import (Comment, Error, Generic, Keyword, Name, Operator,
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 85
diff changeset
21 Punctuation, Text, Whitespace)
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
22
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
23 #
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
24 # Relative imports do not work with pygments.lexers.load_lexer_from_file()
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
25 # in all of our supported Python releases.
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
26 #
164
a4317957148b Move all lexers into a subpackage pygments_lexer_pseudocode2.lexers.
Franz Glasner <fzglas.hg@dom66.de>
parents: 162
diff changeset
27 from pygments_lexer_pseudocode2.lexers.bases import LexBase
85
ae5e741d2a9b Optimize op_explicit_tokentype(): use a pref-computed reversed pygments.token.STANDARD_TYPES
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
28 from pygments_lexer_pseudocode2.utils import REVERSED_STANDARD_TYPES
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
29 from pygments_lexer_pseudocode2 import uniprops
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
30
89
7414eed7b275 Introduce logging for unhandled explicit token types
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
31 #
7414eed7b275 Introduce logging for unhandled explicit token types
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
32 # As in the local imports: use an explicit name because __name__ is
7414eed7b275 Introduce logging for unhandled explicit token types
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
33 # __builtins__
7414eed7b275 Introduce logging for unhandled explicit token types
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
34 #
7414eed7b275 Introduce logging for unhandled explicit token types
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
35 _logger = logging.getLogger("pygments_lexer_pseudocode2.algpseudocode")
7414eed7b275 Introduce logging for unhandled explicit token types
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
36
7414eed7b275 Introduce logging for unhandled explicit token types
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
37
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
38 class AlgPseudocodeLexer(LexBase):
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
39
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
40 """A pseudocode lexer along the lines of CTAN's algpseudocode or
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
41 algpseudocodex.
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
42
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
43 Some ideas (e.g. strings) are borrowed from Pygment's Python lexer.
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
44
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
45 """
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
46
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
47 name = "AlgPseudocode"
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
48 aliases = ["algpseudocode", "algpseudo"]
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
49 filenames = ["*.algpseudo", "*.algpseudocode"]
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
50 mimetypes = []
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
51 flags = re.MULTILINE
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
52
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
53 LANG = "en"
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
54 TRANSLATIONS = {
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
55 "PROG": "PROGRAM",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
56 "PROGRAM": "PROGRAM",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
57 "ALGO": "ALGORITHM",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
58 "ALGORITHM": "ALGORITHM",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
59 "PROC": "PROCEDURE",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
60 "PROCEDURE": "PROCEDURE",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
61 "FUNC": "FUNCTION",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
62 "FUNCTION": "FUNCTION",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
63 "FN": "FUNCTION",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
64 "CLASS": "CLASS",
99
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
65 "INPUT": "Input:",
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
66 "INPUTS": "Inputs:",
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
67 "OUTPUT": "Output:",
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
68 "OUTPUTS": "Outputs:",
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
69 "RETURNS": "Returns:",
106
f6b46a379aba The "\ENSURE" command
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
70 "ENSURE": "Ensure:",
124
f916251d5647 Implement the "\REQUIRE" command from algpseudocodex
Franz Glasner <fzglas.hg@dom66.de>
parents: 123
diff changeset
71 "REQUIRE": "Require:",
61
392745b66969 The "\IS" keyword expansion
Franz Glasner <fzglas.hg@dom66.de>
parents: 58
diff changeset
72 "IS": "IS",
82
36a0ef76a6d7 Implement "\WITH"
Franz Glasner <fzglas.hg@dom66.de>
parents: 81
diff changeset
73 "WITH": "WITH",
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
74 "IF": "IF",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
75 "THEN": "THEN",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
76 "ELSE": "ELSE",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
77 "ELSEIF": "ELSE IF",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
78 "ELSIF": "ELSE IF",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
79 "ELIF": "ELSE IF",
111
d6f3a1d1bedd Some more keywords (\FROM, \TO, \IN)
Franz Glasner <fzglas.hg@dom66.de>
parents: 109
diff changeset
80 "DO": "DO", # in WHILE ... DO
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
81 "WHILE": "WHILE",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
82 "FOR": "FOR",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
83 "FORALL": "FOR ALL",
111
d6f3a1d1bedd Some more keywords (\FROM, \TO, \IN)
Franz Glasner <fzglas.hg@dom66.de>
parents: 109
diff changeset
84 "FROM": "FROM",
d6f3a1d1bedd Some more keywords (\FROM, \TO, \IN)
Franz Glasner <fzglas.hg@dom66.de>
parents: 109
diff changeset
85 "TO": "TO",
d6f3a1d1bedd Some more keywords (\FROM, \TO, \IN)
Franz Glasner <fzglas.hg@dom66.de>
parents: 109
diff changeset
86 "IN": "IN", # as in FOR ... IN
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
87 "STEP": "STEP",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
88 "LOOP": "LOOP",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
89 "REPEAT": "REPEAT",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
90 "UNTIL": "UNTIL",
104
ffe6ea2cf69b Allow "\RETURN" and "\RETURNS" commands.
Franz Glasner <fzglas.hg@dom66.de>
parents: 103
diff changeset
91 "RETURN": "RETURN",
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
92 "BEGIN": "BEGIN",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
93 "END": "END", # not in END_TRANSLATIONS
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
94 }
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
95 END_TRANSLATIONS = {
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
96 "PROG": "END OF PROGRAM",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
97 "PROGRAM": "END OF PROGRAM",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
98 "ALGO": "END OF ALGORITHM",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
99 "ALGORITHM": "END OF ALGORITHM",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
100 "PROC": "END OF PROCEDURE",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
101 "PROCEDURE": "END OF PROCEDURE",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
102 "FUNC": "END OF FUNCTION",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
103 "FUNCTION": "END OF FUNCTION",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
104 "FN": "END OF FUNCTION",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
105 "CLASS": "END OF CLASS",
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
106 "IF": "END IF",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
107 "WHILE": "END WHILE",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
108 "FOR": "END FOR",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
109 "FORALL": "END FOR ALL",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
110 "LOOP": "END LOOP",
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
111 }
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
112 DEFAULT_END_PREFIX = "END OF "
123
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
113 SYMBOL_REMARK = u"▷" # U+25B7: Unicode 1.0 (Geometric Shapes)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
114 # SYMBOL_REMARK = u"▻" # U+25BB: Unicode 1.0 (Geometric Shapes)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
115 # SYMBOL_REMARK = u"⍝" # U+235D: Unicode 1.1 (Misc. Technical, APL)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
116 SYMBOL_BLOCK = u"◆" # U+25C6: Unicode 1.0 (Geometric Shapes)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
117 # SYMBOL_BLOCK = u"┃" # U+2503: Unicode 1.0 (Bow Drawing)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
118 # SYMBOL_BLOCK = u"●" # U+25CF: Unicode 1.0 (Geometric Shapes)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
119 SYMBOL_TEXTSTATEMENT = u"▪" # U+25AA: Unicode 1.0 (Geometric Shapes)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
120 # SYMBOL_TEXTSTATEMENT = u"■" # U+25A0: Unicode 1.0 (Geometric Shapes)
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
121 SYMBOLS = {
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
122 # Group REMARK
58
be065e5c8042 Prepare symbols for a statement which defaults to token type "Text".
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
123 "REMARK": SYMBOL_REMARK,
be065e5c8042 Prepare symbols for a statement which defaults to token type "Text".
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
124 "REM": SYMBOL_REMARK,
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
125 # Group STATEMENT
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
126 "STATEMENT": SYMBOL_BLOCK,
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
127 "STATE": SYMBOL_BLOCK,
58
be065e5c8042 Prepare symbols for a statement which defaults to token type "Text".
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
128 "BLOCK": SYMBOL_BLOCK,
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
129 # Group TEXTSTATEMENT
58
be065e5c8042 Prepare symbols for a statement which defaults to token type "Text".
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
130 "TEXTSTATEMENT": SYMBOL_TEXTSTATEMENT,
78
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
131 "TEXTSTATE": SYMBOL_TEXTSTATEMENT,
58
be065e5c8042 Prepare symbols for a statement which defaults to token type "Text".
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
132 "TSTATEMENT": SYMBOL_TEXTSTATEMENT,
be065e5c8042 Prepare symbols for a statement which defaults to token type "Text".
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
133 "TSTATE": SYMBOL_TEXTSTATEMENT,
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
134 "TEXTBLOCK": SYMBOL_TEXTSTATEMENT,
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
135 "TBLOCK": SYMBOL_TEXTSTATEMENT,
123
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
136 "<-": u"⟵", # U+27F5: Unicode 3.2 (Supplemental Arrows-A)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
137 "->": u"⟶", # U+27F6: Unicode 3.2 (Supplemental Arrows-A)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
138 "<->": u"⟷", # U+27F7: Unicode 3.2 (Supplemental Arrows-A)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
139 # "=>": u"⟹", # U+27F9: Unicode 3.2 (Supplemental Arrows-A)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
140 # "<=>": u"⟺", # U+27FA: Unicode 3.2 (Supplemental Arrows-A)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
141 # "<-": u"←", # U+2190: Unicode 1.0 (Arrows)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
142 # "->": u"→", # U+2192: Unicode 1.0 (Arrows)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
143 # "<->": u"↔", # U+2194: Unicode 1.0 (Arrows)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
144 "=>": u"⇒", # U+21D2: Unicode 1.0 (Arrows)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
145 "<=>": u"⇔", # U+21D4: Unicode 1.0 (Arrows)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
146 "<=": u"≤", # U+2264: Unicode 1.0 (Mathematical Operators)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
147 ">=": u"≥", # U+2265: Unicode 1.0 (Mathematical Operators)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
148 "<>": u"≠", # U+2260: Unicode 1.0 (Mathematical Operators)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
149 "!=": u"≠", # U+2260: Unicode 1.0 (Mathematical Operators)
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
150 ":=": u"∶=", # "≔" U+2254 not recognizable in my (small) mono font
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
151 "=:": u"=∶", # "≕" U+2255 not recognizable in my (small) mono font
4d96ace53ba1 Make it work on Python2 too with all tests by explicitely declaring some strings to be Unicode strings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 122
diff changeset
152 "?=": u"≟", # U+225F: Unicode 1.0 (Mathematical Operators)
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
153 }
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
154
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
155 def op_translate(toktype):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
156
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
157 def _op_translate(lexer, match, ctx=None):
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
158 kw = match.group().upper()
38
69522d4cafac Remove another unneeded local variable
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
159 yield match.start(), toktype, lexer.TRANSLATIONS.get(kw, kw)
83
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
160 if ctx:
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
161 ctx.pos = match.end()
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
162
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
163 return _op_translate
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
164
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
165 def op_opt_end_translate(toktype):
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
166
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
167 def _op_end_translate(lexer, match, ctx=None):
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
168 if not lexer.no_end:
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
169 kw = match.group().upper()
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
170 yield (match.start(),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
171 toktype,
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
172 lexer.END_TRANSLATIONS.get(
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
173 kw,
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
174 lexer.DEFAULT_END_PREFIX + kw))
83
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
175 if ctx:
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
176 ctx.pos = match.end()
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
177
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
178 return _op_end_translate
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
179
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
180 def op_opt_ignore(toktype):
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
181
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
182 def _op_opt_ignore(lexer, match, ctx=None):
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
183 if not lexer.no_end:
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
184 yield match.start(), toktype, match.group()
83
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
185 if ctx:
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
186 ctx.pos = match.end()
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
187
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
188 return _op_opt_ignore
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
189
72
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
190 def op_opt_ignore_or_fixed(toktype, value):
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
191 """Yield a fixed given token type and value or -- if the lexer's
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
192 `no_end` setting evals to ``True`` nothing.
65
3f4223a79d2b Normalize whitespace handling for entity names
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
193
3f4223a79d2b Normalize whitespace handling for entity names
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
194 """
72
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
195
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
196 def _op_opt_ignore_or_fixed(lexer, match, ctx=None):
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
197 if not lexer.no_end:
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
198 yield match.start(), toktype, value
83
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
199 if ctx:
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
200 ctx.pos = match.end()
72
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
201
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
202 return _op_opt_ignore_or_fixed
65
3f4223a79d2b Normalize whitespace handling for entity names
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
203
113
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
204 def op_gets(lexer, match, ctx=None):
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
205 yield match.start(), Operator, lexer.symbol_gets
283
5eba722df93e FIX: Some missing context position updates if a context is given
Franz Glasner <fzglas.hg@dom66.de>
parents: 282
diff changeset
206 if ctx:
5eba722df93e FIX: Some missing context position updates if a context is given
Franz Glasner <fzglas.hg@dom66.de>
parents: 282
diff changeset
207 ctx.pos = match.end()
113
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
208
117
d84f1fd10e64 Allow to customize the symbol for a remark
Franz Glasner <fzglas.hg@dom66.de>
parents: 116
diff changeset
209 def op_remark(lexer, match, ctx=None):
d84f1fd10e64 Allow to customize the symbol for a remark
Franz Glasner <fzglas.hg@dom66.de>
parents: 116
diff changeset
210 yield match.start(), Comment.Single, lexer.symbol_remark
283
5eba722df93e FIX: Some missing context position updates if a context is given
Franz Glasner <fzglas.hg@dom66.de>
parents: 282
diff changeset
211 if ctx:
5eba722df93e FIX: Some missing context position updates if a context is given
Franz Glasner <fzglas.hg@dom66.de>
parents: 282
diff changeset
212 ctx.pos = match.end()
117
d84f1fd10e64 Allow to customize the symbol for a remark
Franz Glasner <fzglas.hg@dom66.de>
parents: 116
diff changeset
213
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
214 def op_symbol(toktype):
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
215
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
216 def _op_symbol(lexer, match, ctx=None):
64
05c53e431c88 Simplify case-handling for "BLOCK" and "REMARK"
Franz Glasner <fzglas.hg@dom66.de>
parents: 62
diff changeset
217 kw = match.group().upper()
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
218 yield match.start(), toktype, lexer.SYMBOLS.get(kw, kw)
83
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
219 if ctx:
cd79d2c76347 If a Pygments callback gets a "Context" it must set the new position explicitely.
Franz Glasner <fzglas.hg@dom66.de>
parents: 82
diff changeset
220 ctx.pos = match.end()
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
221
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
222 return _op_symbol
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
223
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
224 def op_explicit_tokentype(lexer, match, ctx=None):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
225 needed_css = match.group("type")
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 85
diff changeset
226 toktype = REVERSED_STANDARD_TYPES.get(needed_css, None)
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 85
diff changeset
227 if toktype is None:
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 85
diff changeset
228 # Be more error friendly
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 85
diff changeset
229 toktype = Generic.Error
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 85
diff changeset
230 val = match.group()
89
7414eed7b275 Introduce logging for unhandled explicit token types
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
231 _logger.warning("Unhandled explicit token type: %s", val)
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 85
diff changeset
232 else:
282
90946f87d77d Rename RE group "character" -> "characters"
Franz Glasner <fzglas.hg@dom66.de>
parents: 281
diff changeset
233 val = match.group("characters")
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 85
diff changeset
234 yield match.start(), toktype, val
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
235 if ctx:
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
236 ctx.pos = match.end()
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
237
285
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
238 def op_explicit_tokentype_ex_start(lexer, match, ctx):
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
239 needed_css = match.group("type")
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
240 ctx.explicit_token_type = REVERSED_STANDARD_TYPES.get(needed_css, None)
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
241 if ctx.explicit_token_type is None:
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
242 # Be more error friendly
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
243 ctx.explicit_token_type = Generic.Error
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
244 _logger.warning("Unhandled explicit token type: %s", match.group())
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
245 yield match.start(), ctx.explicit_token_type, match.group()
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
246 ctx.pos = match.end()
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
247
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
248 def op_explicit_tokentype_ex_value(lexer, match, ctx):
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
249 yield match.start(), ctx.explicit_token_type, match.group(1)
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
250 ctx.pos = match.end()
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
251
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
252 def op_explicit_tokentype_ex_end(lexer, match, ctx):
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
253 if ctx.explicit_token_type is Generic.Error:
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
254 yield match.start(), ctx.explicit_token_type, match.group()
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
255 ctx.pos = match.end()
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
256 ctx.explicit_token_type = None
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
257
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
258 tokens = {
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
259 "root": [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
260 (r"\n", Whitespace),
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
261 (r"/\*", Comment.Multiline, "multiline-nested-comment"),
115
e1663ac707b0 An alternate multiline comment syntax using "(* ... *)".
Franz Glasner <fzglas.hg@dom66.de>
parents: 114
diff changeset
262 (r"\(\*", Comment.Multiline, "multiline-nested-comment-alt"),
100
7cfad325d3bb Allow also `#' to start single-line comments
Franz Glasner <fzglas.hg@dom66.de>
parents: 99
diff changeset
263 (r"(//|#).*$", Comment.Single),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
264 include("remark"),
90
2af4684c8e78 Restrict whitespace before required and optional arguments.
Franz Glasner <fzglas.hg@dom66.de>
parents: 89
diff changeset
265 (r"(?i)\\(block|state(?:ment)?)[ \t]*(\{)",
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
266 bygroups(op_symbol(Text), LexBase.op_fixed(Whitespace, " ")),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
267 "block-expr"),
78
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
268 (r"(?i)\\("
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
269 r"(?:textstate(?:ment)?)"
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
270 r"|(?:tstate(?:ment)?)"
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
271 r"|(?:textblock)"
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
272 r"|(?:tblock)"
90
2af4684c8e78 Restrict whitespace before required and optional arguments.
Franz Glasner <fzglas.hg@dom66.de>
parents: 89
diff changeset
273 r")[ \t]*(\{)",
78
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
274 bygroups(op_symbol(Text), LexBase.op_fixed(Whitespace, " ")),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
275 "text-statement"),
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
276 (r"(?i)\\("
99
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
277 r"(?:input(?:s)?)"
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
278 r"|(?:output(?:s)?)"
108
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 106
diff changeset
279 r"|(?:ensure)"
124
f916251d5647 Implement the "\REQUIRE" command from algpseudocodex
Franz Glasner <fzglas.hg@dom66.de>
parents: 123
diff changeset
280 r"|(?:require)"
104
ffe6ea2cf69b Allow "\RETURN" and "\RETURNS" commands.
Franz Glasner <fzglas.hg@dom66.de>
parents: 103
diff changeset
281 r"|(?:returns)"
99
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
282 r")[ \t]*(\{)",
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
283 bygroups(op_translate(Keyword),
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
284 LexBase.op_fixed(Whitespace, " ")),
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
285 "text-statement"),
857df2064831 \INPUT, \INPUTS, \OUTPUT, \OUTPUTS, \RETURN and \RETURNS as new commands (keywords)
Franz Glasner <fzglas.hg@dom66.de>
parents: 98
diff changeset
286 (r"(?i)\\("
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
287 r"(?:if)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
288 r"|(?:then)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
289 r"|(?:else)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
290 r"|(?:el(?:s(?:e)?)?if)"
111
d6f3a1d1bedd Some more keywords (\FROM, \TO, \IN)
Franz Glasner <fzglas.hg@dom66.de>
parents: 109
diff changeset
291 r"|(?:do)" # as in WHILE ... DO not DO ... UNTIL
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
292 r"|(?:while)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
293 r"|(?:forall)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
294 r"|(?:for)"
111
d6f3a1d1bedd Some more keywords (\FROM, \TO, \IN)
Franz Glasner <fzglas.hg@dom66.de>
parents: 109
diff changeset
295 r"|(?:from)"
d6f3a1d1bedd Some more keywords (\FROM, \TO, \IN)
Franz Glasner <fzglas.hg@dom66.de>
parents: 109
diff changeset
296 r"|(?:to)"
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
297 r"|(?:step)"
111
d6f3a1d1bedd Some more keywords (\FROM, \TO, \IN)
Franz Glasner <fzglas.hg@dom66.de>
parents: 109
diff changeset
298 r"|(?:in)"
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
299 r"|(?:loop)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
300 r"|(?:repeat)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
301 r"|(?:until)"
104
ffe6ea2cf69b Allow "\RETURN" and "\RETURNS" commands.
Franz Glasner <fzglas.hg@dom66.de>
parents: 103
diff changeset
302 r"|(?:return)"
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
303 r")\b",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
304 bygroups(op_translate(Keyword))),
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
305 (r"\\\n", Text),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
306 (r"(?i)\\("
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
307 r"(?:prog(?:ram)?)"
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
308 r"|(?:algo(?:rithm)?)"
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
309 r"|(?:proc(?:edure)?)"
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
310 r"|(?:func(?:tion)?|(?:fn))"
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
311 r"|(?:class)"
90
2af4684c8e78 Restrict whitespace before required and optional arguments.
Franz Glasner <fzglas.hg@dom66.de>
parents: 89
diff changeset
312 r")[ \t]*(\{)",
72
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
313 bygroups(op_translate(Keyword),
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
314 LexBase.op_fixed(Whitespace, " ")),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
315 "entity-name"),
94
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 93
diff changeset
316 # ENDxxx keywords with optional entity name in two parts:
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 93
diff changeset
317 # 1. with name
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 93
diff changeset
318 (r"(?i)\\end(?:[_\-]|(?:[ \t]+))?("
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
319 r"(?:prog(?:ram)?)"
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
320 r"|(?:algo(?:rithm)?)"
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
321 r"|(?:proc(?:edure)?)"
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
322 r"|(?:func(?:tion)?)"
96
98cd0787f62f FIX: FN alias for function
Franz Glasner <fzglas.hg@dom66.de>
parents: 94
diff changeset
323 r"|(?:fn)"
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
324 r"|(?:class)"
94
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 93
diff changeset
325 r")(?:[_\-]|(?:[\t ]+))?(\{)",
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
326 bygroups(op_opt_end_translate(Keyword),
72
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
327 op_opt_ignore_or_fixed(Whitespace, " ")),
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
328 "entity-name-end"),
94
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 93
diff changeset
329 # 2. without name
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 93
diff changeset
330 # 3. AND keywords that do not allow a param (e.g. endif)
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 93
diff changeset
331 (r"(?i)\\end(?:[_\-]|(?:[ \t]+))?("
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
332 r"(?:prog(?:ram)?)"
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
333 r"|(?:algo(?:rithm)?)"
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
334 r"|(?:proc(?:edure)?)"
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
335 r"|(?:func(?:tion)?)"
96
98cd0787f62f FIX: FN alias for function
Franz Glasner <fzglas.hg@dom66.de>
parents: 94
diff changeset
336 r"|(?:fn)"
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
337 r"|(?:class)"
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
338 r"|(?:if)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
339 r"|(?:while)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
340 r"|(?:for)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
341 r"|(?:forall)"
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
342 r"|(?:loop)"
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
343 r")\b",
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
344 bygroups(op_opt_end_translate(Keyword))),
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
345 #
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
346 # A single begin or end that is never suppressed because
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
347 # it is supposed to be paired with begin
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
348 #
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
349 (r"(?i)\\(begin|end)\b",
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 96
diff changeset
350 bygroups(op_translate(Keyword))),
61
392745b66969 The "\IS" keyword expansion
Franz Glasner <fzglas.hg@dom66.de>
parents: 58
diff changeset
351 # Keywords
392745b66969 The "\IS" keyword expansion
Franz Glasner <fzglas.hg@dom66.de>
parents: 58
diff changeset
352 (r"(?i)\\("
392745b66969 The "\IS" keyword expansion
Franz Glasner <fzglas.hg@dom66.de>
parents: 58
diff changeset
353 r"(?:is)"
82
36a0ef76a6d7 Implement "\WITH"
Franz Glasner <fzglas.hg@dom66.de>
parents: 81
diff changeset
354 r"|(?:with)"
61
392745b66969 The "\IS" keyword expansion
Franz Glasner <fzglas.hg@dom66.de>
parents: 58
diff changeset
355 r")\b",
392745b66969 The "\IS" keyword expansion
Franz Glasner <fzglas.hg@dom66.de>
parents: 58
diff changeset
356 bygroups(op_translate(Keyword))),
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
357 include("expr"),
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
358 include("unicode-separators"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
359 include("unicode-other"),
91
feb41c8d72d2 Instead of "\s+" use "[^\S\n]+" because a \n is not wanted really in this match.
Franz Glasner <fzglas.hg@dom66.de>
parents: 90
diff changeset
360 (r"[^\S\n]+", Text),
160
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
361 # (r".", Generic.Error), # tolerance for errors
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
362 (r".", Error),
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
363 ],
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
364 "remark": [
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
365 (r"(?i)\\(remark|rem)\b(.*)$",
117
d84f1fd10e64 Allow to customize the symbol for a remark
Franz Glasner <fzglas.hg@dom66.de>
parents: 116
diff changeset
366 bygroups(op_remark, Comment.Single)),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
367 ],
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
368 "entity-name": [ # may be multiline
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
369 (r"[^\\}]+", Name.Entity),
77
1a5e192aa950 Move popping states consistently more to the begin
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
370 (r"\}", LexBase.op_ignore, "#pop"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
371 (r"\\\}", LexBase.op_fixed(Name.Entity, "}")),
76
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
372 (r"\\\\", LexBase.op_fixed(Name.Entity, "\\")),
162
11ce0903ff8b Yield lone backslash characters in expressions that not really escape anything as "Generic.Error" now.
Franz Glasner <fzglas.hg@dom66.de>
parents: 161
diff changeset
373 (r"\\", LexBase.op_fixed(Generic.Error, "\\")),
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
374 ],
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
375 "entity-name-end": [ # may be multiline -- suppressed if no_end
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 61
diff changeset
376 (r"[^\\}]+", op_opt_ignore(Name.Entity)),
77
1a5e192aa950 Move popping states consistently more to the begin
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
377 (r"\}", LexBase.op_ignore, "#pop"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
378 (r"\\\}", op_opt_ignore_or_fixed(Name.Entity, "}")),
76
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
379 (r"\\\\", op_opt_ignore_or_fixed(Name.Entity, "\\")),
162
11ce0903ff8b Yield lone backslash characters in expressions that not really escape anything as "Generic.Error" now.
Franz Glasner <fzglas.hg@dom66.de>
parents: 161
diff changeset
380 (r"\\", op_opt_ignore_or_fixed(Generic.Error, "\\")),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
381 ],
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
382 "expr": [
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
383 include("math-symbols"), # must be before punctuation
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
384 include("ascii-punctuation"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
385 include("unicode-punctuation"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
386 include("escaped-string-start"),
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
387 include("py-strings"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
388 include("py-numbers"),
149
380d2607d5c7 Implement the "\NAME" command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 124
diff changeset
389 (r"(?i)\\(call|name)[ \t]*(\{)", LexBase.op_ignore, "entity-name"),
113
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
390 (r"(?i)\\gets\b", op_gets),
281
ee512932d603 Make "\T" and "\E" aliases for "\TEXT" and "\EXPRESSION"
Franz Glasner <fzglas.hg@dom66.de>
parents: 276
diff changeset
391 (r"(?i)\\e(?:xpr(?:ession)?)?[ \t]*\{",
212
18553f595b34 Allow nested \TEXT and \EXPR commands.
Franz Glasner <fzglas.hg@dom66.de>
parents: 211
diff changeset
392 LexBase.op_ignore,
18553f595b34 Allow nested \TEXT and \EXPR commands.
Franz Glasner <fzglas.hg@dom66.de>
parents: 211
diff changeset
393 "block-expr"),
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
394 include("explicit-tokentype"),
281
ee512932d603 Make "\T" and "\E" aliases for "\TEXT" and "\EXPRESSION"
Franz Glasner <fzglas.hg@dom66.de>
parents: 276
diff changeset
395 (r"(?i)\\t(?:ext)?[ \t]*\{", LexBase.op_ignore, "text-statement"),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
396 include("remark"),
81
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
397 include("keyword-constants"),
93
84c0f761c836 Renamed "text-operators" to "word-operators" and allow more case-variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
398 include("word-operators"),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
399 include("math-builtins"),
52
5bfa9113d3c4 First tests with "py-name": names from the Python lexer
Franz Glasner <fzglas.hg@dom66.de>
parents: 41
diff changeset
400 include("py-name"),
70
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 65
diff changeset
401 ],
80
f487f0d322a5 Move "expr-in-braces" up to "expr".
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
402 "expr-in-braces": [
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
403 include("math-symbols"), # must be before punctuation
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
404 include("ascii-punctuation-in-braces"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
405 include("unicode-punctuation"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
406 include("escaped-string-start"),
80
f487f0d322a5 Move "expr-in-braces" up to "expr".
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
407 include("py-strings"),
f487f0d322a5 Move "expr-in-braces" up to "expr".
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
408 include("py-numbers"),
149
380d2607d5c7 Implement the "\NAME" command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 124
diff changeset
409 (r"(?i)\\(call|name)[ \t]*(\{)", LexBase.op_ignore, "entity-name"),
113
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
410 (r"(?i)\\gets\b", op_gets),
281
ee512932d603 Make "\T" and "\E" aliases for "\TEXT" and "\EXPRESSION"
Franz Glasner <fzglas.hg@dom66.de>
parents: 276
diff changeset
411 (r"(?i)\\e(?:xpr(?:ession)?)?[ \t]*\{",
212
18553f595b34 Allow nested \TEXT and \EXPR commands.
Franz Glasner <fzglas.hg@dom66.de>
parents: 211
diff changeset
412 LexBase.op_ignore,
18553f595b34 Allow nested \TEXT and \EXPR commands.
Franz Glasner <fzglas.hg@dom66.de>
parents: 211
diff changeset
413 "block-expr"),
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
414 include("explicit-tokentype"),
281
ee512932d603 Make "\T" and "\E" aliases for "\TEXT" and "\EXPRESSION"
Franz Glasner <fzglas.hg@dom66.de>
parents: 276
diff changeset
415 (r"(?i)\\t(?:ext)?[ \t]*\{", LexBase.op_ignore, "text-statement"),
80
f487f0d322a5 Move "expr-in-braces" up to "expr".
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
416 include("remark"),
81
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
417 include("keyword-constants"),
93
84c0f761c836 Renamed "text-operators" to "word-operators" and allow more case-variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
418 include("word-operators"),
80
f487f0d322a5 Move "expr-in-braces" up to "expr".
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
419 include("math-builtins"),
f487f0d322a5 Move "expr-in-braces" up to "expr".
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
420 include("py-name"),
f487f0d322a5 Move "expr-in-braces" up to "expr".
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
421 ],
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
422 "block-expr": [ # somewhat similar to "root"
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
423 (r"\}", LexBase.op_ignore, "#pop"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
424 (r"\n", Whitespace),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
425 include("expr-in-braces"),
76
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
426 (r"\\\\", LexBase.op_fixed(Text, "\\")),
162
11ce0903ff8b Yield lone backslash characters in expressions that not really escape anything as "Generic.Error" now.
Franz Glasner <fzglas.hg@dom66.de>
parents: 161
diff changeset
427 (r"\\", LexBase.op_fixed(Generic.Error, "\\")),
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
428 include("unicode-separators"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
429 include("unicode-other"),
91
feb41c8d72d2 Instead of "\s+" use "[^\S\n]+" because a \n is not wanted really in this match.
Franz Glasner <fzglas.hg@dom66.de>
parents: 90
diff changeset
430 (r"[^\S\n]+", Text),
160
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
431 (r".", Error),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
432 ],
78
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
433 "text-statement": [ # like block but default to text-mode
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
434 (r"[^\\}\n]+", Text),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
435 (r"\}", LexBase.op_ignore, "#pop"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
436 (r"\n", Whitespace),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
437 (r"\\\}", LexBase.op_fixed(Text, "}")),
281
ee512932d603 Make "\T" and "\E" aliases for "\TEXT" and "\EXPRESSION"
Franz Glasner <fzglas.hg@dom66.de>
parents: 276
diff changeset
438 (r"(?i)\\e(?:xpr(?:ession)?)?[ \t]*\{",
78
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
439 LexBase.op_ignore,
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
440 "block-expr"),
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
441 include("explicit-tokentype"),
281
ee512932d603 Make "\T" and "\E" aliases for "\TEXT" and "\EXPRESSION"
Franz Glasner <fzglas.hg@dom66.de>
parents: 276
diff changeset
442 (r"(?i)\\t(?:ext)?[ \t]*\{", LexBase.op_ignore, "#push"),
78
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
443 include("remark"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
444 (r"\\\\", LexBase.op_fixed(Text, "\\")),
162
11ce0903ff8b Yield lone backslash characters in expressions that not really escape anything as "Generic.Error" now.
Franz Glasner <fzglas.hg@dom66.de>
parents: 161
diff changeset
445 (r"\\", LexBase.op_fixed(Text, "\\")), # in text-mode: leave Text
160
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
446 (r".", Error),
78
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 77
diff changeset
447 ],
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
448 "math-builtins": [
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
449 (words(("sqrt", "pow", "cos", "sin", "tan", "arcos", "arcsin",
102
d8368294413a "min" and "max" as automatic mathematical operators
Franz Glasner <fzglas.hg@dom66.de>
parents: 100
diff changeset
450 "arctan", "arctan2", "mod", "exp", "ln", "log",
d8368294413a "min" and "max" as automatic mathematical operators
Franz Glasner <fzglas.hg@dom66.de>
parents: 100
diff changeset
451 "min", "max"),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
452 prefix=r"(?<!\.)",
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
453 suffix=r"\b"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
454 Name.Builtin),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
455 ],
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
456 "math-symbols": [
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
457 (r"<=>|<->|<-|->|=>|<=|>=|<>|!=|:=|=:|\?=", op_symbol(Operator)),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
458 (r"[!&<>=+\-*/%|~]", Operator), # ASCII
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
459 (u"[%s]" % (uniprops.Sm,), Operator), # other Unicode
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
460 ],
93
84c0f761c836 Renamed "text-operators" to "word-operators" and allow more case-variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
461 "word-operators": [
84c0f761c836 Renamed "text-operators" to "word-operators" and allow more case-variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
462 (words(("IN", "In", "in",
84c0f761c836 Renamed "text-operators" to "word-operators" and allow more case-variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
463 "IS", "Is", "is",
84c0f761c836 Renamed "text-operators" to "word-operators" and allow more case-variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
464 "AND", "And", "and",
84c0f761c836 Renamed "text-operators" to "word-operators" and allow more case-variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
465 "OR", "Or", "or",
84c0f761c836 Renamed "text-operators" to "word-operators" and allow more case-variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
466 "XOR", "Xor", "xor",
84c0f761c836 Renamed "text-operators" to "word-operators" and allow more case-variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
467 "NOT", "Not", "not"),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
468 prefix=r"(?<!\.)",
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
469 suffix=r"\b"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
470 Operator.Word),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
471 ],
81
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
472 "keyword-constants": [
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
473 (words(("True", "TRUE", "true", "False", "FALSE", "false",
103
af97692501ea FIX: Typo: "non" -> "none"
Franz Glasner <fzglas.hg@dom66.de>
parents: 102
diff changeset
474 "None", "NONE", "none", "Nil", "NIL", "nil",
81
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
475 "Null", "NULL", "null",
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
476 "Empty", "EMPTY", "empty"),
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
477 prefix=r"(?<!\.)",
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
478 suffix=r"\b"),
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
479 Keyword.Constant),
6d8b813fb296 Keyword constants like "True", "False", "Nil", "None", "Null" and "Empty"
Franz Glasner <fzglas.hg@dom66.de>
parents: 80
diff changeset
480 ],
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
481 "ascii-punctuation": [
159
4ee0b1536ea6 Handle runs of dots in expressions (., .., ..., ...., ...)
Franz Glasner <fzglas.hg@dom66.de>
parents: 149
diff changeset
482 (r"\.+", Punctuation),
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
483 (r"[{}:(),;[\]?@]", Punctuation),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
484 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
485 "ascii-punctuation-in-braces": [
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
486 #
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
487 # Like "punctuation" but needs an escaped curly brace for } because
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
488 # a single closing curly brace pops the current state here.
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
489 #
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
490 (r"\\\}", LexBase.op_fixed(Punctuation, "}")),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
491 (r"[{:(),;[\]?@]", Punctuation),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
492 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
493 "unicode-separators": [
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
494 (u"[%s]" % (uniprops.Zl,), Whitespace),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
495 (u"[%s]" % (uniprops.Zp,), Whitespace),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
496 (u"[%s]" % (uniprops.Zs,), Whitespace),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
497 ],
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
498 "unicode-punctuation": [
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
499 (u"[%s]" % (uniprops.Pc,), Punctuation),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
500 (u"[%s]" % (uniprops.Pd,), Punctuation),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
501 (u"[%s]" % (uniprops.Ps,), Punctuation),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
502 (u"[%s]" % (uniprops.Pe,), Punctuation),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
503 (u"[%s]" % (uniprops.Pi,), Punctuation),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
504 (u"[%s]" % (uniprops.Pf,), Punctuation),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
505 (u"[%s]" % (uniprops.Po,), Punctuation),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
506 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
507 "unicode-other": [
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
508 (u"[%s]" % (uniprops.Sc,), Text), # Currency
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
509 (u"[%s]" % (uniprops.So,), Text), # Other symbols
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
510 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
511 "escaped-string-start": [
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 104
diff changeset
512 (r"""\\(['"])""", bygroups(Punctuation)),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
513 ],
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
514 "explicit-tokentype": [
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
515 # All these REs are CASE-SENSITIVE!
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
516
285
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
517 #
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
518 # New extended (more flexible, allows escaping)
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
519 #
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
520 (r"""\\ttX[ \t]*\{(?P<type>[^}]+)\}[ \t]*\{""",
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
521 op_explicit_tokentype_ex_start,
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
522 "extended-explicit-tokentype"),
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
523
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
524 #
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
525 # Old variants
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
526 #
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
527
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
528 # Multiple characters possible, but no escaping!
276
397ed930a5ba Allow more separator characters for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
529 (r"""\\ttx\-(?P<type>[a-zA-Z0-9_-]+?)"""
397ed930a5ba Allow more separator characters for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
530 r"""(?P<sep>[/?.,:;%|=*+!\\$~"'#@_-])"""
282
90946f87d77d Rename RE group "character" -> "characters"
Franz Glasner <fzglas.hg@dom66.de>
parents: 281
diff changeset
531 r"""(?P<characters>(.|\n)+?)(?P=sep)""",
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
532 op_explicit_tokentype),
282
90946f87d77d Rename RE group "character" -> "characters"
Franz Glasner <fzglas.hg@dom66.de>
parents: 281
diff changeset
533 (r"\\ttx\-(?P<type>[a-zA-Z0-9_-]+?)\{(?P<characters>[^}]+?)\}",
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 85
diff changeset
534 op_explicit_tokentype),
282
90946f87d77d Rename RE group "character" -> "characters"
Franz Glasner <fzglas.hg@dom66.de>
parents: 281
diff changeset
535 (r"\\ttx\-(?P<type>[a-zA-Z0-9_-]+?)\((?P<characters>[^)]+?)\)",
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
536 op_explicit_tokentype),
282
90946f87d77d Rename RE group "character" -> "characters"
Franz Glasner <fzglas.hg@dom66.de>
parents: 281
diff changeset
537 (r"\\ttx\-(?P<type>[a-zA-Z0-9_-]+?)<(?P<characters>[^>]+?)>",
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
538 op_explicit_tokentype),
282
90946f87d77d Rename RE group "character" -> "characters"
Franz Glasner <fzglas.hg@dom66.de>
parents: 281
diff changeset
539 (r"\\ttx\-(?P<type>[a-zA-Z0-9_-]+?)\[(?P<characters>[^\]]+?)\]",
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
540 op_explicit_tokentype),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
541
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
542 # Every character is possible: no escaping needed!
282
90946f87d77d Rename RE group "character" -> "characters"
Franz Glasner <fzglas.hg@dom66.de>
parents: 281
diff changeset
543 (r"\\tt-(?P<type>[^/]+?)/(?P<characters>(?:.|\n))",
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
544 op_explicit_tokentype),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 83
diff changeset
545 ],
285
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
546 "extended-explicit-tokentype": [
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
547 (r"([^\\}]+)", op_explicit_tokentype_ex_value),
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
548 (r"\}", op_explicit_tokentype_ex_end, "#pop"),
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
549 (r"\\(\})", op_explicit_tokentype_ex_value),
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
550 (r"\\(\\)", op_explicit_tokentype_ex_value),
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
551 (r"\\", LexBase.op_fixed(Generic.Error, "\\")), # weak error
afbca50b7dc1 Implement an alternate syntax for "Explicit Token Types".
Franz Glasner <fzglas.hg@dom66.de>
parents: 283
diff changeset
552 ],
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
553 }
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
554
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
555 def __init__(self, **options):
116
9bfd87544902 Use arrows from Supplemental Arrows-A Unicode block where appropriate: better readability
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
556 self.no_end = pygments.util.get_bool_opt(
9bfd87544902 Use arrows from Supplemental Arrows-A Unicode block where appropriate: better readability
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
557 options, "no_end", default=False)
9bfd87544902 Use arrows from Supplemental Arrows-A Unicode block where appropriate: better readability
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
558 self.symbol_gets = options.get("gets", None)
9bfd87544902 Use arrows from Supplemental Arrows-A Unicode block where appropriate: better readability
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
559 if self.symbol_gets is None:
9bfd87544902 Use arrows from Supplemental Arrows-A Unicode block where appropriate: better readability
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
560 self.symbol_gets = self.SYMBOLS["<-"] # Default: "⟵" # U+27F5
117
d84f1fd10e64 Allow to customize the symbol for a remark
Franz Glasner <fzglas.hg@dom66.de>
parents: 116
diff changeset
561 self.symbol_remark = options.get("remark", None)
d84f1fd10e64 Allow to customize the symbol for a remark
Franz Glasner <fzglas.hg@dom66.de>
parents: 116
diff changeset
562 if self.symbol_remark is None:
d84f1fd10e64 Allow to customize the symbol for a remark
Franz Glasner <fzglas.hg@dom66.de>
parents: 116
diff changeset
563 self.symbol_remark = self.SYMBOLS["REM"] # Default: "▷" # U+25B7
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
564 LexBase.__init__(self, **options)
160
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
565 #
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
566 # Do this after calling the base because a "filters" option should be
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
567 # allowed to set **any** filter.
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
568 #
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
569 self.prohibit_raiseonerror_filter = pygments.util.get_bool_opt(
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
570 options, "prohibit_raiseonerror_filter", default=False)
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
571
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
572 def add_filter(self, filter_, **options):
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
573 #
161
Franz Glasner <fzglas.hg@dom66.de>
parents: 160
diff changeset
574 # May be called by Lexer.__init__ when
Franz Glasner <fzglas.hg@dom66.de>
parents: 160
diff changeset
575 # self.prohibit_raiseonerror_filter is not yet set.
Franz Glasner <fzglas.hg@dom66.de>
parents: 160
diff changeset
576 # This is intentionally so.
160
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
577 #
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
578 if getattr(self, "prohibit_raiseonerror_filter", False):
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
579 if filter_ in ("raiseonerror",):
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
580 return
b4028838e0c8 Implement lexer option "prohibit_raiseonerror_filter".
Franz Glasner <fzglas.hg@dom66.de>
parents: 159
diff changeset
581 LexBase.add_filter(self, filter_, **options)
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
582
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
583
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
584 class AlgPseudocodeLexer_DE(AlgPseudocodeLexer):
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
585
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
586 """
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
587
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
588 .. seealso::
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
589 - https://de.wikipedia.org/wiki/Pseudocode
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
590
122
e39ca08b0609 Move some TODOs into the class docstrings
Franz Glasner <fzglas.hg@dom66.de>
parents: 117
diff changeset
591 .. todo:: Complete german translations
e39ca08b0609 Move some TODOs into the class docstrings
Franz Glasner <fzglas.hg@dom66.de>
parents: 117
diff changeset
592
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
593 """
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
594
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
595 name = "AlgPseudocodeDE"
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
596 aliases = ["algpseudocode-de", "algpseudo-de"]
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
597 filenames = ["*.algpseudo-de", "*.algpseudocode-de"]
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
598
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
599 LANG = "de"
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
600 TRANSLATIONS = AlgPseudocodeLexer.TRANSLATIONS.copy()
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
601 TRANSLATIONS.update({
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
602 "PROG": "PROGRAMM",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
603 "PROGRAM": "PROGRAMM",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
604 "ALGO": "ALGORITHMUS",
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 38
diff changeset
605 "ALGORITHM": "ALGORITHMUS",
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
606 "PROC": "PROZEDUR",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
607 "PROCEDURE": "PROZEDUR",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
608 "FUNC": "FUNKTION",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
609 "FUNCTION": "FUNKTION",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
610 "FN": "FUNKTION",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
611 "CLASS": "KLASSE",
61
392745b66969 The "\IS" keyword expansion
Franz Glasner <fzglas.hg@dom66.de>
parents: 58
diff changeset
612 "IS": "IST",
82
36a0ef76a6d7 Implement "\WITH"
Franz Glasner <fzglas.hg@dom66.de>
parents: 81
diff changeset
613 "WITH": "MIT",
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
614 "IF": "WENN",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
615 "THEN": "DANN",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
616 "ELSE": "ANDERNFALLS",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
617 "ELSEIF": "ANDERNFALLS WENN",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
618 "ELSIF": "ANDERNFALLS WENN",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
619 "ELIF": "ANDERNFALLS WENN",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
620 # "DO": # XXX TBD # in WHILE WHILE ... DO # noqa # in WHILE ... DO
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
621 "WHILE": "SOLANGE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
622 "FOR": "FÜR",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
623 "FORALL": "FÜR ALLE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
624 "FROM": "VON",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
625 "TO": "BIS",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
626 "IN": "IN",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
627 "STEP": "SCHRITTWEITE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
628 # "LOOP": XXX TBD # noqa
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
629 "REPEAT": "WIEDERHOLE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
630 "UNTIL": "BIS",
113
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
631 # "RETURN": XXX TBD # noqa
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
632 "BEGIN": "START",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
633 "END": "ENDE",
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
634 })
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
635 END_TRANSLATIONS = AlgPseudocodeLexer.END_TRANSLATIONS.copy()
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
636 END_TRANSLATIONS.update({
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
637 "PROG": "ENDE DES PROGRAMMS",
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
638 "PROGRAM": "ENDE DES PROGRAMMS",
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
639 "ALGO": "ENDE DES ALGORITHMUS",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
640 "ALGORITHM": "ENDE DES ALGORITHMUS",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
641 "PROC": "ENDE DER PROZEDUR",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
642 "PROCEDURE": "ENDE DER PROZEDUR",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
643 "FUNC": "ENDE DER FUNKTION",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
644 "FUNCTION": "ENDE DER FUNKTION",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
645 "FN": "ENDE DER FUNKTION",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
646 "CLASS": "ENDE DER KLASSE",
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
647 "IF": "ENDE WENN",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
648 "WHILE": "ENDE SOLANGE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
649 "FOR": "ENDE FÜR",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
650 "FORALL": "ENDE FÜR ALLE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
651 # "LOOP": "ENDE XXX", # XXX TBD # noqa
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
652 })
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
653 DEFAULT_END_PREFIX = "ENDE VON "
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
654
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
655
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
656 class AlgPseudocodeLexer_FR(AlgPseudocodeLexer):
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
657
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
658 """
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
659
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
660 .. seealso::
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
661 - https://info.blaisepascal.fr/pseudo-code/
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
662 - https://fr.wikipedia.org/wiki/Pseudo-code
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
663 - https://fr.wikipedia.org/wiki/Structure_de_contr%C3%B4le
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
664
122
e39ca08b0609 Move some TODOs into the class docstrings
Franz Glasner <fzglas.hg@dom66.de>
parents: 117
diff changeset
665 .. todo:: Complete french translations
e39ca08b0609 Move some TODOs into the class docstrings
Franz Glasner <fzglas.hg@dom66.de>
parents: 117
diff changeset
666
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
667 """
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
668
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
669 name = "AlgPseudocodeFR"
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
670 aliases = ["algpseudocode-fr", "algpseudo-fr"]
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
671 filenames = ["*.algpseudo-fr", "*.algpseudocode-fr"]
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
672
109
49e5aa89095f FIX: LANG for the french lexer
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
673 LANG = "fr"
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
674 TRANSLATIONS = AlgPseudocodeLexer.TRANSLATIONS.copy()
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
675 TRANSLATIONS.update({
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
676 "PROG": "PROGRAMME",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
677 "PROGRAM": "PROGRAMME",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
678 "ALGO": "ALGORITHME",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
679 "ALGORITHM": "ALGORITHME",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
680 "PROC": "PROCÉDURE",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
681 "PROCEDURE": "PROCÉDURE",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
682 "FUNC": "FONCTION",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
683 "FUNCTION": "FOUNCTION",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
684 "FN": "FONCTION",
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
685 "CLASS": "CLASSE",
61
392745b66969 The "\IS" keyword expansion
Franz Glasner <fzglas.hg@dom66.de>
parents: 58
diff changeset
686 "IS": "EST",
82
36a0ef76a6d7 Implement "\WITH"
Franz Glasner <fzglas.hg@dom66.de>
parents: 81
diff changeset
687 "WITH": "AVEC",
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
688 "IF": "SI",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
689 "THEN": "ALORS",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
690 "ELSE": "SINON",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
691 "ELSEIF": "SINONSI",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
692 "ELSIF": "SINONSI",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
693 "ELIF": "SINONSI",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
694 "DO": "FAIRE", # as in in WHILE ... DO (not DO ... UNTIL)
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
695 "WHILE": "TANTQUE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
696 "FOR": "POUR",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
697 "FORALL": "POUR CHAQUE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
698 "FROM": "DE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
699 "TO": "JUSQU'À", # or just "À",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
700 "IN": "DANS", # as in FOR ... IN
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
701 "STEP": "PAR PAS DE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
702 "LOOP": "BOUCLE", # XXX FIXME???
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
703 "REPEAT": "RÉPÉTER",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
704 "UNTIL": "JUSQUACEQUE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
705 "RETURN": "RENVOYER",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
706 "BEGIN": "DÉBUT",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
707 "END": "FIN",
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
708 })
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
709 END_TRANSLATIONS = AlgPseudocodeLexer.END_TRANSLATIONS.copy()
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
710 END_TRANSLATIONS.update({
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
711 "PROG": "FIN DE PROGRAMME",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
712 "PROGRAM": "FIN DE PROGRAMME",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
713 "ALGO": "FIN D'ALGORITHME",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
714 "ALGORITHM": "FIN D'ALGORITHME",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
715 "PROC": "FIN DE PROCÉDURE",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
716 "PROCEDURE": "FIN DE PROCÉDURE",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
717 "FUNC": "FIN DE FONCTION",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
718 "FUNCTION": "FIN DE FOUNCTION",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
719 "FN": "FIN DE FONCTION",
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
720 "CLASS": "FIN DE CLASSE",
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
721 "SI": "FIN SI",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
722 "FOR": "FIN POUR",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
723 "FORALL": "FIN POUR CHAQUE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
724 "WHILE": "FIN TANTQUE",
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 111
diff changeset
725 "LOOP": "FIN BOUCLE",
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
726 })
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
727 DEFAULT_END_PREFIX = "FIN DE "