Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2

annotate pygments_lexer_pseudocode2/bases.py @ 56:661461fb4dfc

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Make the "py-name" rules parameterized: allow to provide the token type. For this to work the implemting function must be global, because the access to the class is not yet possible at construction time. So consistently make some previons LexBase members module globals. Make some LexBase members module globals consistently.
author Franz Glasner <fzglas.hg@dom66.de>
date Mon, 27 Apr 2026 12:37:27 +0200
parents 5bfa9113d3c4
children 7153e945a3d6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 # SPDX-FileCopyrightText: © 2026 Franz Glasner
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # SPDX-License-Identifier: MIT
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 r"""Some common bases for the lexers."""
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7
56
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
8 __all__ = ["LexBase", "uni_name", "py_innerstring_rules", "py_name_rules"]
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10
52
5bfa9113d3c4 First tests with "py-name": names from the Python lexer
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
11 from pygments import unistring
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
12 from pygments.lexer import RegexLexer, combined, bygroups, include
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
13 from pygments.token import Error, Name, Number, String, Comment
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
15
56
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
16 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
17 # SPDX-SnippetBegin
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
18 # SPDX-License-Identifier: BSD-2-Clause
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
19 # SPDX-SnippetCopyrightText: Copyright 2006-2023 by the Pygments team
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
20 # SPDX-SnippetCopyrightText: Copyright 2026 by Franz Glasner
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
21 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
22
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
23 uni_name = "[%s][%s]*" % (unistring.xid_start, unistring.xid_continue)
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
24
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
25
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
26 def py_innerstring_rules(ttype):
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
27 return [
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
28 # the old style '%s' % (...) string formatting (still valid in Py3)
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
29 (r'%(\(\w+\))?[-#0 +]*([0-9]+|[*])?(\.([0-9]+|[*]))?'
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
30 '[hlL]?[E-GXc-giorsaux%]', String.Interpol),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
31 # the new style '{}'.format(...) string formatting
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
32 (r'\{'
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
33 r'((\w+)((\.\w+)|(\[[^\]]+\]))*)?' # field name
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
34 r'(\![sra])?' # conversion
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
35 r'(\:(.?[<>=\^])?[-+ ]?#?0?(\d+)?,?(\.\d+)?[E-GXb-gnosx%]?)?'
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
36 r'\}', String.Interpol),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
37 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
38 # backslashes, quotes and formatting signs must be parsed
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
39 # one at a time
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
40 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
41 (r'[^\\\'"%{\n]+', ttype),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
42 (r'[\'"\\]', ttype),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
43 # unhandled string formatting sign
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
44 (r'%|(\{{1,2})', ttype)
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
45 # newlines are an error (use "nl" state)
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
46 ]
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
47
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
48
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
49 def py_name_rules(ttype, deco_ttype=Name.Decorator):
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
50 return [
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
51 # We recognize decorator syntax here
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
52 (r'@' + uni_name, deco_ttype),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
53 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
54 # Python's new matrix multiplication operator:
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
55 # not used here in pseudocode
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
56 # (r'@', Operator),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
57 (uni_name, ttype),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
58 ]
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
59
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
60 # SPDX-SnippetEnd
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
61
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
62
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
63 class LexBase(RegexLexer):
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
64
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
65 """A base that defines some common lexer states.
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
66
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
67 Default flags are not important.
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
68
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
69 """
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
70
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
71 tokens = {
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
72 #
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
73 # These states are borrowed from Pygment's Python lexer.
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
74 # Their names have been prefixed with `py-'.
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
75 #
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
76 # SPDX-SnippetBegin
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
77 # SPDX-License-Identifier: BSD-2-Clause
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
78 # SPDX-SnippetCopyrightText: Copyright 2006-2023 by the Pygments team
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
79 # SPDX-SnippetCopyrightText: Copyright 2026 by Franz Glasner
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
80 #
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
81 'py-numbers': [
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
82 (r'(\d(?:_?\d)*\.(?:\d(?:_?\d)*)?|(?:\d(?:_?\d)*)?\.\d(?:_?\d)*)'
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
83 r'([eE][+-]?\d(?:_?\d)*)?', Number.Float),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
84 (r'\d(?:_?\d)*[eE][+-]?\d(?:_?\d)*j?', Number.Float),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
85 (r'0[oO](?:_?[0-7])+', Number.Oct),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
86 (r'0[bB](?:_?[01])+', Number.Bin),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
87 (r'0[xX](?:_?[a-fA-F0-9])+', Number.Hex),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
88 (r'\d(?:_?\d)*', Number.Integer),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
89 ],
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
90 'py-strings': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
91 # non-raw strings
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
92 ('([uU]?)(""")', bygroups(String.Affix, String.Double),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
93 combined('py-stringescape', 'py-tdqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
94 ("([uU]?)(''')", bygroups(String.Affix, String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
95 combined('py-stringescape', 'py-tsqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
96 ('([uU]?)(")', bygroups(String.Affix, String.Double),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
97 combined('py-stringescape', 'py-dqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
98 ("([uU]?)(')", bygroups(String.Affix, String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
99 combined('py-stringescape', 'py-sqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
100 # non-raw bytes
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
101 ('([bB])(""")', bygroups(String.Affix, String.Double),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
102 combined('py-bytesescape', 'py-tdqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
103 ("([bB])(''')", bygroups(String.Affix, String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
104 combined('py-bytesescape', 'py-tsqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
105 ('([bB])(")', bygroups(String.Affix, String.Double),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
106 combined('py-bytesescape', 'py-dqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
107 ("([bB])(')", bygroups(String.Affix, String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
108 combined('py-bytesescape', 'py-sqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
109 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
110 'py-stringescape': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
111 (r'\\(N\{.*?\}|u[a-fA-F0-9]{4}|U[a-fA-F0-9]{8})', String.Escape),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
112 include('py-bytesescape')
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
113 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
114 'py-bytesescape': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
115 (r'\\([\\abfnrtv"\']|\n|x[a-fA-F0-9]{2}|[0-7]{1,3})',
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
116 String.Escape)
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
117 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
118 'py-dqs': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
119 (r'"', String.Double, '#pop'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
120 (r'\\\\|\\"|\\\n', String.Escape), # included here for raw strings
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
121 include('py-strings-double'),
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
122 (r'\n', Error), # added by fag
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
123 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
124 'py-sqs': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
125 (r"'", String.Single, '#pop'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
126 (r"\\\\|\\'|\\\n", String.Escape), # included here for raw strings
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
127 include('py-strings-single'),
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
128 (r'\n', Error), # added by fag
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
129 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
130 'py-tdqs': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
131 (r'"""', String.Double, '#pop'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
132 include('py-strings-double'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
133 (r'\n', String.Double)
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
134 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
135 'py-tsqs': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
136 (r"'''", String.Single, '#pop'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
137 include('py-strings-single'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
138 (r'\n', String.Single)
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
139 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
140 'py-strings-single': py_innerstring_rules(String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
141 'py-strings-double': py_innerstring_rules(String.Double),
56
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
142 'py-name': py_name_rules(Name.Entity),
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
143 # SPDX-SnippetEnd
42
193ee1eb5013 Document where "multiline-nested-comment" is taken from
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
144 # This snippet is from the Pygments' documentation "Write your own lexer"
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
145 'multiline-nested-comment': [
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
146 (r'[^*/]+', Comment.Multiline),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
147 (r'/\*', Comment.Multiline, '#push'),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
148 (r'\*/', Comment.Multiline, '#pop'),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
149 (r'[*/]', Comment.Multiline),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
150 ]
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
151 }