Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2

annotate pygments_lexer_pseudocode2/bases.py @ 59:fe7fe4caf978

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
ADD: Dummy placeholder files for empty directories in Sphinx (_static, _templates)
author Franz Glasner <fzglas.hg@dom66.de>
date Mon, 27 Apr 2026 19:17:13 +0200
parents 661461fb4dfc
children 7153e945a3d6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 # SPDX-FileCopyrightText: © 2026 Franz Glasner
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # SPDX-License-Identifier: MIT
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 r"""Some common bases for the lexers."""
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7
56
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
8 __all__ = ["LexBase", "uni_name", "py_innerstring_rules", "py_name_rules"]
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10
52
5bfa9113d3c4 First tests with "py-name": names from the Python lexer
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
11 from pygments import unistring
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
12 from pygments.lexer import RegexLexer, combined, bygroups, include
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
13 from pygments.token import Error, Name, Number, String, Comment
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
15
56
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
16 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
17 # SPDX-SnippetBegin
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
18 # SPDX-License-Identifier: BSD-2-Clause
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
19 # SPDX-SnippetCopyrightText: Copyright 2006-2023 by the Pygments team
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
20 # SPDX-SnippetCopyrightText: Copyright 2026 by Franz Glasner
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
21 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
22
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
23 uni_name = "[%s][%s]*" % (unistring.xid_start, unistring.xid_continue)
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
24
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
25
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
26 def py_innerstring_rules(ttype):
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
27 return [
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
28 # the old style '%s' % (...) string formatting (still valid in Py3)
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
29 (r'%(\(\w+\))?[-#0 +]*([0-9]+|[*])?(\.([0-9]+|[*]))?'
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
30 '[hlL]?[E-GXc-giorsaux%]', String.Interpol),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
31 # the new style '{}'.format(...) string formatting
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
32 (r'\{'
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
33 r'((\w+)((\.\w+)|(\[[^\]]+\]))*)?' # field name
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
34 r'(\![sra])?' # conversion
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
35 r'(\:(.?[<>=\^])?[-+ ]?#?0?(\d+)?,?(\.\d+)?[E-GXb-gnosx%]?)?'
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
36 r'\}', String.Interpol),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
37 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
38 # backslashes, quotes and formatting signs must be parsed
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
39 # one at a time
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
40 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
41 (r'[^\\\'"%{\n]+', ttype),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
42 (r'[\'"\\]', ttype),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
43 # unhandled string formatting sign
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
44 (r'%|(\{{1,2})', ttype)
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
45 # newlines are an error (use "nl" state)
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
46 ]
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
47
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
48
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
49 def py_name_rules(ttype, deco_ttype=Name.Decorator):
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
50 return [
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
51 # We recognize decorator syntax here
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
52 (r'@' + uni_name, deco_ttype),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
53 #
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
54 # Python's new matrix multiplication operator:
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
55 # not used here in pseudocode
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
56 # (r'@', Operator),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
57 (uni_name, ttype),
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
58 ]
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
59
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
60 # SPDX-SnippetEnd
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
61
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
62
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
63 class LexBase(RegexLexer):
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
64
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
65 """A base that defines some common lexer states.
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
66
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
67 Default flags are not important.
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
68
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
69 """
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
70
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
71 tokens = {
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
72 #
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
73 # These states are borrowed from Pygment's Python lexer.
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
74 # Their names have been prefixed with `py-'.
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
75 #
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
76 # SPDX-SnippetBegin
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
77 # SPDX-License-Identifier: BSD-2-Clause
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
78 # SPDX-SnippetCopyrightText: Copyright 2006-2023 by the Pygments team
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
79 # SPDX-SnippetCopyrightText: Copyright 2026 by Franz Glasner
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
80 #
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
81 'py-numbers': [
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
82 (r'(\d(?:_?\d)*\.(?:\d(?:_?\d)*)?|(?:\d(?:_?\d)*)?\.\d(?:_?\d)*)'
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
83 r'([eE][+-]?\d(?:_?\d)*)?', Number.Float),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
84 (r'\d(?:_?\d)*[eE][+-]?\d(?:_?\d)*j?', Number.Float),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
85 (r'0[oO](?:_?[0-7])+', Number.Oct),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
86 (r'0[bB](?:_?[01])+', Number.Bin),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
87 (r'0[xX](?:_?[a-fA-F0-9])+', Number.Hex),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
88 (r'\d(?:_?\d)*', Number.Integer),
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
89 ],
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
90 'py-strings': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
91 # non-raw strings
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
92 ('([uU]?)(""")', bygroups(String.Affix, String.Double),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
93 combined('py-stringescape', 'py-tdqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
94 ("([uU]?)(''')", bygroups(String.Affix, String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
95 combined('py-stringescape', 'py-tsqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
96 ('([uU]?)(")', bygroups(String.Affix, String.Double),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
97 combined('py-stringescape', 'py-dqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
98 ("([uU]?)(')", bygroups(String.Affix, String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
99 combined('py-stringescape', 'py-sqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
100 # non-raw bytes
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
101 ('([bB])(""")', bygroups(String.Affix, String.Double),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
102 combined('py-bytesescape', 'py-tdqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
103 ("([bB])(''')", bygroups(String.Affix, String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
104 combined('py-bytesescape', 'py-tsqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
105 ('([bB])(")', bygroups(String.Affix, String.Double),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
106 combined('py-bytesescape', 'py-dqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
107 ("([bB])(')", bygroups(String.Affix, String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
108 combined('py-bytesescape', 'py-sqs')),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
109 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
110 'py-stringescape': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
111 (r'\\(N\{.*?\}|u[a-fA-F0-9]{4}|U[a-fA-F0-9]{8})', String.Escape),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
112 include('py-bytesescape')
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
113 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
114 'py-bytesescape': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
115 (r'\\([\\abfnrtv"\']|\n|x[a-fA-F0-9]{2}|[0-7]{1,3})',
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
116 String.Escape)
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
117 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
118 'py-dqs': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
119 (r'"', String.Double, '#pop'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
120 (r'\\\\|\\"|\\\n', String.Escape), # included here for raw strings
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
121 include('py-strings-double'),
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
122 (r'\n', Error), # added by fag
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
123 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
124 'py-sqs': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
125 (r"'", String.Single, '#pop'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
126 (r"\\\\|\\'|\\\n", String.Escape), # included here for raw strings
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
127 include('py-strings-single'),
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
128 (r'\n', Error), # added by fag
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
129 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
130 'py-tdqs': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
131 (r'"""', String.Double, '#pop'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
132 include('py-strings-double'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
133 (r'\n', String.Double)
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
134 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
135 'py-tsqs': [
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
136 (r"'''", String.Single, '#pop'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
137 include('py-strings-single'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
138 (r'\n', String.Single)
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
139 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
140 'py-strings-single': py_innerstring_rules(String.Single),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents: 33
diff changeset
141 'py-strings-double': py_innerstring_rules(String.Double),
56
661461fb4dfc Make the "py-name" rules parameterized: allow to provide the token type.
Franz Glasner <fzglas.hg@dom66.de>
parents: 52
diff changeset
142 'py-name': py_name_rules(Name.Entity),
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
143 # SPDX-SnippetEnd
42
193ee1eb5013 Document where "multiline-nested-comment" is taken from
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
144 # This snippet is from the Pygments' documentation "Write your own lexer"
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
145 'multiline-nested-comment': [
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
146 (r'[^*/]+', Comment.Multiline),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
147 (r'/\*', Comment.Multiline, '#push'),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
148 (r'\*/', Comment.Multiline, '#pop'),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
149 (r'[*/]', Comment.Multiline),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
150 ]
33
db1bc740a201 FIX: ADD: Forgotten bases.py.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
151 }