Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2

annotate tests/test_algpseudo.py @ 78:abb2e8c65d0f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Implement "TEXTSTATEMENT"
author Franz Glasner <fzglas.hg@dom66.de>
date Thu, 30 Apr 2026 09:28:39 +0200
parents 27e12387154d
children c14e5fd4d193
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 # SPDX-FileCopyrightText: © 2026 Franz Glasner
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # SPDX-License-Identifier: MIT
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
7 from _tsetup import ALGLEXERFILENAME, ALGLEXERCLASS
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9 import unittest
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
11 import pygments
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
12 import pygments.lexers
66
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
13 import pygments.formatters
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
15 import _testhelper
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
16
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
17
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
18 class TestSnippets(unittest.TestCase, _testhelper.TokenAssertHelper):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
19
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
20 def setUp(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
21 self.lexer = pygments.lexers.load_lexer_from_file(
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
22 ALGLEXERFILENAME, ALGLEXERCLASS)
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
23
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
24 def test_lf(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
25 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
26 [("Text.Whitespace", "\n")],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
27 pygments.lex("\n", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
28
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
29 def test_protected_lf(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
30 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
31 [("Text", "\\\n")],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
32 pygments.lex("\\\n", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
33
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
34 def test_number_int(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
35 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
36 [("Number.Integer", "10"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
37 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
38 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
39 pygments.lex("10", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
40
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
41 def test_number_float_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
42 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
43 [("Number.Float", "3.1415926"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
44 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
45 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
46 pygments.lex("3.1415926", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
47
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
48 def test_number_float_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
49 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
50 [("Number.Float", "3.14e-12"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
51 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
52 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
53 pygments.lex("3.14e-12", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
54
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
55 def test_string_s_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
56 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
57 [("String.Single", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
58 ("String.Single", "HU"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
59 ("String.Single", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
60 ("String.Single", "HE HA"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
61 ("String.Escape", "\\'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
62 ("String.Single", "HO"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
63 ("String.Single", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
64 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
65 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
66 pygments.lex("""'HU"HE HA\\'HO'""", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
67
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
68 def test_string_s_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
69 self.assertTokenStreamEqual(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
70 [("String.Single", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
71 ("String.Single", "HUHU"),
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 41
diff changeset
72 ("Error", "\n"),
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
73 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
74 pygments.lex("'HUHU\nHEHE'", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
75
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
76 def test_string_ts_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
77 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
78 [("String.Single", "'''"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
79 ("String.Single", "HUHU HEHE"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
80 ("String.Single", "'''"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
81 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
82 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
83 pygments.lex("'''HUHU HEHE'''", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
84
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
85 def test_string_ts_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
86 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
87 [("String.Single", "'''"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
88 ("String.Single", "HI"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
89 ("String.Single", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
90 ("String.Single", "HU"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
91 ("String.Single", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
92 ("String.Single", "HE"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
93 ("String.Single", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
94 ("String.Single", "HA"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
95 ("String.Single", "'''"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
96 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
97 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
98 pygments.lex("""'''HI'HU\nHE"HA'''""", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
99
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
100 def test_string_d_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
101 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
102 [("String.Double", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
103 ("String.Double", 'HU'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
104 ("String.Double", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
105 ("String.Double", 'HE HA'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
106 ("String.Escape", '\\"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
107 ("String.Double", 'HO'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
108 ("String.Double", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
109 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
110 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
111 pygments.lex('''"HU'HE HA\\"HO"''', self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
112
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
113 def test_string_d_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
114 self.assertTokenStreamEqual(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
115 [("String.Double", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
116 ("String.Double", "HUHU"),
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 41
diff changeset
117 ("Error", "\n"),
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
118 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
119 pygments.lex('"HUHU\nHEHE"', self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
120
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
121 def test_string_td_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
122 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
123 [("String.Double", '"""'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
124 ("String.Double", 'HUHU HAHA'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
125 ("String.Double", '"""'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
126 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
127 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
128 pygments.lex('"""HUHU HAHA"""', self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
129
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
130 def test_string_td_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
131 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
132 [("String.Double", '"""'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
133 ("String.Double", 'HU'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
134 ("String.Double", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
135 ("String.Double", "HO"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
136 ("String.Double", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
137 ("String.Double", "HE"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
138 ("String.Double", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
139 ("String.Double", "HA"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
140 ("String.Double", '"""'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
141 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
142 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
143 pygments.lex('''"""HU"HO\nHE'HA"""''', self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
144
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
145 def test_proc(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
146 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
147 [("Keyword", "PROCEDURE"),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
148 ("Text.Whitespace", " "),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
149 ("Name.Entity", "the name"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
150 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
151 ],
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
152 pygments.lex("\\PROC {the name}", self.lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
153
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
154 def test_endproc(self):
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
155 self.assertTokenStreamEqualComplete(
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
156 [("Keyword", "END OF PROCEDURE"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
157 ("Text.Whitespace", "\n"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
158 ],
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
159 pygments.lex("\\END-PROCEDURE", self.lexer))
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
160
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
161 def test_endproc_with_entityname(self):
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
162 self.assertTokenStreamEqualComplete(
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
163 [("Keyword", "END OF PROCEDURE"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
164 ("Text.Whitespace", " "),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
165 ("Name.Entity", "the procedure name"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
166 ("Text.Whitespace", "\n"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
167 ],
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
168 pygments.lex("\\ENDPROCEDURE {the procedure name}", self.lexer))
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
169
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
170 def test_proc_de(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
171 lexer = pygments.lexers.load_lexer_from_file(
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
172 ALGLEXERFILENAME, "AlgPseudocodeLexer_DE")
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
173 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
174 [("Keyword", "PROZEDUR"),
69
b3dd247b27a5 FIX: Add forgotten tests for the new normalized highlighting
Franz Glasner <fzglas.hg@dom66.de>
parents: 66
diff changeset
175 ("Text.Whitespace", " "),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
176 ("Name.Entity", " also {nichtxs"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
177 ("Name.Entity", "}"),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
178 ("Name.Entity", " hier"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
179 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
180 ],
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
181 pygments.lex("\\PROC { also {nichtxs\\} hier}", lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
182
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
183 def test_function_1(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
184 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
185 [("Keyword", "FUNCTION"),
69
b3dd247b27a5 FIX: Add forgotten tests for the new normalized highlighting
Franz Glasner <fzglas.hg@dom66.de>
parents: 66
diff changeset
186 ("Text.Whitespace", " "),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
187 ("Name.Entity", "1"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
188 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
189 ],
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
190 pygments.lex("\\FUNC{1}", self.lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
191
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
192 def test_function_2(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
193 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
194 [("Keyword", "FUNCTION"),
69
b3dd247b27a5 FIX: Add forgotten tests for the new normalized highlighting
Franz Glasner <fzglas.hg@dom66.de>
parents: 66
diff changeset
195 ("Text.Whitespace", " "),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
196 ("Name.Entity", "line 1\nline 2\n"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
197 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
198 ],
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
199 pygments.lex("\\FUNCTION{line 1\nline 2\n}", self.lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
200
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
201 def test_function_3(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
202 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
203 [("Keyword", "FUNCTION"),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
204 ("Text.Whitespace", " "),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
205 ("Name.Entity", "\\"),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
206 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
207 ("Name.Entity", "}"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
208 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
209 ],
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
210 pygments.lex("\\FN {\\n\\}}", self.lexer))
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
211
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
212 def test_class(self):
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
213 self.assertTokenStreamEqualComplete(
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
214 [("Keyword", "CLASS"),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
215 ("Text.Whitespace", " "),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
216 ("Name.Entity", "\\"),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
217 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
218 ("Name.Entity", "}"),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
219 ("Text.Whitespace", "\n"),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
220 ],
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
221 pygments.lex("\\CLASS {\\n\\}}", self.lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
222
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
223 def test_class_fr(self):
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
224 lexer = pygments.lexers.load_lexer_from_file(
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
225 ALGLEXERFILENAME, "AlgPseudocodeLexer_FR")
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
226 self.assertTokenStreamEqualComplete(
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
227 [("Keyword", "CLASSE"),
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
228 ("Text.Whitespace", " "),
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
229 ("Name.Entity", "\\"),
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
230 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
231 ("Name.Entity", "}"),
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
232 ("Text.Whitespace", "\n"),
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
233 ],
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
234 pygments.lex("\\CLASS {\\n\\}}", lexer))
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
235
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
236 def test_class_de_with_noend_option(self):
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
237 lexer = pygments.lexers.load_lexer_from_file(
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
238 ALGLEXERFILENAME, "AlgPseudocodeLexer_DE", no_end="True")
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
239 self.assertTokenStreamEqualComplete(
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
240 [("Keyword", "KLASSE"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
241 ("Text.Whitespace", " "),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
242 ("Name.Entity", "\\"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
243 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
244 ("Name.Entity", "}"),
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
245 ("Text.Whitespace", "\n"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
246 ("Text.Whitespace", "\n"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
247 ],
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
248 pygments.lex("\\CLASS {\\n\\}}\n\\ENDCLASS", lexer))
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
249
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
250 def test_class_de_with_noend_option_and_name(self):
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
251 lexer = pygments.lexers.load_lexer_from_file(
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
252 ALGLEXERFILENAME, "AlgPseudocodeLexer_DE", no_end="True")
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
253 self.assertTokenStreamEqualComplete(
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
254 [("Keyword", "KLASSE"),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
255 ("Text.Whitespace", " "),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
256 ("Name.Entity", "\\"),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
257 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
258 ("Name.Entity", "}"),
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
259 ("Text.Whitespace", "\n"),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
260 ("Text.Whitespace", "\n"),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
261 ],
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
262 pygments.lex("\\CLASS {\\n\\}}\n\\ENDCLASS {end class}", lexer))
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
263
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
264 def test_remark_1(self):
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
265 self.assertTokenStreamEqualComplete(
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
266 [("Comment.Single", "▷"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
267 ("Comment.Single", " the remark"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
268 ("Text.Whitespace", "\n"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
269 ],
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
270 pygments.lex("\\REMARK the remark\n", self.lexer))
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
271
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
272 def test_remark_2(self):
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
273 self.assertTokenStreamEqualComplete(
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
274 [("Comment.Single", "▷"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
275 ("Comment.Single", " the remark 1"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
276 ("Text.Whitespace", "\n"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
277 ("Comment.Single", "▷"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
278 ("Comment.Single", " the remark 2"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
279 ("Text.Whitespace", "\n"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
280 ],
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
281 pygments.lex(
64
05c53e431c88 Simplify case-handling for "BLOCK" and "REMARK"
Franz Glasner <fzglas.hg@dom66.de>
parents: 62
diff changeset
282 """\\remArk the remark 1
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
283 \\Rem the remark 2
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
284 """, self.lexer))
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
285
70
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
286 def test_expr_and_text(self):
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
287 self.assertTokenStreamEqualComplete(
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
288 [("Name.Entity", "a"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
289 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
290 ("Text", "multiplied by"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
291 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
292 ("Name.Entity", "b"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
293 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
294 ("Text", "is"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
295 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
296 ("Number.Integer", "0"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
297 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
298 ("Text", "mod"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
299 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
300 ("Name.Entity", "p"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
301 ("Text.Whitespace", "\n"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
302 ],
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
303 pygments.lex(
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
304 r"""a \TEXT{multiplied by} b \text{is} 0 \text{mod} p""",
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
305 self.lexer))
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
306
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
307 def test_expr_and_text_with_escaped_characters(self):
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
308 self.assertTokenStreamEqualComplete(
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
309 [("Name.Entity", "a"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
310 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
311 ("Text", "multiplied by {escaped"),
72
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
312 ("Text", "}"),
70
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
313 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
314 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
315 ("Name.Entity", "b"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
316 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
317 ("Text", "is "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
318 ("Text", "\\"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
319 ("Text", " not"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
320 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
321 ("Number.Integer", "0"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
322 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
323 ("Text", "mod"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
324 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
325 ("Name.Entity", "p"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
326 ("Text.Whitespace", "\n"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
327 ],
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
328 pygments.lex(
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
329 r"""a \TEXT{multiplied by {escaped\} } b \text{is \ not} 0"""
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
330 r" \text{mod} p""",
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
331 self.lexer))
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
332
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
333 def test_punctuation(self):
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
334 self.assertTokenStreamEqualComplete(
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
335 [("Punctuation", "{"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
336 ("Punctuation", "}"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
337 ("Punctuation", ":"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
338 ("Punctuation", "("),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
339 ("Punctuation", ")"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
340 ("Punctuation", ","),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
341 ("Punctuation", ";"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
342 ("Punctuation", "["),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
343 ("Punctuation", "]"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
344 ("Text.Whitespace", "\n"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
345 ],
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
346 pygments.lex(r"{}:(),;[]", self.lexer))
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
347
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
348 def test_block_empty(self):
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
349 self.assertTokenStreamEqualComplete(
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
350 [("Text", "◆"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
351 ("Text.Whitespace", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
352 ("Text.Whitespace", "\n"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
353 ],
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
354 pygments.lex(r"\block{}", self.lexer))
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
355
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
356 def test_block_with_text(self):
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
357 self.assertTokenStreamEqualComplete(
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
358 [("Text", "◆"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
359 ("Text.Whitespace", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
360 ("Text", "a b c"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
361 ("Text.Whitespace", "\n"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
362 ],
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
363 pygments.lex(r"\block{\text{a b c}}", self.lexer))
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
364
76
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
365 def test_block_with_escaped_text(self):
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
366 self.assertTokenStreamEqualComplete(
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
367 [("Text", "◆"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
368 ("Text.Whitespace", " "),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
369 ("Text", "\\"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
370 ("Name.Entity", "text"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
371 ("Punctuation", "{"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
372 ("Name.Entity", "a"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
373 ("Text", " "),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
374 ("Name.Entity", "b"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
375 ("Text", " "),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
376 ("Name.Entity", "c"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
377 ("Punctuation", "}"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
378 ("Text.Whitespace", "\n"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
379 ],
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
380 pygments.lex(r"\block{\\text{a b c\}}", self.lexer))
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
381
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
382 def test_block(self):
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
383 self.assertTokenStreamEqualComplete(
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
384 [("Text", "◆"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
385 ("Text.Whitespace", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
386 ("Name.Entity", "a"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
387 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
388 ("Number.Float", "1.2"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
389 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
390 ("Punctuation", "{"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
391 ("Name.Entity", "x"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
392 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
393 ("Operator.Word", "in"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
394 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
395 ("Name.Entity", "X"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
396 ("Punctuation", "}"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
397 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
398 ("Name.Entity", "c"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
399 ("Text.Whitespace", "\n"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
400 ],
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
401 pygments.lex(r"\state{a 1.2 {x in X\} c}", self.lexer))
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
402
78
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
403 def test_tstate_empty(self):
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
404 self.assertTokenStreamEqualComplete(
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
405 [("Text", "▪"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
406 ("Text.Whitespace", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
407 ("Text.Whitespace", "\n"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
408 ],
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
409 pygments.lex(r"\tstate{}", self.lexer))
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
410
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
411 def test_tstate_with_expr(self):
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
412 self.assertTokenStreamEqualComplete(
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
413 [("Text", "▪"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
414 ("Text.Whitespace", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
415 ("Text", "a 1.2 "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
416 ("Name.Entity", "x"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
417 ("Text", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
418 ("Operator.Word", "in"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
419 ("Text", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
420 ("Name.Entity", "X"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
421 ("Punctuation", "}"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
422 ("Text", " c"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
423 ("Text.Whitespace", "\n"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
424 ],
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
425 pygments.lex(r"\tstate{a 1.2 \expr{x in X\}} c}", self.lexer))
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
426
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
427 def test_tstate_with_escaped_expr(self):
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
428 self.assertTokenStreamEqualComplete(
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
429 [("Text", "▪"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
430 ("Text.Whitespace", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
431 ("Text", "a 1.2 "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
432 ("Text", "\\"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
433 ("Text", "expr{x in X"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
434 ("Text", "}"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
435 ("Text", " c"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
436 ("Text.Whitespace", "\n"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
437 ],
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
438 pygments.lex(r"\tstate{a 1.2 \\expr{x in X\} c}", self.lexer))
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
439
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
440
66
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
441 class PygmentizeCompletely(unittest.TestCase):
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
442
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
443 def setUp(self):
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
444 self.lexer = pygments.lexers.load_lexer_from_file(
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
445 ALGLEXERFILENAME, ALGLEXERCLASS)
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
446
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
447 def test_pygmentize_html(self):
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
448 html_formatter = pygments.formatters.get_formatter_by_name("html")
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
449 highlighted = pygments.highlight(
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
450 '"""FOO"""',
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
451 self.lexer,
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
452 html_formatter,
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
453 outfile=None)
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
454 self.assertTrue(highlighted.startswith("""<div class="highlight">"""))
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
455
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
456 def test_pygmentize_latex(self):
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
457 latex_formatter = pygments.formatters.get_formatter_by_name("latex")
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
458 highlighted = pygments.highlight(
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
459 '"""FOO"""',
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
460 self.lexer,
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
461 latex_formatter,
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
462 outfile=None)
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
463 self.assertTrue(highlighted.startswith(r"""\begin{Verbatim}"""))
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
464
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
465
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
466 if __name__ == "__main__":
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
467 unittest.main()