Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2

annotate tests/test_algpseudo.py @ 113:9aea2fd5f037

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
The \GETS command. Its output is customizable with the lexer's "gets" option. It defaults to `←'.
author Franz Glasner <fzglas.hg@dom66.de>
date Tue, 05 May 2026 20:51:20 +0200
parents ec8767cc5493
children be50fe0687d6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 # SPDX-FileCopyrightText: © 2026 Franz Glasner
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # SPDX-License-Identifier: MIT
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
7 from _tsetup import ALGLEXERFILENAME, ALGLEXERCLASS
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
9 import sys
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10 import unittest
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
11
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
12 import pygments
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
13 import pygments.lexers
66
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
14 import pygments.formatters
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
15
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
16 import _testhelper
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
17
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
18
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
19 class TestSnippets(unittest.TestCase, _testhelper.TokenAssertHelper):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
20
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
21 def setUp(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
22 self.lexer = pygments.lexers.load_lexer_from_file(
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
23 ALGLEXERFILENAME, ALGLEXERCLASS)
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
24
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
25 def test_lf(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
26 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
27 [("Text.Whitespace", "\n")],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
28 pygments.lex("\n", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
29
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
30 def test_protected_lf(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
31 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
32 [("Text", "\\\n")],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
33 pygments.lex("\\\n", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
35 def test_number_int(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
36 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
37 [("Number.Integer", "10"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
38 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
39 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
40 pygments.lex("10", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
41
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
42 def test_number_float_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
43 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
44 [("Number.Float", "3.1415926"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
45 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
46 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
47 pygments.lex("3.1415926", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
48
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
49 def test_number_float_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
50 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
51 [("Number.Float", "3.14e-12"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
52 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
53 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
54 pygments.lex("3.14e-12", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
55
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
56 def test_string_s_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
57 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
58 [("String.Single", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
59 ("String.Single", "HU"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
60 ("String.Single", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
61 ("String.Single", "HE HA"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
62 ("String.Escape", "\\'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
63 ("String.Single", "HO"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
64 ("String.Single", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
65 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
66 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
67 pygments.lex("""'HU"HE HA\\'HO'""", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
68
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
69 def test_string_s_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
70 self.assertTokenStreamEqual(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
71 [("String.Single", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
72 ("String.Single", "HUHU"),
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 41
diff changeset
73 ("Error", "\n"),
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
74 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
75 pygments.lex("'HUHU\nHEHE'", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
76
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
77 def test_string_ts_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
78 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
79 [("String.Single", "'''"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
80 ("String.Single", "HUHU HEHE"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
81 ("String.Single", "'''"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
82 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
83 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
84 pygments.lex("'''HUHU HEHE'''", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
85
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
86 def test_string_ts_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
87 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
88 [("String.Single", "'''"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
89 ("String.Single", "HI"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
90 ("String.Single", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
91 ("String.Single", "HU"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
92 ("String.Single", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
93 ("String.Single", "HE"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
94 ("String.Single", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
95 ("String.Single", "HA"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
96 ("String.Single", "'''"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
97 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
98 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
99 pygments.lex("""'''HI'HU\nHE"HA'''""", self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
100
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
101 def test_string_d_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
102 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
103 [("String.Double", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
104 ("String.Double", 'HU'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
105 ("String.Double", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
106 ("String.Double", 'HE HA'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
107 ("String.Escape", '\\"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
108 ("String.Double", 'HO'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
109 ("String.Double", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
110 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
111 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
112 pygments.lex('''"HU'HE HA\\"HO"''', self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
113
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
114 def test_string_d_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
115 self.assertTokenStreamEqual(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
116 [("String.Double", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
117 ("String.Double", "HUHU"),
43
bbef0ac6cfcf FIX: Proper string parsing: explicitely error out when single-line strings contain a linefeed
Franz Glasner <fzglas.hg@dom66.de>
parents: 41
diff changeset
118 ("Error", "\n"),
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
119 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
120 pygments.lex('"HUHU\nHEHE"', self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
121
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
122 def test_string_td_1(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
123 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
124 [("String.Double", '"""'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
125 ("String.Double", 'HUHU HAHA'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
126 ("String.Double", '"""'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
127 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
128 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
129 pygments.lex('"""HUHU HAHA"""', self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
130
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
131 def test_string_td_2(self):
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
132 self.assertTokenStreamEqualComplete(
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
133 [("String.Double", '"""'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
134 ("String.Double", 'HU'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
135 ("String.Double", '"'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
136 ("String.Double", "HO"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
137 ("String.Double", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
138 ("String.Double", "HE"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
139 ("String.Double", "'"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
140 ("String.Double", "HA"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
141 ("String.Double", '"""'),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
142 ("Text.Whitespace", "\n"),
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
143 ],
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
144 pygments.lex('''"""HU"HO\nHE'HA"""''', self.lexer))
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
145
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
146 def test_proc(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
147 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
148 [("Keyword", "PROCEDURE"),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
149 ("Text.Whitespace", " "),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
150 ("Name.Entity", "the name"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
151 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
152 ],
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
153 pygments.lex("\\PROC {the name}", self.lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
154
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
155 def test_proc_with_symbols_in_name(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
156 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
157 [("Keyword", "PROCEDURE"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
158 ("Text.Whitespace", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
159 ("Name.Entity", "the name sqrt ! <="),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
160 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
161 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
162 pygments.lex("\\PROC {the name sqrt ! <=}", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
163
79
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
164 def test_proc_with_escape_in_name(self):
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
165 self.assertTokenStreamEqualComplete(
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
166 [("Keyword", "PROCEDURE"),
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
167 ("Text.Whitespace", " "),
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
168 ("Name.Entity", "t"),
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
169 ("Name.Entity", "\\"),
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
170 ("Name.Entity", "he "),
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
171 ("Name.Entity", "\\"),
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
172 ("Name.Entity", "}"),
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
173 ("Name.Entity", "name"),
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
174 ("Text.Whitespace", "\n"),
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
175 ],
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
176 pygments.lex(r"\PROC {t\he \\\}name}", self.lexer))
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
177
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
178 def test_endproc(self):
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
179 self.assertTokenStreamEqualComplete(
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
180 [("Keyword", "END OF PROCEDURE"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
181 ("Text.Whitespace", "\n"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
182 ],
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
183 pygments.lex("\\END-PROCEDURE", self.lexer))
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
184
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
185 def test_endproc_with_entityname(self):
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
186 self.assertTokenStreamEqualComplete(
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
187 [("Keyword", "END OF PROCEDURE"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
188 ("Text.Whitespace", " "),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
189 ("Name.Entity", "the procedure name"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
190 ("Text.Whitespace", "\n"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
191 ],
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
192 pygments.lex("\\ENDPROCEDURE {the procedure name}", self.lexer))
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
193
94
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
194 def test_endproc_with_entityname_2(self):
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
195 self.assertTokenStreamEqualComplete(
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
196 [("Keyword", "END OF PROCEDURE"),
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
197 ("Text.Whitespace", " "),
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
198 ("Name.Entity", "the procedure name"),
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
199 ("Text.Whitespace", "\n"),
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
200 ],
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
201 pygments.lex("\\END-PROC {the procedure name}", self.lexer))
dc17b778a52b Refactor \END variants
Franz Glasner <fzglas.hg@dom66.de>
parents: 92
diff changeset
202
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
203 def test_endproc_with_entityname_3(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
204 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
205 [("Keyword", "END OF PROCEDURE"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
206 ("Text.Whitespace", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
207 ("Name.Entity", "the procedure name with pow and symbols ! <= "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
208 ("Name.Entity", "}"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
209 ("Name.Entity", "<-"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
210 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
211 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
212 pygments.lex(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
213 "\\END-PROC"
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
214 " {the procedure name with pow and symbols ! <= \\}<-}",
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
215 self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
216
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
217 def test_proc_de(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
218 lexer = pygments.lexers.load_lexer_from_file(
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
219 ALGLEXERFILENAME, "AlgPseudocodeLexer_DE")
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
220 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
221 [("Keyword", "PROZEDUR"),
69
b3dd247b27a5 FIX: Add forgotten tests for the new normalized highlighting
Franz Glasner <fzglas.hg@dom66.de>
parents: 66
diff changeset
222 ("Text.Whitespace", " "),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
223 ("Name.Entity", " also {nichtxs"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
224 ("Name.Entity", "}"),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
225 ("Name.Entity", " hier"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
226 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
227 ],
79
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
228 pygments.lex(r"\PROC { also {nichtxs\} hier}", lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
229
112
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
230 def test_loop_de_not_yet_translated(self):
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
231 lexer = pygments.lexers.load_lexer_from_file(
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
232 ALGLEXERFILENAME, "AlgPseudocodeLexer_DE")
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
233 self.assertTokenStreamEqualComplete(
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
234 [("Keyword", "LOOP"),
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
235 ("Text", " "),
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
236 ("Name.Entity", "a"),
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
237 ("Text.Whitespace", "\n"),
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
238 ],
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
239 pygments.lex(r"\LOOP a", lexer))
ec8767cc5493 More translations (FR and DE)
Franz Glasner <fzglas.hg@dom66.de>
parents: 108
diff changeset
240
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
241 def test_function_1(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
242 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
243 [("Keyword", "FUNCTION"),
69
b3dd247b27a5 FIX: Add forgotten tests for the new normalized highlighting
Franz Glasner <fzglas.hg@dom66.de>
parents: 66
diff changeset
244 ("Text.Whitespace", " "),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
245 ("Name.Entity", "1"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
246 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
247 ],
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
248 pygments.lex("\\FUNC{1}", self.lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
249
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
250 def test_function_2(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
251 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
252 [("Keyword", "FUNCTION"),
69
b3dd247b27a5 FIX: Add forgotten tests for the new normalized highlighting
Franz Glasner <fzglas.hg@dom66.de>
parents: 66
diff changeset
253 ("Text.Whitespace", " "),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
254 ("Name.Entity", "line 1\nline 2\n"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
255 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
256 ],
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
257 pygments.lex("\\FUNCTION{line 1\nline 2\n}", self.lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
258
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
259 def test_function_3(self):
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
260 self.assertTokenStreamEqualComplete(
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
261 [("Keyword", "FUNCTION"),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
262 ("Text.Whitespace", " "),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
263 ("Name.Entity", "\\"),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
264 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
265 ("Name.Entity", "}"),
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
266 ("Text.Whitespace", "\n"),
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
267 ],
79
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
268 pygments.lex(r"\FN {\n\}}", self.lexer))
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
269
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
270 def test_class(self):
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
271 self.assertTokenStreamEqualComplete(
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
272 [("Keyword", "CLASS"),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
273 ("Text.Whitespace", " "),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
274 ("Name.Entity", "\\"),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
275 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
276 ("Name.Entity", "}"),
40
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
277 ("Text.Whitespace", "\n"),
df08226a6984 Names for some entities (program, algorithm, function, procedure, class).
Franz Glasner <fzglas.hg@dom66.de>
parents: 39
diff changeset
278 ],
79
c14e5fd4d193 More tests with escape characters
Franz Glasner <fzglas.hg@dom66.de>
parents: 78
diff changeset
279 pygments.lex(r"\CLASS {\n\}}", self.lexer))
35
d9a3551a1038 Basics of translating some keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 34
diff changeset
280
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
281 def test_class_fr(self):
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
282 lexer = pygments.lexers.load_lexer_from_file(
53
39151225fb84 Rename the new pseudocode implementation to AlgPseudocode.
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
283 ALGLEXERFILENAME, "AlgPseudocodeLexer_FR")
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
284 self.assertTokenStreamEqualComplete(
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
285 [("Keyword", "CLASSE"),
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
286 ("Text.Whitespace", " "),
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
287 ("Name.Entity", "\\"),
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
288 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
289 ("Name.Entity", "}"),
41
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
290 ("Text.Whitespace", "\n"),
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
291 ],
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
292 pygments.lex("\\CLASS {\\n\\}}", lexer))
4ccf9a8d0bf2 For the german and french PseudocodeLexer: Use en translations as fallback.
Franz Glasner <fzglas.hg@dom66.de>
parents: 40
diff changeset
293
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
294 def test_class_de_with_noend_option(self):
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
295 lexer = pygments.lexers.load_lexer_from_file(
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
296 ALGLEXERFILENAME, "AlgPseudocodeLexer_DE", no_end="True")
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
297 self.assertTokenStreamEqualComplete(
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
298 [("Keyword", "KLASSE"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
299 ("Text.Whitespace", " "),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
300 ("Name.Entity", "\\"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
301 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
302 ("Name.Entity", "}"),
57
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
303 ("Text.Whitespace", "\n"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
304 ("Text.Whitespace", "\n"),
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
305 ],
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
306 pygments.lex("\\CLASS {\\n\\}}\n\\ENDCLASS", lexer))
e8f4af9e20a8 Some "ENDxxx" commands
Franz Glasner <fzglas.hg@dom66.de>
parents: 53
diff changeset
307
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
308 def test_class_de_with_noend_option_and_name(self):
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
309 lexer = pygments.lexers.load_lexer_from_file(
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
310 ALGLEXERFILENAME, "AlgPseudocodeLexer_DE", no_end="True")
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
311 self.assertTokenStreamEqualComplete(
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
312 [("Keyword", "KLASSE"),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
313 ("Text.Whitespace", " "),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
314 ("Name.Entity", "\\"),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
315 ("Name.Entity", "n"),
74
c1357674622d Remove an escaping backslash where appropriate
Franz Glasner <fzglas.hg@dom66.de>
parents: 72
diff changeset
316 ("Name.Entity", "}"),
62
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
317 ("Text.Whitespace", "\n"),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
318 ("Text.Whitespace", "\n"),
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
319 ],
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
320 pygments.lex("\\CLASS {\\n\\}}\n\\ENDCLASS {end class}", lexer))
7153e945a3d6 Implement ignoring of \ENDxxx including its optional name parts
Franz Glasner <fzglas.hg@dom66.de>
parents: 57
diff changeset
321
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
322 def test_remark_1(self):
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
323 self.assertTokenStreamEqualComplete(
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
324 [("Comment.Single", "▷"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
325 ("Comment.Single", " the remark"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
326 ("Text.Whitespace", "\n"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
327 ],
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
328 pygments.lex("\\REMARK the remark\n", self.lexer))
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
329
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
330 def test_remark_2(self):
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
331 self.assertTokenStreamEqualComplete(
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
332 [("Comment.Single", "▷"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
333 ("Comment.Single", " the remark 1"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
334 ("Text.Whitespace", "\n"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
335 ("Comment.Single", "▷"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
336 ("Comment.Single", " the remark 2"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
337 ("Text.Whitespace", "\n"),
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
338 ],
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
339 pygments.lex(
64
05c53e431c88 Simplify case-handling for "BLOCK" and "REMARK"
Franz Glasner <fzglas.hg@dom66.de>
parents: 62
diff changeset
340 """\\remArk the remark 1
39
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
341 \\Rem the remark 2
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
342 """, self.lexer))
a3151d837258 Some basic keywords for programs, algorithms, procedures and functions; also comments (single and multiline) and "remarks"
Franz Glasner <fzglas.hg@dom66.de>
parents: 35
diff changeset
343
108
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
344 def test_remark_in_text(self):
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
345 self.assertTokenStreamEqualComplete(
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
346 [("Text", "the text "),
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
347 ("Comment.Single", "▷"),
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
348 ("Comment.Single", " the remark"),
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
349 ("Text.Whitespace", "\n"),
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
350 ("Text", "the next text line"),
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
351 ("Text.Whitespace", "\n"),
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
352 ],
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
353 pygments.lex(
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
354 """\\TEXT{the text \\rem the remark\nthe next text line}""",
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
355 self.lexer))
6cebd3e7bc97 Also allow \REM within a \TEXT{}
Franz Glasner <fzglas.hg@dom66.de>
parents: 105
diff changeset
356
101
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
357 def test_comment_single_1(self):
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
358 self.assertTokenStreamEqualComplete(
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
359 [("Comment.Single", "// foo bar"),
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
360 ("Text.Whitespace", "\n"),
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
361 ],
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
362 pygments.lex(r"// foo bar", self.lexer))
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
363
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
364 def test_comment_single_2(self):
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
365 self.assertTokenStreamEqualComplete(
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
366 [("Comment.Single", "# foo bar"),
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
367 ("Text.Whitespace", "\n"),
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
368 ],
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
369 pygments.lex(r"# foo bar", self.lexer))
aae16e3624e2 Tests for single-line comments (// and #)
Franz Glasner <fzglas.hg@dom66.de>
parents: 97
diff changeset
370
70
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
371 def test_expr_and_text(self):
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
372 self.assertTokenStreamEqualComplete(
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
373 [("Name.Entity", "a"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
374 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
375 ("Text", "multiplied by"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
376 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
377 ("Name.Entity", "b"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
378 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
379 ("Text", "is"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
380 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
381 ("Number.Integer", "0"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
382 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
383 ("Text", "mod"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
384 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
385 ("Name.Entity", "p"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
386 ("Text.Whitespace", "\n"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
387 ],
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
388 pygments.lex(
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
389 r"""a \TEXT{multiplied by} b \text{is} 0 \text{mod} p""",
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
390 self.lexer))
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
391
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
392 def test_expr_and_text_with_escaped_characters(self):
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
393 self.assertTokenStreamEqualComplete(
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
394 [("Name.Entity", "a"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
395 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
396 ("Text", "multiplied by {escaped"),
72
206017a08ed7 Refactor: Make the "op_space()" and related methods more flexible and allow a given fixed token type and value
Franz Glasner <fzglas.hg@dom66.de>
parents: 70
diff changeset
397 ("Text", "}"),
70
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
398 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
399 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
400 ("Name.Entity", "b"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
401 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
402 ("Text", "is "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
403 ("Text", "\\"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
404 ("Text", " not"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
405 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
406 ("Number.Integer", "0"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
407 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
408 ("Text", "mod"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
409 ("Text", " "),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
410 ("Name.Entity", "p"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
411 ("Text.Whitespace", "\n"),
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
412 ],
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
413 pygments.lex(
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
414 r"""a \TEXT{multiplied by {escaped\} } b \text{is \ not} 0"""
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
415 r" \text{mod} p""",
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
416 self.lexer))
5517b0be67f0 Basic "\TEXT" escape for expressions
Franz Glasner <fzglas.hg@dom66.de>
parents: 69
diff changeset
417
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
418 def test_punctuation(self):
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
419 self.assertTokenStreamEqualComplete(
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
420 [("Punctuation", "{"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
421 ("Punctuation", "}"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
422 ("Punctuation", ":"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
423 ("Punctuation", "("),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
424 ("Punctuation", ")"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
425 ("Punctuation", ","),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
426 ("Punctuation", ";"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
427 ("Punctuation", "["),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
428 ("Punctuation", "]"),
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
429 ("Punctuation", "?"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
430 ("Punctuation", "@"),
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
431 ("Text.Whitespace", "\n"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
432 ],
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
433 pygments.lex(r"{}:(),;[]?@", self.lexer))
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
434
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
435 def test_block_empty(self):
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
436 self.assertTokenStreamEqualComplete(
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
437 [("Text", "◆"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
438 ("Text.Whitespace", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
439 ("Text.Whitespace", "\n"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
440 ],
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
441 pygments.lex(r"\block{}", self.lexer))
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
442
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
443 def test_block_with_text(self):
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
444 self.assertTokenStreamEqualComplete(
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
445 [("Text", "◆"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
446 ("Text.Whitespace", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
447 ("Text", "a b c"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
448 ("Text.Whitespace", "\n"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
449 ],
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
450 pygments.lex(r"\block{\text{a b c}}", self.lexer))
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
451
76
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
452 def test_block_with_escaped_text(self):
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
453 self.assertTokenStreamEqualComplete(
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
454 [("Text", "◆"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
455 ("Text.Whitespace", " "),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
456 ("Text", "\\"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
457 ("Name.Entity", "text"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
458 ("Punctuation", "{"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
459 ("Name.Entity", "a"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
460 ("Text", " "),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
461 ("Name.Entity", "b"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
462 ("Text", " "),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
463 ("Name.Entity", "c"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
464 ("Punctuation", "}"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
465 ("Text.Whitespace", "\n"),
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
466 ],
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
467 pygments.lex(r"\block{\\text{a b c\}}", self.lexer))
27e12387154d FIX: Handle escape character more thoroughly: also explicitely handle "\\"
Franz Glasner <fzglas.hg@dom66.de>
parents: 75
diff changeset
468
75
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
469 def test_block(self):
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
470 self.assertTokenStreamEqualComplete(
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
471 [("Text", "◆"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
472 ("Text.Whitespace", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
473 ("Name.Entity", "a"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
474 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
475 ("Number.Float", "1.2"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
476 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
477 ("Punctuation", "{"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
478 ("Name.Entity", "x"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
479 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
480 ("Operator.Word", "in"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
481 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
482 ("Name.Entity", "X"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
483 ("Punctuation", "}"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
484 ("Text", " "),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
485 ("Name.Entity", "c"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
486 ("Text.Whitespace", "\n"),
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
487 ],
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
488 pygments.lex(r"\state{a 1.2 {x in X\} c}", self.lexer))
711f8d19e27a New implementation of "STATEMENT" (also aliased to "STATE" and "BLOCK").
Franz Glasner <fzglas.hg@dom66.de>
parents: 74
diff changeset
489
78
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
490 def test_tstate_empty(self):
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
491 self.assertTokenStreamEqualComplete(
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
492 [("Text", "▪"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
493 ("Text.Whitespace", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
494 ("Text.Whitespace", "\n"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
495 ],
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
496 pygments.lex(r"\tstate{}", self.lexer))
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
497
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
498 def test_tstate_with_expr(self):
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
499 self.assertTokenStreamEqualComplete(
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
500 [("Text", "▪"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
501 ("Text.Whitespace", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
502 ("Text", "a 1.2 "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
503 ("Name.Entity", "x"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
504 ("Text", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
505 ("Operator.Word", "in"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
506 ("Text", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
507 ("Name.Entity", "X"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
508 ("Punctuation", "}"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
509 ("Text", " c"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
510 ("Text.Whitespace", "\n"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
511 ],
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
512 pygments.lex(r"\tstate{a 1.2 \expr{x in X\}} c}", self.lexer))
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
513
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
514 def test_tstate_with_escaped_expr(self):
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
515 self.assertTokenStreamEqualComplete(
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
516 [("Text", "▪"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
517 ("Text.Whitespace", " "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
518 ("Text", "a 1.2 "),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
519 ("Text", "\\"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
520 ("Text", "expr{x in X"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
521 ("Text", "}"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
522 ("Text", " c"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
523 ("Text.Whitespace", "\n"),
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
524 ],
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
525 pygments.lex(r"\tstate{a 1.2 \\expr{x in X\} c}", self.lexer))
abb2e8c65d0f Implement "TEXTSTATEMENT"
Franz Glasner <fzglas.hg@dom66.de>
parents: 76
diff changeset
526
92
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
527 def test_text_in_expr(self):
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
528 self.assertTokenStreamEqualComplete(
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
529 [("Name.Entity", "first"),
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
530 ("Text", " "),
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
531 ("Name.Entity", "expression"),
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
532 ("Text", " "),
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
533 ("Name.Entity", "second"),
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
534 ("Text", " "),
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
535 ("Name.Entity", "expression"),
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
536 ("Text.Whitespace", "\n"),
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
537 ],
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
538 pygments.lex(
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
539 r"\TEXT{\EXPRESSION{first expression} "
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
540 r"\EXPR{second expression}}",
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
541 self.lexer))
3f37832c405d FIX: Wrong state for \TEXT command
Franz Glasner <fzglas.hg@dom66.de>
parents: 87
diff changeset
542
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
543 def test_explicit_extended_single_tokentype_1(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
544 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
545 [("Operator", "%"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
546 ("Text.Whitespace", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
547 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
548 pygments.lex(r"\ttx-o:%:", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
549
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
550 def test_explicit_extended_single_tokentype_2(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
551 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
552 [("Operator", "{"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
553 ("Text.Whitespace", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
554 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
555 pygments.lex(r"\ttx-o{{}", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
556
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
557 def test_explicit_extended_single_tokentype_3(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
558 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
559 [("Operator", "<"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
560 ("Text.Whitespace", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
561 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
562 pygments.lex(r"\ttx-o<<>", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
563
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
564 def test_explicit_extended_single_tokentype_4(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
565 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
566 [("Operator", "("),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
567 ("Text.Whitespace", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
568 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
569 pygments.lex(r"\ttx-o(()", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
570
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
571 def test_explicit_extended_multi_tokentype_1(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
572 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
573 [("Operator", "xxx in A"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
574 ("Text.Whitespace", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
575 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
576 pygments.lex(r"\ttx-o/xxx in A/", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
577
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
578 def test_explicit_extended_multi_tokentype_2(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
579 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
580 [("Operator", "xxx in B"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
581 ("Text.Whitespace", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
582 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
583 pygments.lex(r"\ttx-o{xxx in B}", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
584
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
585 def test_explicit_extended_multi_tokentype_3(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
586 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
587 [("Operator", "xxx in C"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
588 ("Text.Whitespace", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
589 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
590 pygments.lex(r"\ttx-o<xxx in C>", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
591
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
592 def test_explicit_extended_multi_tokentype_4(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
593 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
594 [("Operator", "xxx in D"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
595 ("Text.Whitespace", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
596 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
597 pygments.lex(r"\ttx-o(xxx in D)", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
598
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
599 def test_explicit_tokentype_simple_1(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
600 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
601 [("Operator", "}"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
602 ("Operator", "/"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
603 ("Text.Whitespace", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
604 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
605 pygments.lex(r"\tt-o/}\tt-o//", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
606
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
607 def test_explicit_tokentype_simple_2(self):
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
608 self.assertTokenStreamEqualComplete(
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
609 [("Operator", "\n"),
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
610 ("Operator", "/"),
95
aed3204723ac FIX: Test results because of changed whitespace handling
Franz Glasner <fzglas.hg@dom66.de>
parents: 94
diff changeset
611 ("Text", " "),
aed3204723ac FIX: Test results because of changed whitespace handling
Franz Glasner <fzglas.hg@dom66.de>
parents: 94
diff changeset
612 ("Text.Whitespace", "\n"),
84
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
613 ],
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
614 pygments.lex("\\tt-o/\n\\tt-o// ", self.lexer))
3ac1c4502ad0 Implement "\tt-XXX" and "ttx-XXX" for explicit token types.
Franz Glasner <fzglas.hg@dom66.de>
parents: 79
diff changeset
615
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
616 def test_explicit_tokentype_with_remark(self):
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
617 self.assertTokenStreamEqualComplete(
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
618 [("Operator", u"∈ ∌"),
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
619 ("Text", " "),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
620 ("Comment.Single", "▷"),
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
621 ("Comment.Single", u" ∈ ∌ as (ordinary) operators"),
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
622 ("Text.Whitespace", "\n"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
623 ],
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
624 pygments.lex(
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
625 u"""\\ttx-o<∈ ∌> \\rem ∈ ∌ as (ordinary) operators""",
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
626 self.lexer))
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
627
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
628 def test_explicit_tokentype_with_remark_2(self):
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
629 self.assertTokenStreamEqualComplete(
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
630 [("Operator", "new_operator"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
631 ("Text", " "),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
632 ("Comment.Single", "▷"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
633 ("Comment.Single", " a (synthesized) operator"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
634 ("Text.Whitespace", "\n"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
635 ],
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
636 pygments.lex(
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
637 r"""\ttx-o<new_operator> \rem a (synthesized) operator""",
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
638 self.lexer))
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
639
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
640 def test_explicit_tokentype_with_possibly_conflicting_parens(self):
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
641 self.assertTokenStreamEqualComplete(
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
642 [("Name.Function", u"∈_∌"),
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
643 ("Punctuation", "("),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
644 ("Name.Entity", "p1"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
645 ("Punctuation", ","),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
646 ("Text", " "),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
647 ("Name.Entity", "p2"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
648 ("Punctuation", ")"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
649 ("Text.Whitespace", "\n"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
650 ],
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
651 pygments.lex(
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
652 u"""\\ttx-nf<∈_∌>(p1, p2)""",
87
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
653 self.lexer))
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
654
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
655 def test_explicit_tokentype_with_possibly_conflicting_parens_2(self):
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
656 self.assertTokenStreamEqualComplete(
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
657 [("Name.Decorator", "a_Decorator"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
658 ("Punctuation", "("),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
659 ("Name.Entity", "p1"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
660 ("Punctuation", ","),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
661 ("Text", " "),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
662 ("Name.Entity", "p2"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
663 ("Punctuation", ")"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
664 ("Text", " "),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
665 ("Comment.Single", "▷"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
666 ("Comment.Single", " (Python) decorator"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
667 ("Text.Whitespace", "\n"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
668 ],
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
669 pygments.lex(
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
670 r"""\ttx-nd<a_Decorator>(p1, p2) \rem (Python) decorator""",
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
671 self.lexer))
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
672
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
673 def test_explicit_tokentype_with_possibly_conflicting_parens_3(self):
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
674 self.assertTokenStreamEqualComplete(
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
675 [("Name.Decorator", "a_Decorator"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
676 ("Punctuation", "("),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
677 ("Name.Entity", "p1"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
678 ("Punctuation", ","),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
679 ("Text", " "),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
680 ("Name.Entity", "p2"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
681 ("Punctuation", ")"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
682 ("Text", " "),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
683 ("Comment.Single", "▷"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
684 ("Comment.Single", " (Python) annotation"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
685 ("Text.Whitespace", "\n"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
686 ],
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
687 pygments.lex(
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
688 r"""\ttx-nd[a_Decorator](p1, p2) \rem (Python) annotation""",
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
689 self.lexer))
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
690
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
691 def test_explicit_tokentype_error(self):
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
692 self.assertTokenStreamEqualComplete(
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
693 [("Generic.Error", r"""\ttx-non-existing[a_Decorator]"""),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
694 ("Text.Whitespace", "\n"),
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
695 ],
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
696 pygments.lex(
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
697 r"""\ttx-non-existing[a_Decorator]""",
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
698 self.lexer))
d8ca835c74ea FIX: Erroneous parsing of \tt-XXX and \ttx-XXX:
Franz Glasner <fzglas.hg@dom66.de>
parents: 84
diff changeset
699
97
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
700 def test_end_combinations(self):
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
701 self.assertTokenStreamEqualComplete(
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
702 [("Keyword", "BEGIN"),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
703 ("Text", " "),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
704 ("Keyword", "END"),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
705 ("Text", " "),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
706 ("Keyword", "END OF FUNCTION"),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
707 ("Text", " "),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
708 ("Keyword", "END OF FUNCTION"),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
709 ("Text.Whitespace", " "),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
710 ("Name.Entity", "An End of a Function"),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
711 ("Text", " "),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
712 ("Keyword", "END OF FUNCTION"),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
713 ("Text.Whitespace", " "),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
714 ("Name.Entity", "The End of the Next Function"),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
715 ("Text", " "),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
716 ("Text.Whitespace", "\n"),
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
717 ],
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
718 pygments.lex(
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
719 r"""\begin \end \end-fn \end-fn {An End of a Function}"""
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
720 r""" \end fn {The End of the Next Function} """,
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
721 self.lexer))
dd4eb937485c A couple of new keywords
Franz Glasner <fzglas.hg@dom66.de>
parents: 95
diff changeset
722
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
723 def test_unicode_math_operator(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
724 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
725 [("Operator", u"∈"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
726 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
727 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
728 pygments.lex(u"∈", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
729
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
730 def test_ascii_math_operator(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
731 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
732 [("Operator", "="),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
733 ("Operator", "!"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
734 ("Operator", "&"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
735 ("Operator", "<"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
736 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
737 ("Operator", ">"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
738 ("Operator", "+"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
739 ("Operator", "-"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
740 ("Operator", "*"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
741 ("Operator", "/"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
742 ("Operator", "%"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
743 ("Operator", "|"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
744 ("Operator", "~"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
745 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
746 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
747 pygments.lex("=!&< >+-*/%|~", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
748
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
749 @unittest.skipIf(sys.version_info[0] <= 2, "Unicode issues on Python 2")
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
750 def test_ascii_math_operator_with_replacements(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
751 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
752 [("Operator", u"⇔"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
753 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
754 ("Operator", u"↔"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
755 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
756 ("Operator", u"←"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
757 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
758 ("Operator", u"→"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
759 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
760 ("Operator", u"⇒"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
761 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
762 ("Operator", u"≤"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
763 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
764 ("Operator", u"≥"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
765 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
766 ("Operator", u"≠"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
767 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
768 ("Operator", u"≠"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
769 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
770 ("Operator", u"∶="), # u"≔"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
771 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
772 ("Operator", u"=∶"), # u"≕"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
773 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
774 ("Operator", u"≟"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
775 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
776 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
777 pygments.lex(u"<=> <-> <- -> => <= >= <> != := =: ?=", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
778
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
779 def test_word_operators(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
780 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
781 [("Operator.Word", "IN"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
782 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
783 ("Operator.Word", "is"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
784 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
785 ("Operator.Word", "And"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
786 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
787 ("Operator.Word", "Or"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
788 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
789 ("Operator.Word", "XOR"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
790 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
791 ("Operator.Word", "not"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
792 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
793 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
794 pygments.lex("IN is And Or XOR not", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
795
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
796 def test_keyword_constants(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
797 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
798 [("Keyword.Constant", "true"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
799 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
800 ("Keyword.Constant", "FALSE"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
801 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
802 ("Keyword.Constant", "None"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
803 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
804 ("Keyword.Constant", "nil"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
805 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
806 ("Keyword.Constant", "NULL"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
807 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
808 ("Keyword.Constant", "Empty"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
809 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
810 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
811 pygments.lex("true FALSE None nil NULL Empty", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
812
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
813 def test_math_builtins(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
814 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
815 [("Name.Builtin", "sqrt"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
816 ("Punctuation", "("),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
817 ("Name.Entity", "Foo"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
818 ("Punctuation", ")"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
819 ("Punctuation", ";"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
820 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
821 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
822 pygments.lex("sqrt(Foo);", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
823
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
824 def test_math_builtins_nested(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
825 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
826 [("Name.Builtin", "pow"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
827 ("Punctuation", "("),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
828 ("Number.Integer", "2"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
829 ("Punctuation", ","),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
830 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
831 ("Number.Integer", "8"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
832 ("Punctuation", ")"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
833 ("Punctuation", ";"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
834 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
835 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
836 pygments.lex("\\text{\\expr{pow(2, 8);}}", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
837
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
838 def test_math_letters(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
839 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
840 [("Name.Entity", "a"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
841 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
842 ("Operator.Word", "in"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
843 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
844 ("Name.Entity", u"ℂ"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
845 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
846 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
847 pygments.lex(u"a in ℂ", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
848
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
849 def test_other_symbols(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
850 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
851 [("Name.Entity", "b"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
852 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
853 ("Operator.Word", "in"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
854 ("Text", " "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
855 ("Text", u"℀"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
856 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
857 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
858 pygments.lex(u"b in ℀", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
859
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
860 def test_escaped_string_start_1(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
861 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
862 [("Text", "flow "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
863 ("Name.Entity", "f"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
864 ("Punctuation", "'"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
865 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
866 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
867 pygments.lex(r"\TEXT{flow \expr{f\'}}", self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
868
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
869 def test_escaped_string_start_2(self):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
870 self.assertTokenStreamEqualComplete(
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
871 [("Text", "flow "),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
872 ("Name.Entity", "f"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
873 ("Punctuation", '"'),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
874 ("Text.Whitespace", "\n"),
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
875 ],
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
876 pygments.lex(r'\TEXT{flow \expr{f\"}}', self.lexer))
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 101
diff changeset
877
113
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
878 def test_gets_default(self):
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
879 self.assertTokenStreamEqualComplete(
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
880 [("Name.Entity", "a"),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
881 ("Text", " "),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
882 ("Operator", "←"),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
883 ("Text", " "),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
884 ("Number.Integer", "2"),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
885 ("Text.Whitespace", "\n"),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
886 ],
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
887 pygments.lex(r"a \gets 2", self.lexer))
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
888
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
889 def test_gets_custom(self):
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
890 lexer = pygments.lexers.load_lexer_from_file(
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
891 ALGLEXERFILENAME, ALGLEXERCLASS, gets="::=")
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
892 self.assertTokenStreamEqualComplete(
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
893 [("Name.Entity", "a"),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
894 ("Text", " "),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
895 ("Operator", "::="),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
896 ("Text", " "),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
897 ("Number.Float", "2.7"),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
898 ("Text.Whitespace", "\n"),
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
899 ],
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
900 pygments.lex(r"a \gets 2.7", lexer))
9aea2fd5f037 The \GETS command.
Franz Glasner <fzglas.hg@dom66.de>
parents: 112
diff changeset
901
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
902
66
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
903 class PygmentizeCompletely(unittest.TestCase):
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
904
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
905 def setUp(self):
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
906 self.lexer = pygments.lexers.load_lexer_from_file(
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
907 ALGLEXERFILENAME, ALGLEXERCLASS)
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
908
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
909 def test_pygmentize_html(self):
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
910 html_formatter = pygments.formatters.get_formatter_by_name("html")
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
911 highlighted = pygments.highlight(
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
912 '"""FOO"""',
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
913 self.lexer,
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
914 html_formatter,
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
915 outfile=None)
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
916 self.assertTrue(highlighted.startswith("""<div class="highlight">"""))
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
917
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
918 def test_pygmentize_latex(self):
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
919 latex_formatter = pygments.formatters.get_formatter_by_name("latex")
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
920 highlighted = pygments.highlight(
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
921 '"""FOO"""',
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
922 self.lexer,
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
923 latex_formatter,
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
924 outfile=None)
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
925 self.assertTrue(highlighted.startswith(r"""\begin{Verbatim}"""))
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
926
9ba1584d4b4a Also add a test that completely highlights a document with Pygments
Franz Glasner <fzglas.hg@dom66.de>
parents: 64
diff changeset
927
34
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
928 if __name__ == "__main__":
1f741934205e Begin a new Pseudocode lexer using numbers and strings from Python
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
929 unittest.main()