Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2

annotate tests/test_fr.py @ 160:b4028838e0c8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Implement lexer option "prohibit_raiseonerror_filter". Sphinx raises by default when an Error token is seen (by means of the "raiseonerror" filter that is applied by default to lexers in Sphinx). This option skips this and allows error locations to be seen and highlighted properly. While there convert most Generic.Error tokens to Error tokens because now they can be handled by a lexer with "prohibit_raiseonerror_filter=True".
author Franz Glasner <fzglas.hg@dom66.de>
date Fri, 08 May 2026 17:46:28 +0200
parents c5fa8ee08059
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
16
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 # SPDX-FileCopyrightText: © 2026 Franz Glasner
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # SPDX-License-Identifier: MIT
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6
28
de1f67eff9d5 Move the original French pseudocode lexter into a sub-module.
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
7 from _tsetup import FRLEXERFILENAME, FRLEXERCLASS
16
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9 import unittest
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
11 import pygments
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
12 import pygments.lexers
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
13 from pygments.token import Token
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
14
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
15 import _testhelper
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
16
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
17
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
18 class TestLoading(unittest.TestCase):
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
19
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
20 def test_load(self):
28
de1f67eff9d5 Move the original French pseudocode lexter into a sub-module.
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
21 pygments.lexers.load_lexer_from_file(FRLEXERFILENAME, FRLEXERCLASS)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
22
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
23
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
24 class TestFrLexer(unittest.TestCase, _testhelper.TokenAssertHelper):
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
25
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
26 def setUp(self):
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
27 self.lexer = pygments.lexers.load_lexer_from_file(
28
de1f67eff9d5 Move the original French pseudocode lexter into a sub-module.
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
28 FRLEXERFILENAME, FRLEXERCLASS)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
29
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
30 def test_types(self):
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
31 tokens = pygments.lex("huhu", self.lexer)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
32 self.assertEqual((Token.Name.Variable, "huhu"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
33 self.assertEqual((Token.Text.Whitespace, "\n"), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
34 self.assertRaises(StopIteration, next, tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
35
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
36 def test_op_replace(self):
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
37 tokens = pygments.lex("<= >= <> != <- -> => ^ < = > <-> <=>",
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
38 self.lexer)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
39 self.assertEqual((Token.Operator, "≤"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
40 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
41 self.assertEqual((Token.Operator, "≥"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
42 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
43 self.assertEqual((Token.Operator, "≠"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
44 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
20
3db60b64abf6 Allow also "!=" as inequality operator (in addition to "<>")
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
45 self.assertEqual((Token.Operator, "≠"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
46 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
47 self.assertEqual((Token.Operator, "←"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
48 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
49 self.assertEqual((Token.Operator, "→"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
50 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
51 self.assertEqual((Token.Operator, "⇒"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
52 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
53 self.assertEqual((Token.Operator, "↑"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
54 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
55 self.assertEqual((Token.Operator, "<"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
56 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
57 self.assertEqual((Token.Operator, "="), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
58 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
59 self.assertEqual((Token.Operator, ">"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
60 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
61 self.assertEqual((Token.Operator, "↔"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
62 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
63 self.assertEqual((Token.Operator, "⇔"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
64 self.assertEqual((Token.Text.Whitespace, "\n"), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
65 self.assertRaises(StopIteration, next, tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
66
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
67 def test_op_replace_list(self):
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
68 tokens = pygments.lex("<= >= <> != <- -> => ^ < = > <-> <=>",
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
69 self.lexer)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
70 self.assertTokenStreamEqual(
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
71 [("Token.Operator", "≤"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
72 ("Token.Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
73 ("Token.Operator", "≥"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
74 ("Token.Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
75 ("Token.Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
76 ("Token.Text.Whitespace", " "),
20
3db60b64abf6 Allow also "!=" as inequality operator (in addition to "<>")
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
77 ("Token.Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
78 ("Token.Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
79 ("Token.Operator", "←"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
80 ("Token.Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
81 ("Token.Operator", "→"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
82 ("Token.Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
83 ("Token.Operator", "⇒"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
84 ("Token.Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
85 ("Token.Operator", "↑"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
86 ("Token.Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
87 ("Token.Operator", "<"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
88 ("Token.Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
89 ("Token.Operator", "="),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
90 ("Token.Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
91 ("Token.Operator", ">"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
92 ("Token.Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
93 ("Token.Operator", "↔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
94 ("Token.Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
95 ("Token.Operator", "⇔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
96 ("Token.Text.Whitespace", "\n"),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
97 ],
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
98 tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
99 self.assertRaises(StopIteration, next, tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
100
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
101 def test_op_replace_list_with_short_names(self):
29
c5fa8ee08059 Put a flake8 configuration into the new file "tox.ini".
Franz Glasner <fzglas.hg@dom66.de>
parents: 28
diff changeset
102 tokens = pygments.lex("<= >= <> != <- -> => ^ < = > <-> <=> ",
c5fa8ee08059 Put a flake8 configuration into the new file "tox.ini".
Franz Glasner <fzglas.hg@dom66.de>
parents: 28
diff changeset
103 self.lexer)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
104 self.assertTokenStreamEqual(
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
105 [("Operator", "≤"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
106 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
107 ("Operator", "≥"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
108 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
109 ("Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
110 ("Text.Whitespace", " "),
20
3db60b64abf6 Allow also "!=" as inequality operator (in addition to "<>")
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
111 ("Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
112 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
113 ("Operator", "←"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
114 ("Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
115 ("Operator", "→"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
116 ("Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
117 ("Operator", "⇒"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
118 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
119 ("Operator", "↑"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
120 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
121 ("Operator", "<"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
122 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
123 ("Operator", "="),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
124 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
125 ("Operator", ">"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
126 ("Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
127 ("Operator", "↔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
128 ("Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
129 ("Operator", "⇔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
130 ("Text.Whitespace", " \n"),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
131 ],
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
132 tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
133 self.assertRaises(StopIteration, next, tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
134
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
135 def test_op_replace_complete(self):
29
c5fa8ee08059 Put a flake8 configuration into the new file "tox.ini".
Franz Glasner <fzglas.hg@dom66.de>
parents: 28
diff changeset
136 tokens = pygments.lex("<= >= <> != <- -> => ^ > = < <=> <-> ",
c5fa8ee08059 Put a flake8 configuration into the new file "tox.ini".
Franz Glasner <fzglas.hg@dom66.de>
parents: 28
diff changeset
137 self.lexer)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
138 self.assertTokenStreamEqualComplete(
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
139 [("Operator", "≤"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
140 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
141 ("Operator", "≥"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
142 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
143 ("Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
144 ("Text.Whitespace", " "),
20
3db60b64abf6 Allow also "!=" as inequality operator (in addition to "<>")
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
145 ("Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
146 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
147 ("Operator", "←"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
148 ("Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
149 ("Operator", "→"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
150 ("Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
151 ("Operator", "⇒"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
152 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
153 ("Operator", "↑"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
154 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
155 ("Operator", ">"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
156 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
157 ("Operator", "="),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
158 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
159 ("Operator", "<"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
160 ("Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
161 ("Operator", "⇔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
162 ("Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
163 ("Operator", "↔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
164 ("Text.Whitespace", " \n"),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
165 ],
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
166 tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
167
16
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
168
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
169 if __name__ == "__main__":
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
170 unittest.main()