Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2

annotate tests/test_fr.py @ 55:baf4ed7ac81a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
FIX: Add DE and FR lexers also to Pygments' entry points
author Franz Glasner <fzglas.hg@dom66.de>
date Sun, 26 Apr 2026 19:19:55 +0200
parents c5fa8ee08059
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
16
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 # SPDX-FileCopyrightText: © 2026 Franz Glasner
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # SPDX-License-Identifier: MIT
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6
28
de1f67eff9d5 Move the original French pseudocode lexter into a sub-module.
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
7 from _tsetup import FRLEXERFILENAME, FRLEXERCLASS
16
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9 import unittest
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
11 import pygments
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
12 import pygments.lexers
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
13 from pygments.token import Token
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
14
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
15 import _testhelper
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
16
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
17
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
18 class TestLoading(unittest.TestCase):
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
19
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
20 def test_load(self):
28
de1f67eff9d5 Move the original French pseudocode lexter into a sub-module.
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
21 pygments.lexers.load_lexer_from_file(FRLEXERFILENAME, FRLEXERCLASS)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
22
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
23
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
24 class TestFrLexer(unittest.TestCase, _testhelper.TokenAssertHelper):
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
25
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
26 def setUp(self):
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
27 self.lexer = pygments.lexers.load_lexer_from_file(
28
de1f67eff9d5 Move the original French pseudocode lexter into a sub-module.
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
28 FRLEXERFILENAME, FRLEXERCLASS)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
29
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
30 def test_types(self):
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
31 tokens = pygments.lex("huhu", self.lexer)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
32 self.assertEqual((Token.Name.Variable, "huhu"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
33 self.assertEqual((Token.Text.Whitespace, "\n"), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
34 self.assertRaises(StopIteration, next, tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
35
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
36 def test_op_replace(self):
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
37 tokens = pygments.lex("<= >= <> != <- -> => ^ < = > <-> <=>",
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
38 self.lexer)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
39 self.assertEqual((Token.Operator, "≤"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
40 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
41 self.assertEqual((Token.Operator, "≥"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
42 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
43 self.assertEqual((Token.Operator, "≠"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
44 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
20
3db60b64abf6 Allow also "!=" as inequality operator (in addition to "<>")
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
45 self.assertEqual((Token.Operator, "≠"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
46 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
47 self.assertEqual((Token.Operator, "←"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
48 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
49 self.assertEqual((Token.Operator, "→"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
50 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
51 self.assertEqual((Token.Operator, "⇒"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
52 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
53 self.assertEqual((Token.Operator, "↑"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
54 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
55 self.assertEqual((Token.Operator, "<"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
56 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
57 self.assertEqual((Token.Operator, "="), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
58 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
59 self.assertEqual((Token.Operator, ">"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
60 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
61 self.assertEqual((Token.Operator, "↔"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
62 self.assertEqual((Token.Text.Whitespace, " "), next(tokens))
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
63 self.assertEqual((Token.Operator, "⇔"), next(tokens))
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
64 self.assertEqual((Token.Text.Whitespace, "\n"), next(tokens))
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
65 self.assertRaises(StopIteration, next, tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
66
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
67 def test_op_replace_list(self):
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
68 tokens = pygments.lex("<= >= <> != <- -> => ^ < = > <-> <=>",
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
69 self.lexer)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
70 self.assertTokenStreamEqual(
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
71 [("Token.Operator", "≤"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
72 ("Token.Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
73 ("Token.Operator", "≥"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
74 ("Token.Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
75 ("Token.Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
76 ("Token.Text.Whitespace", " "),
20
3db60b64abf6 Allow also "!=" as inequality operator (in addition to "<>")
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
77 ("Token.Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
78 ("Token.Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
79 ("Token.Operator", "←"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
80 ("Token.Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
81 ("Token.Operator", "→"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
82 ("Token.Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
83 ("Token.Operator", "⇒"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
84 ("Token.Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
85 ("Token.Operator", "↑"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
86 ("Token.Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
87 ("Token.Operator", "<"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
88 ("Token.Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
89 ("Token.Operator", "="),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
90 ("Token.Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
91 ("Token.Operator", ">"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
92 ("Token.Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
93 ("Token.Operator", "↔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
94 ("Token.Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
95 ("Token.Operator", "⇔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
96 ("Token.Text.Whitespace", "\n"),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
97 ],
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
98 tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
99 self.assertRaises(StopIteration, next, tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
100
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
101 def test_op_replace_list_with_short_names(self):
29
c5fa8ee08059 Put a flake8 configuration into the new file "tox.ini".
Franz Glasner <fzglas.hg@dom66.de>
parents: 28
diff changeset
102 tokens = pygments.lex("<= >= <> != <- -> => ^ < = > <-> <=> ",
c5fa8ee08059 Put a flake8 configuration into the new file "tox.ini".
Franz Glasner <fzglas.hg@dom66.de>
parents: 28
diff changeset
103 self.lexer)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
104 self.assertTokenStreamEqual(
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
105 [("Operator", "≤"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
106 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
107 ("Operator", "≥"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
108 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
109 ("Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
110 ("Text.Whitespace", " "),
20
3db60b64abf6 Allow also "!=" as inequality operator (in addition to "<>")
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
111 ("Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
112 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
113 ("Operator", "←"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
114 ("Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
115 ("Operator", "→"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
116 ("Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
117 ("Operator", "⇒"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
118 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
119 ("Operator", "↑"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
120 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
121 ("Operator", "<"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
122 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
123 ("Operator", "="),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
124 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
125 ("Operator", ">"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
126 ("Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
127 ("Operator", "↔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
128 ("Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
129 ("Operator", "⇔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
130 ("Text.Whitespace", " \n"),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
131 ],
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
132 tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
133 self.assertRaises(StopIteration, next, tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
134
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
135 def test_op_replace_complete(self):
29
c5fa8ee08059 Put a flake8 configuration into the new file "tox.ini".
Franz Glasner <fzglas.hg@dom66.de>
parents: 28
diff changeset
136 tokens = pygments.lex("<= >= <> != <- -> => ^ > = < <=> <-> ",
c5fa8ee08059 Put a flake8 configuration into the new file "tox.ini".
Franz Glasner <fzglas.hg@dom66.de>
parents: 28
diff changeset
137 self.lexer)
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
138 self.assertTokenStreamEqualComplete(
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
139 [("Operator", "≤"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
140 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
141 ("Operator", "≥"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
142 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
143 ("Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
144 ("Text.Whitespace", " "),
20
3db60b64abf6 Allow also "!=" as inequality operator (in addition to "<>")
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
145 ("Operator", "≠"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
146 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
147 ("Operator", "←"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
148 ("Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
149 ("Operator", "→"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
150 ("Text.Whitespace", " "),
21
5705ac8a4fd5 Arrow "=>" -> "⇒"
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
151 ("Operator", "⇒"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
152 ("Text.Whitespace", " "),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
153 ("Operator", "↑"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
154 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
155 ("Operator", ">"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
156 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
157 ("Operator", "="),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
158 ("Text.Whitespace", " "),
19
2e67c4eae6d9 Use an alternate replace algorithm with a dict
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
159 ("Operator", "<"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
160 ("Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
161 ("Operator", "⇔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
162 ("Text.Whitespace", " "),
23
bb2fd6d4ad69 More arrows
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
163 ("Operator", "↔"),
27
bf13850f21fb Use the more correct "Token.Text.Whitespace" for whitespace instead of "Token.Text"
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
164 ("Text.Whitespace", " \n"),
18
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
165 ],
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
166 tokens)
859ab8abce0a The first real tests about lexing with the Pygments API for the tests.
Franz Glasner <fzglas.hg@dom66.de>
parents: 16
diff changeset
167
16
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
168
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
169 if __name__ == "__main__":
b92f660f2461 Begin a test infrastructure: using unittest and flake8.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
170 unittest.main()