Mercurial > hgrepos > Python > libs > pygments-lexer-pseudocode2

annotate pygments_lexer_pseudocode2/uniprops.py @ 160:b4028838e0c8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Implement lexer option "prohibit_raiseonerror_filter". Sphinx raises by default when an Error token is seen (by means of the "raiseonerror" filter that is applied by default to lexers in Sphinx). This option skips this and allows error locations to be seen and highlighted properly. While there convert most Generic.Error tokens to Error tokens because now they can be handled by a lexer with "prohibit_raiseonerror_filter=True".
author Franz Glasner <fzglas.hg@dom66.de>
date Fri, 08 May 2026 17:46:28 +0200
parents cec52d83869a
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
105
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 # SPDX-FileCopyrightText: © 2026 Franz Glasner
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # SPDX-License-Identifier: MIT
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 r"""A somewhat changed variant of :mod:`pygments.unistring`.
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8 We handle ASCII characters mostly ourself.
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10 """
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
11
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
12 __all__ = []
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
13
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
15 import pygments.unistring
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
16
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
17
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
18 def _remove_ascii(s):
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
19 """Remove the characters in the ASCII range from `s` and return the
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
20 adjusted string.
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
21
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
22 Assumes that in `s` the ASCII chars are sorted before the Unicode
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
23 codepoints as in :mod:`pygments.unistring`.
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
24
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
25 """
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
26 idx = 0
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
27 while ord(s[idx]) < 0x80:
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
28 idx += 1
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
29 if idx > 0:
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
30 return s[idx:]
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
31 else:
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
32 # nothing changed
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
33 return s
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
34
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
35
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
36 Pc = _remove_ascii(pygments.unistring.Pc)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
37 Pd = _remove_ascii(pygments.unistring.Pd)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
38 Pe = _remove_ascii(pygments.unistring.Pe)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
39 Ps = _remove_ascii(pygments.unistring.Ps)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
40 Pi = _remove_ascii(pygments.unistring.Pi)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
41 Pf = _remove_ascii(pygments.unistring.Pf)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
42 Po = _remove_ascii(pygments.unistring.Po)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
43 Sc = _remove_ascii(pygments.unistring.Sc)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
44 So = _remove_ascii(pygments.unistring.So)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
45 Sm = _remove_ascii(pygments.unistring.Sm)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
46 Zl = _remove_ascii(pygments.unistring.Zl)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
47 Zp = _remove_ascii(pygments.unistring.Zp)
cec52d83869a Handle much more characters from the Unicode codeset in expressions.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
48 Zs = _remove_ascii(pygments.unistring.Zs)