Mercurial > hgrepos > Python > apps > py-cutils

annotate cutils/util/__init__.py @ 376:54a6d4534ef4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Convert some further value checks from plain Python assertions into real checks
author Franz Glasner <fzglas.hg@dom66.de>
date Mon, 05 May 2025 13:04:49 +0200
parents bfe1160fbfd3
children 6d7659a709f2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
115
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
323
48430941c18c Adopt copyright and license wordings from https://reuse.software/faq/.
Franz Glasner <fzglas.hg@dom66.de>
parents: 307
diff changeset
3 # SPDX-FileCopyrightText: © 2020-2025 Franz Glasner
48430941c18c Adopt copyright and license wordings from https://reuse.software/faq/.
Franz Glasner <fzglas.hg@dom66.de>
parents: 307
diff changeset
4 # SPDX-License-Identifier: BSD-3-Clause
115
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 r"""Utility package.
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8 """
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9
196
0f4febf646f5 Prepare for more Python2/3 compatibility: everywhere import print_function and absolute_import
Franz Glasner <fzglas.hg@dom66.de>
parents: 188
diff changeset
10 from __future__ import print_function, absolute_import
0f4febf646f5 Prepare for more Python2/3 compatibility: everywhere import print_function and absolute_import
Franz Glasner <fzglas.hg@dom66.de>
parents: 188
diff changeset
11
0f4febf646f5 Prepare for more Python2/3 compatibility: everywhere import print_function and absolute_import
Franz Glasner <fzglas.hg@dom66.de>
parents: 188
diff changeset
12
164
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 163
diff changeset
13 __all__ = ["PY2",
173
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
14 "PY35",
188
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
15 "n", "b", "u",
164
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 163
diff changeset
16 "normalize_filename",
372
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
17 "escape_for_output",
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
18 "argv2algo",
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
19 "algotag2algotype",
307
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
20 "algotag2digest_size",
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
21 "get_blake2b",
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
22 "get_blake2b_256",
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
23 "get_blake2s",
172
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
24 "default_algotag",
163
fa7dd54e9715 FIX: Add "fsencode" to public functions in cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 124
diff changeset
25 "fsencode",
188
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
26 "interpolate_bytes",
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
27 ]
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
28
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
29
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
30 import argparse
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
31 import hashlib
124
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
32 import os
164
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 163
diff changeset
33 import sys
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 163
diff changeset
34
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 163
diff changeset
35
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 163
diff changeset
36 PY2 = sys.version_info[0] < 3
173
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
37 PY35 = sys.version_info[:2] >= (3, 5)
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
38
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
39
188
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
40 if PY2:
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
41
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
42 def n(s, encoding="ascii"):
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
43 """Convert `s` to the native string implementation"""
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
44 if isinstance(s, unicode): # noqa: F821 undefined name 'unicode'
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
45 return s.encode(encoding)
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
46 return s
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
47
372
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
48 def b(s, encoding="ascii", errors="strict"):
188
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
49 """Convert `s` to bytes"""
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
50 if isinstance(s, unicode): # noqa: F821 undefined name 'unicode'
372
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
51 return s.encode(encoding, errors)
188
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
52 return s
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
53
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
54 def u(s, encoding="ascii"):
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
55 """Convert `s` to a unicode string"""
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
56 if isinstance(s, str):
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
57 return s.decode(encoding)
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
58 return s
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
59
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
60 else:
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
61
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
62 def n(s, encoding="ascii"):
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
63 """Convert `s` to the native string implementation"""
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
64 if isinstance(s, (bytes, bytearray)):
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
65 return s.decode(encoding)
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
66 return s
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
67
372
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
68 def b(s, encoding="ascii", errors="strict"):
188
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
69 """Convert `s` to bytes"""
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
70 if isinstance(s, str):
372
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
71 return s.encode(encoding, errors)
188
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
72 return s
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
73
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
74 u = n
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
75
2784fdcc99e5 Implement basic parsing of treesum output.
Franz Glasner <fzglas.hg@dom66.de>
parents: 185
diff changeset
76
372
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
77 def escape_for_output(what):
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
78 """Escape `what` in such a way that the output can be safely written into
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
79 a line and/or column-oriented output file
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
80
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
81 """
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
82 if isinstance(what, bytes):
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
83 return (what.replace(b'\\', b"\\\\")
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
84 .replace(b'\n', b"\\x0a")
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
85 .replace(b'\r', b"\\x0d")
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
86 .replace(b'\t', b"\\x09"))
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
87 else:
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
88 return (what.replace(u'\\', u"\\\\")
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
89 .replace(u'\n', u"\\x0a")
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
90 .replace(u'\r', u"\\x0d")
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
91 .replace(u'\t', u"\\x09"))
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
92
bfe1160fbfd3 treesum: Make ERROR outputs more consistent: use native paths where possible
Franz Glasner <fzglas.hg@dom66.de>
parents: 365
diff changeset
93
172
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
94 def default_algotag():
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
95 """Determine the "best" default algorithm.
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
96
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
97 Depend on availability in :mod:`hashlib`.
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
98
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
99 Prefer BLAKE2b-256, SHA256 or SHA1 -- in this order.
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
100
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
101 Does not consider :mod:`pyblake2` if it is available eventually.
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
102
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
103 """
365
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
104 # Python <2.7.9 has no algorithms_available: fall back to algorithms
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
105 try:
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
106 algos = hashlib.algorithms_available
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
107 except AttributeError:
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
108 try:
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
109 algos = hashlib.algorithms
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
110 except AttributeError:
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
111 algos = []
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
112 if "blake2b" in algos:
172
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
113 return "BLAKE2b-256"
365
d5c920ace3cb treesum: FIX: Python<2.7.9 has no hashlib.algorithms_available: fall back to hashlib.algorithms in this case
Franz Glasner <fzglas.hg@dom66.de>
parents: 323
diff changeset
114 if "sha256" in algos:
172
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
115 return "SHA256"
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
116 return "SHA1"
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
117
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
118
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
119 def get_blake2b():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
120 """Get the factory for blake2b"""
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
121 try:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
122 return hashlib.blake2b
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
123 except AttributeError:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
124 import pyblake2
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
125 return pyblake2.blake2b
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
126
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
127
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
128 def get_blake2b_256():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
129 """Get the factory for blake2b-256"""
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
130
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
131 try:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
132 hashlib.blake2b
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
133 except AttributeError:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
134 import pyblake2
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
135
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
136 def _get_blake():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
137 return pyblake2.blake2b(digest_size=32)
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
138
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
139 else:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
140
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
141 def _get_blake():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
142 return hashlib.blake2b(digest_size=32)
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
143
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
144 return _get_blake
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
145
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
146
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
147 def get_blake2s():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
148 """Get the factory for blake2s"""
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
149 try:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
150 return hashlib.blake2s
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
151 except AttributeError:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
152 import pyblake2
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
153 return pyblake2.blake2s
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
154
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
155
184
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
156 def get_crc(name):
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
157 """Get the factory for a CRC"""
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
158
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
159 from ..crcmod.predefined import PredefinedCrc
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
160
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
161 def _crc_type():
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
162 return PredefinedCrc(name)
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
163
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
164 return _crc_type
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
165
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
166
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
167 def argv2algo(s):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
168 """Convert a command line algorithm specifier into a tuple with the
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
169 type/factory of the digest and the algorithms tag for output purposes.
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
170
172
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
171 :param str s: the specifier from the command line; should include all
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib
Franz Glasner <fzglas.hg@dom66.de>
parents: 166
diff changeset
172 algorithm tags also (for proper round-tripping)
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
173 :return: the internal digest specification
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
174 :rtype: a tuple (digest_type_or_factory, name_in_output)
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
175 :raises argparse.ArgumentTypeError: for unrecognized algorithms or names
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
176
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
177 String comparisons are done case-insensitively.
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
178
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
179 """
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
180 s = s.lower()
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
181 if s in ("1", "sha1"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
182 return (hashlib.sha1, "SHA1")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
183 elif s in ("224", "sha224"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
184 return (hashlib.sha224, "SHA224")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
185 elif s in ("256", "sha256"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
186 return (hashlib.sha256, "SHA256")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
187 elif s in ("384", "sha384"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
188 return (hashlib.sha384, "SHA384")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
189 elif s in ("512", "sha512"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
190 return (hashlib.sha512, "SHA512")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
191 elif s in ("3-224", "sha3-224"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
192 return (hashlib.sha3_224, "SHA3-224")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
193 elif s in ("3-256", "sha3-256"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
194 return (hashlib.sha3_256, "SHA3-256")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
195 elif s in ("3-384", "sha3-384"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
196 return (hashlib.sha3_384, "SHA3-384")
120
a548783381b6 More accurate description of --algorithm.
Franz Glasner <fzglas.hg@dom66.de>
parents: 118
diff changeset
197 elif s in ("3", "3-512", "sha3-512"):
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
198 return (hashlib.sha3_512, "SHA3-512")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
199 elif s in ("blake2b", "blake2b-512", "blake2", "blake2-512"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
200 return (get_blake2b(), "BLAKE2b")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
201 elif s in ("blake2s", "blake2s-256"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
202 return (get_blake2s(), "BLAKE2s")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
203 elif s in ("blake2-256", "blake2b-256"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
204 return (get_blake2b_256(), "BLAKE2b-256")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
205 elif s == "md5":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
206 return (hashlib.md5, "MD5")
184
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
207 elif s in ("crc24", "crc-24",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
208 "crc24-openpgp", "crc-24-openpgp"):
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
209 return (get_crc("crc-24"), "CRC-24")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
210 elif s in ("crc32", "crc-32",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
211 "crc32-pkzip", "crc-32-pkzip",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
212 "crc32-iso", "crc-32-iso",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
213 "crc32-iso-hdlc", "crc-32-iso-hdlc"):
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
214 return (get_crc("crc-32"), "CRC-32-ISO")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
215 elif s in ("crc32-posix", "crc-32-posix",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
216 "crc32-cksum", "crc-32-cksum",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
217 "posix"):
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
218 return (get_crc("posix"), "CRC-32-POSIX")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
219 elif s in ("crc64", "crc-64",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
220 "crc64-iso", "crc-64-iso"):
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
221 return (get_crc("crc-64"), "CRC-64-ISO")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
222 elif s in ("crc64-2", "crc-64-2",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
223 "crc64-iso-2", "crc-64-iso-2",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
224 "crc64-mcrc64", "crc-64-mcrc64"):
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
225 return (get_crc("crc-64-2"), "CRC-64-ISO-2")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
226 elif s in ("crc64-ecma", "crc-64-ecma"):
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
227 return (get_crc("crc-64-ecma"), "CRC-64-ECMA")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
228 elif s in ("crc64-xz", "crc-64-xz",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
229 "crc64-go-ecma", "crc-64-go-ecma"):
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
230 return (get_crc("crc-64-xz"), "CRC-64-XZ")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
231 elif s in ("crc64-go", "crc-64-go",
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
232 "crc64-go-iso", "crc-64-go-iso"):
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
233 return (get_crc("crc-64-go"), "CRC-64-GO-ISO")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
234 elif s in ("crc64-redis", "crc-64-redis"):
185
f04d4b1c14b3 FIX: "digest" had overwritteh the digest module
Franz Glasner <fzglas.hg@dom66.de>
parents: 184
diff changeset
235 return (get_crc("crc-64-redis"), "CRC-64-REDIS")
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
236 else:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
237 raise argparse.ArgumentTypeError(
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
238 "`{}' is not a recognized algorithm".format(s))
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
239
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
240
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
241 def algotag2algotype(s):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
242 """Convert the algorithm specifier in a BSD-style digest file to the
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
243 type/factory of the corresponding algorithm.
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
244
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
245 :param str s: the tag (i.e. normalized name) or the algorithm
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
246 :return: the digest type or factory for `s`
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
247 :raises ValueError: on unknown and/or unhandled algorithms
307
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
248 :raises ImportError: if a module that is required to handle given
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
249 specifier `s` is not available (e.g. BLAKE2b on
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
250 Python 2)
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
251
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
252 All string comparisons are case-sensitive.
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
253
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
254 """
307
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
255 # Standard in Python2.7
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
256 if s == "MD5":
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
257 return hashlib.md5
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
258 elif s == "SHA1":
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
259 return hashlib.sha1
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
260 elif s == "SHA224":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
261 return hashlib.sha224
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
262 elif s == "SHA256":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
263 return hashlib.sha256
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
264 elif s == "SHA384":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
265 return hashlib.sha384
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
266 elif s == "SHA512":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
267 return hashlib.sha512
307
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
268 # Available in Python 3.6+
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
269 elif s == "SHA3-224":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
270 return hashlib.sha3_224
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
271 elif s == "SHA3-256":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
272 return hashlib.sha3_256
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
273 elif s == "SHA3-384":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
274 return hashlib.sha3_384
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
275 elif s == "SHA3-512":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
276 return hashlib.sha3_512
307
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
277 # Available in Python 3.6+ or if pyblake2 is installed
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
278 elif s in ("BLAKE2b", "BLAKE2b-512", "BLAKE2b512"): # compat for openssl
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
279 return get_blake2b()
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
280 elif s in ("BLAKE2s", "BLAKE2s-256", "BLAKE2s256"): # compat for openssl
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
281 return get_blake2s()
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
282 elif s in ("BLAKE2b-256", "BLAKE2b256"): # also compat for openssl dgst
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
283 return get_blake2b_256()
307
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
284 # Vendored in cutils.crcmod
184
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
285 elif s == "CRC-24":
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
286 return get_crc("crc-24")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
287 elif s == "CRC-32-ISO":
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
288 return get_crc("crc-32")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
289 elif s == "CRC-32-POSIX":
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
290 return get_crc("posix")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
291 elif s == "CRC-64-ISO":
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
292 return get_crc("crc-64")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
293 elif s == "CRC-64-ISO-2":
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
294 return get_crc("crc-64-2")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
295 elif s == "CRC-64-ECMA":
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
296 return get_crc("crc-64-ecma")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
297 elif s == "CRC-64-XZ":
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
298 return get_crc("crc-64-xz")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
299 elif s == "CRC-64-GO-ISO":
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
300 return get_crc("crc-64-go")
6154b8e4ba94 Include some new algorithms: CRC
Franz Glasner <fzglas.hg@dom66.de>
parents: 173
diff changeset
301 elif s == "CRC-64-REDIS":
185
f04d4b1c14b3 FIX: "digest" had overwritteh the digest module
Franz Glasner <fzglas.hg@dom66.de>
parents: 184
diff changeset
302 return get_crc("crc-64-redis")
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
303 else:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
304 raise ValueError("unknown algorithm: {}".format(s))
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
305
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
306
307
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
307 def algotag2digest_size(s):
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
308 """Get the `digest_size` in bytes from given algorithm specifier `s`.
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
309
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
310 Contains a small static database of digest sizes for algorithms that
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
311 are not available by default in older Python versions.
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
312
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
313 :raises ValueError: on unknown and/or unhandled algorithms
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
314 :raises ImportError: if a module that is required to handle given
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
315 specifier `s` is not available (e.g. BLAKE2b on
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
316 Python 2)
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
317
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
318 All string comparisons are case-sensitive.
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
319
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
320 """
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
321 try:
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
322 dgst = algotag2algotype(s)()
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
323 return dgst.digest_size
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
324 except ImportError:
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
325 sz = {
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
326 "SHA3-224": 28,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
327 "SHA3-256": 32,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
328 "SHA3-384": 48,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
329 "SHA3-512": 64,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
330 "BLAKE2b": 64,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
331 "BLAKE2b-512": 64,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
332 "BLAKE2b512": 64,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
333 "BLAKE2b-256": 32,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
334 "BLAKE2b256": 32,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
335 "BLAKE2s": 32,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
336 "BLAKE2s-256": 32,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
337 "BLAKE2s256": 32,
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
338 }.get(s, None)
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
339 if not sz:
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
340 raise
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
341 return sz
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
342
64df94bf4659 treesum: Build a little static database of digest sizes.
Franz Glasner <fzglas.hg@dom66.de>
parents: 279
diff changeset
343
278
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed.
Franz Glasner <fzglas.hg@dom66.de>
parents: 196
diff changeset
344 def normalize_filename(filename, strip_dot_slashes=False):
124
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
345 if isinstance(filename, bytes):
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
346 filename = filename.replace(b"\\", b"/")
278
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed.
Franz Glasner <fzglas.hg@dom66.de>
parents: 196
diff changeset
347 if strip_dot_slashes:
124
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
348 while filename.startswith(b"./"):
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
349 filename = filename[2:]
278
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed.
Franz Glasner <fzglas.hg@dom66.de>
parents: 196
diff changeset
350 # This also handles adjacent /./ cases
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed.
Franz Glasner <fzglas.hg@dom66.de>
parents: 196
diff changeset
351 while b"/./" in filename:
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed.
Franz Glasner <fzglas.hg@dom66.de>
parents: 196
diff changeset
352 filename = filename.replace(b"/./", b"/", 1)
279
44e62e36cad4 treesum: when normalizing filenames also remove trailing "/."
Franz Glasner <fzglas.hg@dom66.de>
parents: 278
diff changeset
353 while filename.endswith(b"/."):
44e62e36cad4 treesum: when normalizing filenames also remove trailing "/."
Franz Glasner <fzglas.hg@dom66.de>
parents: 278
diff changeset
354 filename = filename[:-2]
124
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
355 else:
166
ed45abb4940f FIX: in util.normalize_filename(): use the 'u"' prefix in the else part for non-byte strings
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
356 filename = filename.replace(u"\\", u"/")
278
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed.
Franz Glasner <fzglas.hg@dom66.de>
parents: 196
diff changeset
357 if strip_dot_slashes:
166
ed45abb4940f FIX: in util.normalize_filename(): use the 'u"' prefix in the else part for non-byte strings
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
358 while filename.startswith(u"./"):
124
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
359 filename = filename[2:]
278
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed.
Franz Glasner <fzglas.hg@dom66.de>
parents: 196
diff changeset
360 # This also handles adjacent /./ cases
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed.
Franz Glasner <fzglas.hg@dom66.de>
parents: 196
diff changeset
361 while u"/./" in filename:
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed.
Franz Glasner <fzglas.hg@dom66.de>
parents: 196
diff changeset
362 filename = filename.replace(u"/./", u"/", 1)
279
44e62e36cad4 treesum: when normalizing filenames also remove trailing "/."
Franz Glasner <fzglas.hg@dom66.de>
parents: 278
diff changeset
363 while filename.endswith(u"/."):
44e62e36cad4 treesum: when normalizing filenames also remove trailing "/."
Franz Glasner <fzglas.hg@dom66.de>
parents: 278
diff changeset
364 filename = filename[:-2]
118
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents: 115
diff changeset
365 return filename
124
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
366
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
367
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
368 def fsencode(what):
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
369 """A somewhat compatibility function for :func:`os.fsencode`.
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
370
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
371 If `what` is of type :class:`bytes` no :func:`os.fsencode` is required.
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
372
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
373 """
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
374 if isinstance(what, bytes):
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
375 return what
3bd3f32b5e60 A first version of "treesum" is working
Franz Glasner <fzglas.hg@dom66.de>
parents: 120
diff changeset
376 return os.fsencode(what)
173
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
377
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
378
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
379 def interpolate_bytes(formatstr, *values):
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
380 """Interpolate byte strings also on Python 3.4.
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
381
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
382 :param bytes formatstr:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
383 :param values: params for interpolation: may *not* contain Unicode strings
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
384 :rvalue: the formatted octet
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
385 :rtype: bytes
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
386
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
387 """
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
388 assert isinstance(formatstr, bytes)
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
389 # Python 3.5+ or Python2 know how to interpolate byte strings
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
390 if PY35 or PY2:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
391 return formatstr % values
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
392 # Workaround with a Latin-1 dance
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
393 tformatstr = formatstr.decode("latin1")
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
394 tvalues = []
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
395 for v in values:
376
54a6d4534ef4 Convert some further value checks from plain Python assertions into real checks
Franz Glasner <fzglas.hg@dom66.de>
parents: 372
diff changeset
396 # Do not automatically convert text (unicode) string values into bytes.
173
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
397 if PY2:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
398 if isinstance(v, unicode): # noqa: F821 undefined name 'unicode'
376
54a6d4534ef4 Convert some further value checks from plain Python assertions into real checks
Franz Glasner <fzglas.hg@dom66.de>
parents: 372
diff changeset
399 raise TypeError(
54a6d4534ef4 Convert some further value checks from plain Python assertions into real checks
Franz Glasner <fzglas.hg@dom66.de>
parents: 372
diff changeset
400 "unicode values not supported when interpolating"
54a6d4534ef4 Convert some further value checks from plain Python assertions into real checks
Franz Glasner <fzglas.hg@dom66.de>
parents: 372
diff changeset
401 " into bytes")
173
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
402 else:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
403 if isinstance(v, str):
376
54a6d4534ef4 Convert some further value checks from plain Python assertions into real checks
Franz Glasner <fzglas.hg@dom66.de>
parents: 372
diff changeset
404 raise TypeError(
54a6d4534ef4 Convert some further value checks from plain Python assertions into real checks
Franz Glasner <fzglas.hg@dom66.de>
parents: 372
diff changeset
405 "unicode (native string) values not supported when"
54a6d4534ef4 Convert some further value checks from plain Python assertions into real checks
Franz Glasner <fzglas.hg@dom66.de>
parents: 372
diff changeset
406 " interpolating into bytes")
173
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
407 if isinstance(v, bytes):
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
408 tvalues.append(v.decode("latin1"))
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
409 else:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
410 tvalues.append(v)
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting.
Franz Glasner <fzglas.hg@dom66.de>
parents: 172
diff changeset
411 return (tformatstr % tuple(tvalues)).encode("latin1")