Python/apps/py-cutils: cutils/util/__init_

annotate cutils/util/init.py @ 173:e081b6ee5570

treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. No extra module is required for it to run using sha SHA and SHA-2 family of digests.

author	Franz Glasner <fzglas.hg@dom66.de>
date	Fri, 10 Jan 2025 12:46:44 +0100
parents	804a823c63f5
children	6154b8e4ba94

rev	line source
115 e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1 # -- coding: utf-8 --
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2 # :-
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	3 # :Copyright: (c) 2020-2025 Franz Glasner
115 e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	4 # :License: BSD-3-Clause
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	5 # :-
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	6 r"""Utility package.
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	7
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	8 """
e15b3d1ff0d9 New subpackage with a "contextlib.nullcontext" for older Python versions Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	9
164 a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util Franz Glasner <fzglas.hg@dom66.de> parents: 163 diff changeset	10 __all__ = ["PY2",
173 e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	11 "PY35",
164 a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util Franz Glasner <fzglas.hg@dom66.de> parents: 163 diff changeset	12 "normalize_filename",
118 12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	13 "argv2algo",
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	14 "algotag2algotype",
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	15 "get_blake2b",
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	16 "get_blake2b_256",
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	17 "get_blake2s",
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	18 "default_algotag",
163 fa7dd54e9715 FIX: Add "fsencode" to public functions in cutils.util Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	19 "fsencode",
118 12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	20 ]
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	21
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	22
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	23 import argparse
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	24 import hashlib
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	25 import os
164 a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util Franz Glasner <fzglas.hg@dom66.de> parents: 163 diff changeset	26 import sys
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util Franz Glasner <fzglas.hg@dom66.de> parents: 163 diff changeset	27
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util Franz Glasner <fzglas.hg@dom66.de> parents: 163 diff changeset	28
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util Franz Glasner <fzglas.hg@dom66.de> parents: 163 diff changeset	29 PY2 = sys.version_info[0] < 3
173 e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	30 PY35 = sys.version_info[:2] >= (3, 5)
118 12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	31
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	32
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	33 def default_algotag():
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	34 """Determine the "best" default algorithm.
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	35
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	36 Depend on availability in :mod:`hashlib`.
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	37
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	38 Prefer BLAKE2b-256, SHA256 or SHA1 -- in this order.
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	39
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	40 Does not consider :mod:`pyblake2` if it is available eventually.
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	41
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	42 """
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	43 if "blake2b" in hashlib.algorithms_available:
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	44 return "BLAKE2b-256"
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	45 if "sha256" in hashlib.algorithms_available:
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	46 return "SHA256"
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	47 return "SHA1"
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	48
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	49
118 12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	50 def get_blake2b():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	51 """Get the factory for blake2b"""
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	52 try:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	53 return hashlib.blake2b
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	54 except AttributeError:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	55 import pyblake2
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	56 return pyblake2.blake2b
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	57
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	58
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	59 def get_blake2b_256():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	60 """Get the factory for blake2b-256"""
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	61
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	62 try:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	63 hashlib.blake2b
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	64 except AttributeError:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	65 import pyblake2
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	66
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	67 def _get_blake():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	68 return pyblake2.blake2b(digest_size=32)
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	69
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	70 else:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	71
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	72 def _get_blake():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	73 return hashlib.blake2b(digest_size=32)
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	74
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	75 return _get_blake
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	76
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	77
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	78 def get_blake2s():
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	79 """Get the factory for blake2s"""
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	80 try:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	81 return hashlib.blake2s
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	82 except AttributeError:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	83 import pyblake2
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	84 return pyblake2.blake2s
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	85
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	86
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	87 def argv2algo(s):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	88 """Convert a command line algorithm specifier into a tuple with the
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	89 type/factory of the digest and the algorithms tag for output purposes.
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	90
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	91 :param str s: the specifier from the command line; should include all
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 166 diff changeset	92 algorithm tags also (for proper round-tripping)
118 12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	93 :return: the internal digest specification
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	94 :rtype: a tuple (digest_type_or_factory, name_in_output)
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	95 :raises argparse.ArgumentTypeError: for unrecognized algorithms or names
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	96
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	97 String comparisons are done case-insensitively.
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	98
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	99 """
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	100 s = s.lower()
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	101 if s in ("1", "sha1"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	102 return (hashlib.sha1, "SHA1")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	103 elif s in ("224", "sha224"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	104 return (hashlib.sha224, "SHA224")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	105 elif s in ("256", "sha256"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	106 return (hashlib.sha256, "SHA256")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	107 elif s in ("384", "sha384"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	108 return (hashlib.sha384, "SHA384")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	109 elif s in ("512", "sha512"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	110 return (hashlib.sha512, "SHA512")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	111 elif s in ("3-224", "sha3-224"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	112 return (hashlib.sha3_224, "SHA3-224")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	113 elif s in ("3-256", "sha3-256"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	114 return (hashlib.sha3_256, "SHA3-256")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	115 elif s in ("3-384", "sha3-384"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	116 return (hashlib.sha3_384, "SHA3-384")
120 a548783381b6 More accurate description of --algorithm. Franz Glasner <fzglas.hg@dom66.de> parents: 118 diff changeset	117 elif s in ("3", "3-512", "sha3-512"):
118 12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	118 return (hashlib.sha3_512, "SHA3-512")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	119 elif s in ("blake2b", "blake2b-512", "blake2", "blake2-512"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	120 return (get_blake2b(), "BLAKE2b")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	121 elif s in ("blake2s", "blake2s-256"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	122 return (get_blake2s(), "BLAKE2s")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	123 elif s in ("blake2-256", "blake2b-256"):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	124 return (get_blake2b_256(), "BLAKE2b-256")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	125 elif s == "md5":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	126 return (hashlib.md5, "MD5")
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	127 else:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	128 raise argparse.ArgumentTypeError(
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	129 "`{}' is not a recognized algorithm".format(s))
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	130
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	131
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	132 def algotag2algotype(s):
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	133 """Convert the algorithm specifier in a BSD-style digest file to the
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	134 type/factory of the corresponding algorithm.
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	135
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	136 :param str s: the tag (i.e. normalized name) or the algorithm
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	137 :return: the digest type or factory for `s`
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	138 :raises ValueError: on unknown and/or unhandled algorithms
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	139
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	140 All string comparisons are case-sensitive.
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	141
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	142 """
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	143 if s == "SHA1":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	144 return hashlib.sha1
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	145 elif s == "SHA224":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	146 return hashlib.sha224
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	147 elif s == "SHA256":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	148 return hashlib.sha256
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	149 elif s == "SHA384":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	150 return hashlib.sha384
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	151 elif s == "SHA512":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	152 return hashlib.sha512
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	153 elif s == "SHA3-224":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	154 return hashlib.sha3_224
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	155 elif s == "SHA3-256":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	156 return hashlib.sha3_256
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	157 elif s == "SHA3-384":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	158 return hashlib.sha3_384
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	159 elif s == "SHA3-512":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	160 return hashlib.sha3_512
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	161 elif s in ("BLAKE2b", "BLAKE2b-512", "BLAKE2b512"): # compat for openssl
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	162 return get_blake2b()
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	163 elif s in ("BLAKE2s", "BLAKE2s-256", "BLAKE2s256"): # compat for openssl
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	164 return get_blake2s()
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	165 elif s in ("BLAKE2b-256", "BLAKE2b256"): # also compat for openssl dgst
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	166 return get_blake2b_256()
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	167 elif s == "MD5":
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	168 return hashlib.md5
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	169 else:
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	170 raise ValueError("unknown algorithm: {}".format(s))
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	171
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	172
12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	173 def normalize_filename(filename, strip_leading_dot_slash=False):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	174 if isinstance(filename, bytes):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	175 filename = filename.replace(b"\\", b"/")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	176 if strip_leading_dot_slash:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	177 while filename.startswith(b"./"):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	178 filename = filename[2:]
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	179 else:
166 ed45abb4940f FIX: in util.normalize_filename(): use the 'u"' prefix in the else part for non-byte strings Franz Glasner <fzglas.hg@dom66.de> parents: 164 diff changeset	180 filename = filename.replace(u"\\", u"/")
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	181 if strip_leading_dot_slash:
166 ed45abb4940f FIX: in util.normalize_filename(): use the 'u"' prefix in the else part for non-byte strings Franz Glasner <fzglas.hg@dom66.de> parents: 164 diff changeset	182 while filename.startswith(u"./"):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	183 filename = filename[2:]
118 12339ac2148d Move some functions into cutils.util (i.e. algorithms and their aliases) Franz Glasner <fzglas.hg@dom66.de> parents: 115 diff changeset	184 return filename
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	185
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	186
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	187 def fsencode(what):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	188 """A somewhat compatibility function for :func:`os.fsencode`.
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	189
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	190 If `what` is of type :class:`bytes` no :func:`os.fsencode` is required.
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	191
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	192 """
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	193 if isinstance(what, bytes):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	194 return what
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: 120 diff changeset	195 return os.fsencode(what)
173 e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	196
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	197
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	198 def interpolate_bytes(formatstr, *values):
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	199 """Interpolate byte strings also on Python 3.4.
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	200
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	201 :param bytes formatstr:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	202 :param values: params for interpolation: may not contain Unicode strings
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	203 :rvalue: the formatted octet
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	204 :rtype: bytes
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	205
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	206 """
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	207 assert isinstance(formatstr, bytes)
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	208 # Python 3.5+ or Python2 know how to interpolate byte strings
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	209 if PY35 or PY2:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	210 return formatstr % values
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	211 # Workaround with a Latin-1 dance
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	212 tformatstr = formatstr.decode("latin1")
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	213 tvalues = []
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	214 for v in values:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	215 if PY2:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	216 if isinstance(v, unicode): # noqa: F821 undefined name 'unicode'
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	217 assert False
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	218 else:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	219 if isinstance(v, str):
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	220 assert False
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	221 if isinstance(v, bytes):
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	222 tvalues.append(v.decode("latin1"))
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	223 else:
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	224 tvalues.append(v)
e081b6ee5570 treesum.py now runs on Python3.4 also: use a workaround for its missing byte % formatting. Franz Glasner <fzglas.hg@dom66.de> parents: 172 diff changeset	225 return (tformatstr % tuple(tvalues)).encode("latin1")

Mercurial > hgrepos > Python > apps > py-cutils

annotate cutils/util/__init__.py @ 173:e081b6ee5570

annotate cutils/util/init.py @ 173:e081b6ee5570