Mercurial > hgrepos > Python > apps > py-cutils

annotate shasum.py @ 52:5935055edea6

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
More proper formal checks (with regexp) for valid hex and base64 encoding of digests
author Franz Glasner <f.glasner@feldmann-mg.com>
date Wed, 26 Jan 2022 14:49:24 +0100
parents 58d5a0b6e5b3
children 6e91c530545f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 r"""
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 :Author: Franz Glasner
43
d856432a1cbb Extend copyright year to 2022
Franz Glasner <fzglas.hg@dom66.de>
parents: 42
diff changeset
3 :Copyright: (c) 2020-2022 Franz Glasner.
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 All rights reserved.
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 :License: BSD 3-Clause "New" or "Revised" License.
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 See :ref:`LICENSE <license>` for details.
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7 If you cannot find LICENSE see
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8 <https://opensource.org/licenses/BSD-3-Clause>
26
ea4bb192c437 FIX: RCS keyword syntax
Franz Glasner <fzglas.hg@dom66.de>
parents: 23
diff changeset
9 :ID: @(#) $HGid$
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
11 """
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
12
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
13 from __future__ import print_function
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14
13
db64e282b049 Implement a version option
Franz Glasner <fzglas.hg@dom66.de>
parents: 12
diff changeset
15
30
ffcce6062bee Put the version number info the optinal common module _cutils.py
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
16 try:
ffcce6062bee Put the version number info the optinal common module _cutils.py
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
17 from _cutils import __version__
ffcce6062bee Put the version number info the optinal common module _cutils.py
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
18 except ImportError:
ffcce6062bee Put the version number info the optinal common module _cutils.py
Franz Glasner <fzglas.hg@dom66.de>
parents: 27
diff changeset
19 __version__ = "unknown"
13
db64e282b049 Implement a version option
Franz Glasner <fzglas.hg@dom66.de>
parents: 12
diff changeset
20
27
c29bd96dcd4f Put the HG revision into the version output also
Franz Glasner <fzglas.hg@dom66.de>
parents: 26
diff changeset
21 __revision__ = "|VCSRevision|"
c29bd96dcd4f Put the HG revision into the version output also
Franz Glasner <fzglas.hg@dom66.de>
parents: 26
diff changeset
22 __date__ = "|VCSJustDate|"
c29bd96dcd4f Put the HG revision into the version output also
Franz Glasner <fzglas.hg@dom66.de>
parents: 26
diff changeset
23
13
db64e282b049 Implement a version option
Franz Glasner <fzglas.hg@dom66.de>
parents: 12
diff changeset
24
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
25 import argparse
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
26 import base64
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
27 import binascii
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
28 import hashlib
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
29 import io
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
30 try:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
31 import mmap
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
32 except ImportError:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
33 mmap = None
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
34 import os
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
35 import re
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
36 import stat
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
37 import sys
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
38
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
39
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
40 PY2 = sys.version_info[0] < 3
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
41
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
42 CHUNK_SIZE = 1024*1024
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
43 MAP_CHUNK_SIZE = 64*1024*1024
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
44
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
45
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
46 def main(argv=None):
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
47 aparser = argparse.ArgumentParser(
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
48 description="Python implementation of shasum",
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
49 fromfile_prefix_chars='@')
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
50 aparser.add_argument(
7
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
51 "--algorithm", "-a", action="store", type=argv2algo,
8
048b97213a23 Change the default algorithm to SHA1 to be compatible with Perl's shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 7
diff changeset
52 help="1 (default), 224, 256, 384, 512, 3-224, 3-256, 3-384, 3-512, blake2b, blake2s, md5")
7
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
53 aparser.add_argument(
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
54 "--base64", action="store_true",
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
55 help="output checksums in base64 notation, not hexadecimal (OpenBSD).")
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
56 aparser.add_argument(
3
5a6ed622846c Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents: 2
diff changeset
57 "--binary", "-b", action="store_false", dest="text_mode", default=False,
5a6ed622846c Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents: 2
diff changeset
58 help="read in binary mode (default)")
5a6ed622846c Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents: 2
diff changeset
59 aparser.add_argument(
9
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
60 "--bsd", "-B", action="store_true", dest="bsd", default=False,
17
184ab1da1307 Extend help messages
Franz Glasner <fzglas.hg@dom66.de>
parents: 14
diff changeset
61 help="Write BSD style output. This is also the default output format of :command:`openssl dgst`.")
9
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
62 aparser.add_argument(
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
63 "--check", "-c", action="store_true",
17
184ab1da1307 Extend help messages
Franz Glasner <fzglas.hg@dom66.de>
parents: 14
diff changeset
64 help="""Read digests from FILEs and check them.
184ab1da1307 Extend help messages
Franz Glasner <fzglas.hg@dom66.de>
parents: 14
diff changeset
65 If this option is specified, the FILE options become checklists. Each
184ab1da1307 Extend help messages
Franz Glasner <fzglas.hg@dom66.de>
parents: 14
diff changeset
66 checklist should contain hash results in a supported format, which will
184ab1da1307 Extend help messages
Franz Glasner <fzglas.hg@dom66.de>
parents: 14
diff changeset
67 be verified against the specified paths. Output consists of the digest
184ab1da1307 Extend help messages
Franz Glasner <fzglas.hg@dom66.de>
parents: 14
diff changeset
68 used, the file name, and an OK, FAILED, or MISSING for the result of
184ab1da1307 Extend help messages
Franz Glasner <fzglas.hg@dom66.de>
parents: 14
diff changeset
69 the comparison. This will validate any of the supported checksums.
184ab1da1307 Extend help messages
Franz Glasner <fzglas.hg@dom66.de>
parents: 14
diff changeset
70 If no file is given, stdin is used.""")
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
71 aparser.add_argument(
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
72 "--checklist", "-C", metavar="CHECKLIST",
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
73 help="""Compare the checksum of each FILE against the checksums in
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
74 the CHECKLIST. Any specified FILE that is not listed in the CHECKLIST will
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
75 generate an error.""")
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
76
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
77 aparser.add_argument(
9
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
78 "--reverse", "-r", action="store_false", dest="bsd", default=False,
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
79 help="explicitely select normal coreutils style output (to be option compatible with BSD style commands and :command:`openssl dgst -r`)")
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
80 aparser.add_argument(
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
81 "--tag", action="store_true", dest="bsd", default=False,
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
82 help="alias for the `--bsd' option (to be compatible with :command:`b2sum`)")
5
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
83 aparser.add_argument(
3
5a6ed622846c Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents: 2
diff changeset
84 "--text", "-t", action="store_true", dest="text_mode", default=False,
6
a21e83c855cc Help message: drop "yet" for text-mode flag
Franz Glasner <fzglas.hg@dom66.de>
parents: 5
diff changeset
85 help="read in text mode (not supported)")
3
5a6ed622846c Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents: 2
diff changeset
86 aparser.add_argument(
27
c29bd96dcd4f Put the HG revision into the version output also
Franz Glasner <fzglas.hg@dom66.de>
parents: 26
diff changeset
87 "--version", "-v", action="version", version="%s (rv:%s)" % (__version__, __revision__))
13
db64e282b049 Implement a version option
Franz Glasner <fzglas.hg@dom66.de>
parents: 12
diff changeset
88 aparser.add_argument(
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
89 "files", nargs="*", metavar="FILE")
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
90
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
91 opts = aparser.parse_args(args=argv)
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
92
3
5a6ed622846c Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents: 2
diff changeset
93 if opts.text_mode:
5a6ed622846c Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents: 2
diff changeset
94 print("ERROR: text mode not supported", file=sys.stderr)
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
95 sys.exit(78) # :manpage:`sysexits(3)` EX_CONFIG
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
96
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
97 if opts.check and opts.checklist:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
98 print("ERROR: only one of --check or --checklist allowed",
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
99 file=sys.stderr)
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
100 sys.exit(64) # :manpage:`sysexits(3)` EX_USAGE
5
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
101
7
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
102 if not opts.algorithm:
8
048b97213a23 Change the default algorithm to SHA1 to be compatible with Perl's shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 7
diff changeset
103 opts.algorithm = argv2algo("1")
7
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
104
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
105 opts.dest = None
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
106
45
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
107 return shasum(opts)
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
108
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
109
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
110 def gen_opts(files=[], algorithm="SHA1", bsd=False, text_mode=False,
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
111 checklist=False, check=False, dest=None, base64=False):
45
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
112 if text_mode:
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
113 raise ValueError("text mode not supported")
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
114 if checklist and check:
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
115 raise ValueError("only one of `checklist' or `check' is allowed")
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
116 opts = argparse.Namespace(files=files,
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
117 algorithm=(algotag2algotype(algorithm),
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
118 algorithm),
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
119 bsd=bsd,
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
120 checklist=checklist,
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
121 check=check,
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
122 text_mode=False,
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
123 dest=dest,
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
124 base64=base64)
45
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
125 return opts
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
126
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
127
b25ef7293bf2 Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents: 43
diff changeset
128 def shasum(opts):
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
129 if opts.check:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
130 return verify_digests_from_files(opts)
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
131 elif opts.checklist:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
132 return verify_digests_with_checklist(opts)
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
133 else:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
134 return generate_digests(opts)
10
77446cd3ea6f Move the digest generation loop from "main()" into an own function "generate_digests()".
Franz Glasner <fzglas.hg@dom66.de>
parents: 9
diff changeset
135
77446cd3ea6f Move the digest generation loop from "main()" into an own function "generate_digests()".
Franz Glasner <fzglas.hg@dom66.de>
parents: 9
diff changeset
136
77446cd3ea6f Move the digest generation loop from "main()" into an own function "generate_digests()".
Franz Glasner <fzglas.hg@dom66.de>
parents: 9
diff changeset
137 def generate_digests(opts):
5
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
138 if opts.bsd:
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
139 out = out_bsd
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
140 else:
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
141 out = out_std
20
8f0241ed4a00 Do not append "-" to an empty FILE list any more but check explicitely for an empty list
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
142 if not opts.files or (len(opts.files) == 1 and opts.files[0] == '-'):
4
67d10529ce88 FIX: "-" filename handling now consistent with Perl shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 3
diff changeset
143 if PY2:
67d10529ce88 FIX: "-" filename handling now consistent with Perl shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 3
diff changeset
144 if sys.platform == "win32":
11
15c3416d3677 FIX: "msvcrt" is a top-level module on Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 10
diff changeset
145 import os, msvcrt # noqa: E401
4
67d10529ce88 FIX: "-" filename handling now consistent with Perl shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 3
diff changeset
146 msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY)
67d10529ce88 FIX: "-" filename handling now consistent with Perl shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 3
diff changeset
147 source = sys.stdin
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
148 else:
4
67d10529ce88 FIX: "-" filename handling now consistent with Perl shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 3
diff changeset
149 source = sys.stdin.buffer
5
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
150 out(sys.stdout,
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
151 compute_digest_stream(opts.algorithm[0], source),
5
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
152 None,
7
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
153 opts.algorithm[1],
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
154 True,
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
155 opts.base64)
4
67d10529ce88 FIX: "-" filename handling now consistent with Perl shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 3
diff changeset
156 else:
67d10529ce88 FIX: "-" filename handling now consistent with Perl shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 3
diff changeset
157 for fn in opts.files:
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
158 out(opts.dest or sys.stdout,
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
159 compute_digest_file(opts.algorithm[0], fn),
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
160 fn,
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
161 opts.algorithm[1],
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
162 True,
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
163 opts.base64)
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
164 return 0
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
165
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
166
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
167 def compare_digests_equal(given_digest, expected_digest, algo):
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
168 """Compare a newly computed binary digest `given_digest` with a digest
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
169 string (hex or base64) in `expected_digest`.
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
170
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
171 :param bytes given_digest:
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
172 :param str expected_digest: hexlified or base64 encoded digest
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
173 :param algo: The algorithm (factory)
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
174 :return: `True` if the digests are equal, `False` if not
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
175 :rtype: bool
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
176
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
177 """
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
178 if len(expected_digest) == algo().digest_size * 2:
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
179 # hex
52
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
180 if re.search(r"\A[a-fA-F0-9]+\Z", expected_digest):
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
181 try:
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
182 exd = binascii.unhexlify(expected_digest)
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
183 except TypeError:
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
184 return False
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
185 else:
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
186 return False
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
187 else:
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
188 # base64
52
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
189 if re.search(
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
190 r"\A(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)?\Z",
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
191 expected_digest):
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
192 try:
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
193 exd = base64.b64decode(expected_digest)
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
194 except TypeError:
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
195 return False
5935055edea6 More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 51
diff changeset
196 else:
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
197 return False
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
198 return given_digest == exd
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
199
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
200
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
201 def verify_digests_with_checklist(opts):
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
202 dest = opts.dest or sys.stdout
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
203 exit_code = 0
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
204 if not opts.files or (len(opts.files) == 1 and opts.files[0] == '-'):
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
205 if PY2:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
206 if sys.platform == "win32":
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
207 import os, msvcrt # noqa: E401
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
208 msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY)
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
209 source = sys.stdin
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
210 else:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
211 source = sys.stdin.buffer
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
212 pl = get_parsed_digest_line_from_checklist(opts.checklist, opts, None)
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
213 if pl is None:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
214 exit_code = 1
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
215 print("-: MISSING", file=dest)
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
216 else:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
217 tag, algo, cl_filename, cl_digest = pl
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
218 computed_digest = compute_digest_stream(algo, source)
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
219 if compare_digests_equal(computed_digest, cl_digest, algo):
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
220 res = "OK"
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
221 else:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
222 res = "FAILED"
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
223 exit_code = 1
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
224 print("{}: {}: {}".format(tag, "-", res), file=dest)
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
225 else:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
226 for fn in opts.files:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
227 pl = get_parsed_digest_line_from_checklist(opts.checklist, opts, fn)
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
228 if pl is None:
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
229 print("{}: MISSING".format(fn), file=dest)
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
230 exit_code = 1
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
231 else:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
232 tag, algo, cl_filename, cl_digest = pl
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
233 computed_digest = compute_digest_file(algo, fn)
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
234 if compare_digests_equal(computed_digest, cl_digest, algo):
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
235 res = "OK"
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
236 else:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
237 exit_code = 1
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
238 res = "FAILED"
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
239 print("{}: {}: {}".format(tag, fn, res), file=dest)
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
240 return exit_code
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
241
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
242
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
243 def verify_digests_from_files(opts):
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
244 dest = opts.dest or sys.stdout
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
245 exit_code = 0
20
8f0241ed4a00 Do not append "-" to an empty FILE list any more but check explicitely for an empty list
Franz Glasner <fzglas.hg@dom66.de>
parents: 19
diff changeset
246 if not opts.files or (len(opts.files) == 1 and opts.files[0] == '-'):
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
247 for checkline in sys.stdin:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
248 if not checkline:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
249 continue
18
285848db0b52 When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents: 17
diff changeset
250 r, fn, tag = handle_checkline(opts, checkline)
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
251 print("{}: {}: {}".format(tag, fn, r.upper()), file=dest)
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
252 if r != "ok" and exit_code == 0:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
253 exit_code = 1
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
254 else:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
255 for fn in opts.files:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
256 with io.open(fn, "rt", encoding="utf-8") as checkfile:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
257 for checkline in checkfile:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
258 if not checkline:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
259 continue
18
285848db0b52 When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents: 17
diff changeset
260 r, fn, tag = handle_checkline(opts, checkline)
47
5bec7a5d894a Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents: 45
diff changeset
261 print("{}: {}: {}".format(tag, fn, r.upper()), file=dest)
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
262 if r != "ok" and exit_code == 0:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
263 exit_code = 1
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
264 return exit_code
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
265
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
266
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
267 def handle_checkline(opts, line):
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
268 """
18
285848db0b52 When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents: 17
diff changeset
269 :return: a tuple with static "ok", "missing", or "failed", the filename and
285848db0b52 When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents: 17
diff changeset
270 the digest used
285848db0b52 When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents: 17
diff changeset
271 :rtype: tuple(str, str, str)
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
272
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
273 """
21
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
274 parts = parse_digest_line(opts, line)
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
275 if not parts:
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
276 raise ValueError(
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
277 "improperly formatted digest line: {}".format(line))
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
278 tag, algo, fn, digest = parts
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
279 try:
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
280 d = compute_digest_file(algo, fn)
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
281 if compare_digests_equal(d, digest, algo):
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
282 return ("ok", fn, tag)
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
283 else:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
284 return ("failed", fn, tag)
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
285 except EnvironmentError:
18
285848db0b52 When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents: 17
diff changeset
286 return ("missing", fn, tag)
5
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
287
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
288
22
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
289 def get_parsed_digest_line_from_checklist(checklist, opts, filename):
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
290 if filename is None:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
291 filenames = ("-", "stdin", "", )
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
292 else:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
293 filenames = (
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
294 normalize_filename(filename, strip_leading_dot_slash=True),)
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
295 with io.open(checklist, "rt", encoding="utf-8") as clf:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
296 for checkline in clf:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
297 if not checkline:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
298 continue
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
299 parts = parse_digest_line(opts, checkline)
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
300 if not parts:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
301 raise ValueError(
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
302 "improperly formatted digest line: {}".format(checkline))
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
303 fn = normalize_filename(parts[2], strip_leading_dot_slash=True)
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
304 if fn in filenames:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
305 return parts
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
306 else:
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
307 return None
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
308
6bdfc5ad4656 Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents: 21
diff changeset
309
21
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
310 def parse_digest_line(opts, line):
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
311 """Parse a `line` of a digest file and return its parts.
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
312
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
313 :return: a tuple of the normalized algorithm tag, the algorithm
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
314 constructor, the filename and the hex digest;
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
315 if `line` cannot be parsed successfully `None` is returned
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
316 :rtype: tuple(str, obj, str, str) or None
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
317
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
318 Handles coreutils and BSD-style file formats.
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
319
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
320 """
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
321 # determine checkfile format (BSD or coreutils)
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
322 # BSD?
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
323 mo = re.search(r"\A(\S+)\s*\((.*)\)\s*=\s*(.+)\n?\Z", line)
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
324 if mo:
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
325 # (tag, algorithm, filename, digest)
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
326 return (mo.group(1),
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
327 algotag2algotype(mo.group(1)),
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
328 mo.group(2),
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
329 mo.group(3))
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
330 else:
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
331 # coreutils?
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
332 mo = re.search(r"([^\ ]+) [\*\ ]?(.+)\n?\Z", line)
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
333 if mo:
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
334 # (tag, algorithm, filename, digest)
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
335 return (opts.algorithm[1],
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
336 opts.algorithm[0],
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
337 mo.group(2),
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
338 mo.group(1))
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
339 else:
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
340 return None
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
341
f2d634270e1c Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents: 20
diff changeset
342
7
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
343 def argv2algo(s):
42
e5f6af8364db Typo in docs
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 30
diff changeset
344 """Convert a command line algorithm specifier into a tuple with the
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
345 type/factory of the digest and the algorithms tag for output purposes.
7
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
346
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
347 :param str s: the specifier from the commane line
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
348 :return: the internal digest specification
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
349 :rtype: a tuple (digest_type_or_factory, name_in_output)
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
350
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
351 String comparisons are done case-insensitively.
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
352
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
353 """
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
354 s = s.lower()
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
355 if s in ("1", "sha1"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
356 return (hashlib.sha1, "SHA1")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
357 elif s in ("224", "sha224"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
358 return (hashlib.sha224, "SHA224")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
359 elif s in ("256", "sha256"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
360 return (hashlib.sha256, "SHA256")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
361 elif s in ("384", "sha384"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
362 return (hashlib.sha384, "SHA384")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
363 elif s in ("512", "sha512"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
364 return (hashlib.sha512, "SHA512")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
365 elif s in ("3-224", "sha3-224"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
366 return (hashlib.sha3_224, "SHA3-224")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
367 elif s in ("3-256", "sha3-256"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
368 return (hashlib.sha3_256, "SHA3-256")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
369 elif s in ("3-384", "sha3-384"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
370 return (hashlib.sha3_384, "SHA3-384")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
371 elif s in ("3-512", "sha3-512"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
372 return (hashlib.sha3_512, "SHA3-512")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
373 elif s in ("blake2b", "blake2b-512"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
374 return (hashlib.blake2b, "BLAKE2b")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
375 elif s in ("blake2s", "blake2s-256"):
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
376 return (hashlib.blake2s, "BLAKE2s")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
377 elif s == "md5":
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
378 return (hashlib.md5, "MD5")
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
379 else:
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
380 raise argparse.ArgumentTypeError(
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
381 "`{}' is not a recognized algorithm".format(s))
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
382
47b4c98e4d40 Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents: 6
diff changeset
383
12
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
384 def algotag2algotype(s):
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
385 """Convert the algorithm specifier in a BSD-style digest file to the
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
386 type/factory of the corresponding algorithm.
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
387
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
388 :param str s: the tag (i.e. normalized name) or the algorithm
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
389 :return: the digest type or factory for `s`
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
390
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
391 All string comparisons are case-sensitive.
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
392
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
393 """
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
394 if s == "SHA1":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
395 return hashlib.sha1
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
396 elif s == "SHA224":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
397 return hashlib.sha224
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
398 elif s == "SHA256":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
399 return hashlib.sha256
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
400 elif s == "SHA384":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
401 return hashlib.sha384
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
402 elif s == "SHA512":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
403 return hashlib.sha512
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
404 elif s == "SHA3-224":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
405 return hashlib.sha3_224
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
406 elif s == "SHA3-256":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
407 return hashlib.sha3_256
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
408 elif s == "SHA3-384":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
409 return hashlib.sha3_384
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
410 elif s == "SHA3-512":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
411 return hashlib.sha3_512
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
412 elif s == "BLAKE2b":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
413 return hashlib.blake2b
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
414 elif s == "BLAKE2s":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
415 return hashlib.blake2s
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
416 elif s == "MD5":
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
417 return hashlib.md5
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
418 else:
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
419 raise ValueError("unknown algorithm: {}".format(s))
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
420
5e2c9123f93f Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents: 11
diff changeset
421
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
422 def out_bsd(dest, digest, filename, digestname, binary, use_base64):
9
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
423 """BSD format output, also :command:`openssl dgst` and
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
424 :command:`b2sum --tag" format output
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
425
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
426 """
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
427 if use_base64:
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
428 digest = base64.b64encode(digest).decode("ascii")
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
429 else:
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
430 digest = binascii.hexlify(digest).decode("ascii")
5
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
431 if filename is None:
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
432 print(digest, file=dest)
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
433 else:
19
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
434 print("{} ({}) = {}".format(digestname,
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
435 normalize_filename(filename),
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
436 digest),
5
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
437 file=dest)
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
438
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
439
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
440 def out_std(dest, digest, filename, digestname, binary, use_base64):
9
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
441 """Coreutils format (:command:`shasum` et al.)
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
442
81f28bf89c26 Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 8
diff changeset
443 """
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
444 if use_base64:
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
445 digest = base64.b64encode(digest).decode("ascii")
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
446 else:
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
447 digest = binascii.hexlify(digest).decode("ascii")
19
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
448 print("{} {}{}".format(
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
449 digest,
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
450 '*' if binary else ' ',
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
451 '-' if filename is None else normalize_filename(filename)),
5
bbcb225640de Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents: 4
diff changeset
452 file=dest)
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
453
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
454
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
455 def compute_digest_file(hashobj, filename):
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
456 """
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
457 :param hashobj: a :mod:`hashlib` compatible hash algorithm type or factory
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
458 :param str filename: filename within the filesystem
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
459 :return: the digest in hex form
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
460 :rtype: str
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
461
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
462 """
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
463 h = hashobj()
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
464 flags = os.O_RDONLY
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
465 try:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
466 flags |= os.O_BINARY
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
467 except AttributeError:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
468 pass
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
469 fd = os.open(filename, flags)
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
470 try:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
471 st = os.fstat(fd)
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
472 filesize = st[stat.ST_SIZE]
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
473 #
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
474 # On Windows mmapped file with length 0 are not supported
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
475 # -> use low-level IO
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
476 #
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
477 if mmap is None:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
478 while True:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
479 buf = os.read(fd, CHUNK_SIZE)
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
480 if len(buf) == 0:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
481 break
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
482 h.update(buf)
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
483 else:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
484 # mmap
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
485 if filesize < MAP_CHUNK_SIZE:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
486 mapsize = filesize
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
487 mapoffset = 0
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
488 else:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
489 mapsize = MAP_CHUNK_SIZE
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
490 mapoffset = 0
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
491 rest = filesize
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
492 while rest > 0:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
493 m = mmap.mmap(fd,
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
494 mapsize,
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
495 access=mmap.ACCESS_READ,
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
496 offset=mapoffset)
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
497 try:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
498 h.update(m)
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
499 finally:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
500 m.close()
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
501 rest -= mapsize
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
502 mapoffset += mapsize
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
503 if rest < mapsize:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
504 mapsize = rest
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
505 finally:
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
506 os.close(fd)
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
507 return h.digest()
23
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
508
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
509
232063b73e45 Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents: 22
diff changeset
510 def compute_digest_stream(hashobj, instream):
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
511 """
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
512
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
513 :param hashobj: a :mod:`hashlib` compatible hash algorithm type or factory
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
514 :param instream: a bytes input stream to read the data to be hashed from
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
515 :return: the digest in hex form
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
516 :rtype: str
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
517
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
518 """
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
519 h = hashobj()
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
520 while True:
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
521 buf = instream.read(CHUNK_SIZE)
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
522 if buf is not None:
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
523 if len(buf) == 0:
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
524 break
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
525 h.update(buf)
51
58d5a0b6e5b3 Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 47
diff changeset
526 return h.digest()
2
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
527
5510a39a2d04 Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents: 1
diff changeset
528
19
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
529 def normalize_filename(filename, strip_leading_dot_slash=False):
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
530 filename = filename.replace("\\", "/")
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
531 if strip_leading_dot_slash:
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
532 while filename.startswith("./"):
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
533 filename = filename[2:]
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
534 return filename
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
535
2f9e702e3f7a Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents: 18
diff changeset
536
1
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
537 if __name__ == "__main__":
bbf4e0f5b651 Begin the shasum.py script
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
538 sys.exit(main())