Mercurial > hgrepos > Python > apps > py-cutils
annotate cutils/shasum.py @ 177:089c40240061
Add an alternate implementation for generating directory tree digests:
- Do not use something like os.walk() but use os.scandir() directly.
- Recursively generate the subdirectory digests only when needed and in
the right order.
This fixes that the order of subdirectories in the output did not
match the application order of its directory digests.
The new implementation also should make filtering (that will be
implemented later) easier.
NOTE: The tree digests of the old and the new implementation are identical.
| author | Franz Glasner <fzglas.hg@dom66.de> |
|---|---|
| date | Sat, 11 Jan 2025 17:41:28 +0100 |
| parents | ffd14e2de130 |
| children | f04d4b1c14b3 |
| rev | line source |
|---|---|
|
73
c3268f4e752f
Adjust all license notes to (a) more literally comply with the BSD3 templates and to the style guide
Franz Glasner <fzglas.hg@dom66.de>
parents:
72
diff
changeset
|
1 # -*- coding: utf-8 -*- |
|
c3268f4e752f
Adjust all license notes to (a) more literally comply with the BSD3 templates and to the style guide
Franz Glasner <fzglas.hg@dom66.de>
parents:
72
diff
changeset
|
2 # :- |
|
122
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
3 # :Copyright: (c) 2020-2025 Franz Glasner |
|
73
c3268f4e752f
Adjust all license notes to (a) more literally comply with the BSD3 templates and to the style guide
Franz Glasner <fzglas.hg@dom66.de>
parents:
72
diff
changeset
|
4 # :License: BSD-3-Clause |
|
c3268f4e752f
Adjust all license notes to (a) more literally comply with the BSD3 templates and to the style guide
Franz Glasner <fzglas.hg@dom66.de>
parents:
72
diff
changeset
|
5 # :- |
| 113 | 6 r"""Pure Python implementation of `shasum`. |
| 1 | 7 |
| 8 """ | |
| 9 | |
|
72
ae2df602beb4
Make shasum.py and dos2unix sub-modules to the new "cutils" package
Franz Glasner <fzglas.hg@dom66.de>
parents:
71
diff
changeset
|
10 from __future__ import print_function, absolute_import |
|
13
db64e282b049
Implement a version option
Franz Glasner <fzglas.hg@dom66.de>
parents:
12
diff
changeset
|
11 |
|
73
c3268f4e752f
Adjust all license notes to (a) more literally comply with the BSD3 templates and to the style guide
Franz Glasner <fzglas.hg@dom66.de>
parents:
72
diff
changeset
|
12 |
|
c3268f4e752f
Adjust all license notes to (a) more literally comply with the BSD3 templates and to the style guide
Franz Glasner <fzglas.hg@dom66.de>
parents:
72
diff
changeset
|
13 __all__ = [] |
|
c3268f4e752f
Adjust all license notes to (a) more literally comply with the BSD3 templates and to the style guide
Franz Glasner <fzglas.hg@dom66.de>
parents:
72
diff
changeset
|
14 |
|
13
db64e282b049
Implement a version option
Franz Glasner <fzglas.hg@dom66.de>
parents:
12
diff
changeset
|
15 |
| 1 | 16 import argparse |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
17 import base64 |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
18 import binascii |
|
66
c52e5f86b0ab
Handle EAGAIN and EWOULDBLOCK when reading files
Franz Glasner <fzglas.hg@dom66.de>
parents:
61
diff
changeset
|
19 import errno |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
20 import io |
|
23
232063b73e45
Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents:
22
diff
changeset
|
21 import os |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
22 import re |
| 1 | 23 import sys |
| 24 | |
|
118
12339ac2148d
Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents:
117
diff
changeset
|
25 from . import (__version__, __revision__) |
|
12339ac2148d
Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents:
117
diff
changeset
|
26 from . import util |
|
122
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
27 from .util import digest |
|
95
fc2dd6afd594
Style: canonical global variable and import ordering
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
94
diff
changeset
|
28 |
| 1 | 29 |
| 30 def main(argv=None): | |
| 31 aparser = argparse.ArgumentParser( | |
| 32 description="Python implementation of shasum", | |
| 33 fromfile_prefix_chars='@') | |
|
2
5510a39a2d04
Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents:
1
diff
changeset
|
34 aparser.add_argument( |
|
118
12339ac2148d
Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents:
117
diff
changeset
|
35 "--algorithm", "-a", action="store", type=util.argv2algo, |
| 167 | 36 help="""1 (default, aka sha1), 224, 256, 384, 512, |
| 37 3 (alias for sha3-512), 3-224, 3-256, 3-384, 3-512, | |
| 38 blake2b, blake2b-256, blake2s, blake2 (alias for blake2b), | |
| 39 blake2-256 (alias for blake2b-256), md5""") | |
|
7
47b4c98e4d40
Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents:
6
diff
changeset
|
40 aparser.add_argument( |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
41 "--base64", action="store_true", |
|
69
a23371a8780f
Writing style: Begin all help messages with an uppercase letter
Franz Glasner <fzglas.hg@dom66.de>
parents:
68
diff
changeset
|
42 help="Output checksums in base64 notation, not hexadecimal (OpenBSD).") |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
43 aparser.add_argument( |
| 167 | 44 "--binary", "-b", action="store_false", dest="text_mode", |
| 45 default=False, | |
|
69
a23371a8780f
Writing style: Begin all help messages with an uppercase letter
Franz Glasner <fzglas.hg@dom66.de>
parents:
68
diff
changeset
|
46 help="Read in binary mode (default)") |
|
3
5a6ed622846c
Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents:
2
diff
changeset
|
47 aparser.add_argument( |
|
9
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
48 "--bsd", "-B", action="store_true", dest="bsd", default=False, |
| 167 | 49 help="""Write BSD style output. This is also the default output format |
| 50 of :command:`openssl dgst`.""") | |
|
9
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
51 aparser.add_argument( |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
52 "--check", "-c", action="store_true", |
| 17 | 53 help="""Read digests from FILEs and check them. |
| 54 If this option is specified, the FILE options become checklists. Each | |
| 55 checklist should contain hash results in a supported format, which will | |
| 56 be verified against the specified paths. Output consists of the digest | |
| 57 used, the file name, and an OK, FAILED, or MISSING for the result of | |
| 58 the comparison. This will validate any of the supported checksums. | |
| 59 If no file is given, stdin is used.""") | |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
60 aparser.add_argument( |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
61 "--checklist", "-C", metavar="CHECKLIST", |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
62 help="""Compare the checksum of each FILE against the checksums in |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
63 the CHECKLIST. Any specified FILE that is not listed in the CHECKLIST will |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
64 generate an error.""") |
|
83
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
65 aparser.add_argument( |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
66 "--checklist-allow-distinfo", action="store_true", |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
67 dest="allow_distinfo", |
| 167 | 68 help='''Allow FreeBSD "distinfo" formatted checklists: |
| 69 ignore SIZE and TIMESTAMP lines.''') | |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
70 |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
71 aparser.add_argument( |
|
105
b0631f320efd
Implement "--follow-symlinks" to allow to control whether to follow directory symlinks
Franz Glasner <fzglas.hg@dom66.de>
parents:
104
diff
changeset
|
72 "--follow-symlinks", action="store_true", dest="follow_symlinks", |
| 167 | 73 help="""Also follow symlinks that resolve to directories. |
| 74 Only effective if `--recurse` is activated.""") | |
|
105
b0631f320efd
Implement "--follow-symlinks" to allow to control whether to follow directory symlinks
Franz Glasner <fzglas.hg@dom66.de>
parents:
104
diff
changeset
|
75 |
|
b0631f320efd
Implement "--follow-symlinks" to allow to control whether to follow directory symlinks
Franz Glasner <fzglas.hg@dom66.de>
parents:
104
diff
changeset
|
76 aparser.add_argument( |
|
90
42419f57eda9
Allow to control the use of mmap from the command-line
Franz Glasner <fzglas.hg@dom66.de>
parents:
89
diff
changeset
|
77 "--mmap", action="store_true", dest="mmap", default=None, |
|
42419f57eda9
Allow to control the use of mmap from the command-line
Franz Glasner <fzglas.hg@dom66.de>
parents:
89
diff
changeset
|
78 help="""Use mmap if available. Default is to determine automatically |
|
42419f57eda9
Allow to control the use of mmap from the command-line
Franz Glasner <fzglas.hg@dom66.de>
parents:
89
diff
changeset
|
79 from the filesize.""") |
|
42419f57eda9
Allow to control the use of mmap from the command-line
Franz Glasner <fzglas.hg@dom66.de>
parents:
89
diff
changeset
|
80 aparser.add_argument( |
|
42419f57eda9
Allow to control the use of mmap from the command-line
Franz Glasner <fzglas.hg@dom66.de>
parents:
89
diff
changeset
|
81 "--no-mmap", action="store_false", dest="mmap", default=None, |
|
42419f57eda9
Allow to control the use of mmap from the command-line
Franz Glasner <fzglas.hg@dom66.de>
parents:
89
diff
changeset
|
82 help="Dont use mmap.") |
|
42419f57eda9
Allow to control the use of mmap from the command-line
Franz Glasner <fzglas.hg@dom66.de>
parents:
89
diff
changeset
|
83 |
|
42419f57eda9
Allow to control the use of mmap from the command-line
Franz Glasner <fzglas.hg@dom66.de>
parents:
89
diff
changeset
|
84 aparser.add_argument( |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
85 "--recurse", action="store_true", |
| 167 | 86 help="""Recurse into sub-directories while interpreting every |
| 87 FILE as a directory.""") | |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
88 |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
89 aparser.add_argument( |
|
9
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
90 "--reverse", "-r", action="store_false", dest="bsd", default=False, |
| 167 | 91 help="""Explicitely select normal coreutils style output |
| 92 (to be option compatible with BSD style commands and | |
| 93 :command:`openssl dgst -r`)""") | |
|
9
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
94 aparser.add_argument( |
|
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
95 "--tag", action="store_true", dest="bsd", default=False, |
| 167 | 96 help="""Alias for the `--bsd' option (to be compatible with |
| 97 :command:`b2sum`)""") | |
|
5
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
98 aparser.add_argument( |
|
3
5a6ed622846c
Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents:
2
diff
changeset
|
99 "--text", "-t", action="store_true", dest="text_mode", default=False, |
|
69
a23371a8780f
Writing style: Begin all help messages with an uppercase letter
Franz Glasner <fzglas.hg@dom66.de>
parents:
68
diff
changeset
|
100 help="Read in text mode (not supported)") |
|
3
5a6ed622846c
Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents:
2
diff
changeset
|
101 aparser.add_argument( |
| 167 | 102 "--version", "-v", action="version", |
| 103 version="%s (rv:%s)" % (__version__, __revision__)) | |
|
13
db64e282b049
Implement a version option
Franz Glasner <fzglas.hg@dom66.de>
parents:
12
diff
changeset
|
104 aparser.add_argument( |
|
2
5510a39a2d04
Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents:
1
diff
changeset
|
105 "files", nargs="*", metavar="FILE") |
| 1 | 106 |
| 107 opts = aparser.parse_args(args=argv) | |
| 108 | |
|
3
5a6ed622846c
Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents:
2
diff
changeset
|
109 if opts.text_mode: |
|
5a6ed622846c
Add comman line switches for reading in binary and text mode
Franz Glasner <fzglas.hg@dom66.de>
parents:
2
diff
changeset
|
110 print("ERROR: text mode not supported", file=sys.stderr) |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
111 sys.exit(78) # :manpage:`sysexits(3)` EX_CONFIG |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
112 |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
113 if opts.check and opts.checklist: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
114 print("ERROR: only one of --check or --checklist allowed", |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
115 file=sys.stderr) |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
116 sys.exit(64) # :manpage:`sysexits(3)` EX_USAGE |
|
5
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
117 |
|
7
47b4c98e4d40
Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents:
6
diff
changeset
|
118 if not opts.algorithm: |
|
118
12339ac2148d
Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents:
117
diff
changeset
|
119 opts.algorithm = util.argv2algo("1") |
|
7
47b4c98e4d40
Allow algorithm selection for all algorithms in :mod:`hashlib`.
Franz Glasner <fzglas.hg@dom66.de>
parents:
6
diff
changeset
|
120 |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
121 opts.dest = None |
|
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
122 |
|
45
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
123 return shasum(opts) |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
124 |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
125 |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
126 def gen_opts(files=[], algorithm="SHA1", bsd=False, text_mode=False, |
|
100
f95918115c6b
FIX: Implement some new commandline flags in "gen_opts()" also
Franz Glasner <fzglas.hg@dom66.de>
parents:
96
diff
changeset
|
127 checklist=False, check=False, dest=None, base64=False, |
|
106
5fe6f63f0be7
Implement "--recurse" and "--follow-symlinks" in "gen_opts()" also
Franz Glasner <fzglas.hg@dom66.de>
parents:
105
diff
changeset
|
128 allow_distinfo=False, mmap=None, recurse=False, |
|
5fe6f63f0be7
Implement "--recurse" and "--follow-symlinks" in "gen_opts()" also
Franz Glasner <fzglas.hg@dom66.de>
parents:
105
diff
changeset
|
129 follow_symlinks=False): |
|
45
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
130 if text_mode: |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
131 raise ValueError("text mode not supported") |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
132 if checklist and check: |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
133 raise ValueError("only one of `checklist' or `check' is allowed") |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
134 opts = argparse.Namespace(files=files, |
|
118
12339ac2148d
Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents:
117
diff
changeset
|
135 algorithm=(util.algotag2algotype(algorithm), |
|
45
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
136 algorithm), |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
137 bsd=bsd, |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
138 checklist=checklist, |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
139 check=check, |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
140 text_mode=False, |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
141 dest=dest, |
|
100
f95918115c6b
FIX: Implement some new commandline flags in "gen_opts()" also
Franz Glasner <fzglas.hg@dom66.de>
parents:
96
diff
changeset
|
142 base64=base64, |
|
f95918115c6b
FIX: Implement some new commandline flags in "gen_opts()" also
Franz Glasner <fzglas.hg@dom66.de>
parents:
96
diff
changeset
|
143 allow_distinfo=allow_distinfo, |
|
106
5fe6f63f0be7
Implement "--recurse" and "--follow-symlinks" in "gen_opts()" also
Franz Glasner <fzglas.hg@dom66.de>
parents:
105
diff
changeset
|
144 mmap=mmap, |
|
5fe6f63f0be7
Implement "--recurse" and "--follow-symlinks" in "gen_opts()" also
Franz Glasner <fzglas.hg@dom66.de>
parents:
105
diff
changeset
|
145 recurse=recurse, |
|
5fe6f63f0be7
Implement "--recurse" and "--follow-symlinks" in "gen_opts()" also
Franz Glasner <fzglas.hg@dom66.de>
parents:
105
diff
changeset
|
146 follow_symlinks=follow_symlinks) |
|
45
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
147 return opts |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
148 |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
149 |
|
b25ef7293bf2
Enhance shasum.py to allow it to be used as Python module from within other programs more easily
Franz Glasner <fzglas.hg@dom66.de>
parents:
43
diff
changeset
|
150 def shasum(opts): |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
151 if opts.check: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
152 return verify_digests_from_files(opts) |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
153 elif opts.checklist: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
154 return verify_digests_with_checklist(opts) |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
155 else: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
156 return generate_digests(opts) |
|
10
77446cd3ea6f
Move the digest generation loop from "main()" into an own function "generate_digests()".
Franz Glasner <fzglas.hg@dom66.de>
parents:
9
diff
changeset
|
157 |
|
77446cd3ea6f
Move the digest generation loop from "main()" into an own function "generate_digests()".
Franz Glasner <fzglas.hg@dom66.de>
parents:
9
diff
changeset
|
158 |
|
77446cd3ea6f
Move the digest generation loop from "main()" into an own function "generate_digests()".
Franz Glasner <fzglas.hg@dom66.de>
parents:
9
diff
changeset
|
159 def generate_digests(opts): |
|
5
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
160 if opts.bsd: |
|
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
161 out = out_bsd |
|
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
162 else: |
|
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
163 out = out_std |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
164 if opts.recurse: |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
165 if not opts.files: |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
166 opts.files.append(".") |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
167 for dn in opts.files: |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
168 if not os.path.isdir(dn): |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
169 if os.path.exists(dn): |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
170 raise OSError(errno.ENOTDIR, "not a directory", dn) |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
171 else: |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
172 raise OSError(errno.ENOENT, "directory does not exist", dn) |
|
105
b0631f320efd
Implement "--follow-symlinks" to allow to control whether to follow directory symlinks
Franz Glasner <fzglas.hg@dom66.de>
parents:
104
diff
changeset
|
173 for dirpath, dirnames, dirfiles in os.walk( |
|
b0631f320efd
Implement "--follow-symlinks" to allow to control whether to follow directory symlinks
Franz Glasner <fzglas.hg@dom66.de>
parents:
104
diff
changeset
|
174 dn, followlinks=opts.follow_symlinks): |
|
110
3060aa4cb252
When generating digests recursively normalize the output somewhat by sorting directory names and filenames
Franz Glasner <fzglas.hg@dom66.de>
parents:
106
diff
changeset
|
175 dirnames.sort() |
|
3060aa4cb252
When generating digests recursively normalize the output somewhat by sorting directory names and filenames
Franz Glasner <fzglas.hg@dom66.de>
parents:
106
diff
changeset
|
176 dirfiles.sort() |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
177 for fn in dirfiles: |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
178 path = os.path.join(dirpath, fn) |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
179 out(opts.dest or sys.stdout, |
|
122
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
180 digest.compute_digest_file( |
|
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
181 opts.algorithm[0], path, use_mmap=opts.mmap), |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
182 path, |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
183 opts.algorithm[1], |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
184 True, |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
185 opts.base64) |
|
4
67d10529ce88
FIX: "-" filename handling now consistent with Perl shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
3
diff
changeset
|
186 else: |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
187 if not opts.files or (len(opts.files) == 1 and opts.files[0] == '-'): |
|
164
a813094ae4f5
Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents:
122
diff
changeset
|
188 if util.PY2: |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
189 if sys.platform == "win32": |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
190 import msvcrt # noqa: E401 |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
191 msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY) |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
192 source = sys.stdin |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
193 else: |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
194 source = sys.stdin.buffer |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
195 out(sys.stdout, |
|
122
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
196 digest.compute_digest_stream(opts.algorithm[0], source), |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
197 None, |
|
23
232063b73e45
Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents:
22
diff
changeset
|
198 opts.algorithm[1], |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
199 True, |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
200 opts.base64) |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
201 else: |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
202 for fn in opts.files: |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
203 out(opts.dest or sys.stdout, |
|
122
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
204 digest.compute_digest_file( |
|
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
205 opts.algorithm[0], fn, use_mmap=opts.mmap), |
|
104
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
206 fn, |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
207 opts.algorithm[1], |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
208 True, |
|
08fd0609fdd4
Implement "--recurse" for shasum: recurse into directories
Franz Glasner <fzglas.hg@dom66.de>
parents:
100
diff
changeset
|
209 opts.base64) |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
210 return 0 |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
211 |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
212 |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
213 def compare_digests_equal(given_digest, expected_digest, algo): |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
214 """Compare a newly computed binary digest `given_digest` with a digest |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
215 string (hex or base64) in `expected_digest`. |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
216 |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
217 :param bytes given_digest: |
|
71
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
218 :param expected_digest: digest (as bytes) or hexlified or base64 encoded |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
219 digest (as str) |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
220 :type expected_digest: str or bytes or bytearray |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
221 :param algo: The algorithm (factory) |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
222 :return: `True` if the digests are equal, `False` if not |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
223 :rtype: bool |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
224 |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
225 """ |
|
71
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
226 if isinstance(expected_digest, (bytes, bytearray)) \ |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
227 and len(expected_digest) == algo().digest_size: |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
228 exd = expected_digest |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
229 else: |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
230 if len(expected_digest) == algo().digest_size * 2: |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
231 # hex |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
232 if re.search(r"\A[a-fA-F0-9]+\Z", expected_digest): |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
233 try: |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
234 exd = binascii.unhexlify(expected_digest) |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
235 except TypeError: |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
236 return False |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
237 else: |
|
52
5935055edea6
More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
51
diff
changeset
|
238 return False |
|
5935055edea6
More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
51
diff
changeset
|
239 else: |
|
71
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
240 # base64 |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
241 if re.search( |
| 167 | 242 r"\A(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)?\Z", # noqa: E501 line too long |
|
71
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
243 expected_digest): |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
244 try: |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
245 exd = base64.b64decode(expected_digest) |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
246 except TypeError: |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
247 return False |
|
29fb33aa639a
"compare_digests_equal()" now accepts binary (aka un-encoded) expected digests also
Franz Glasner <fzglas.hg@dom66.de>
parents:
70
diff
changeset
|
248 else: |
|
52
5935055edea6
More proper formal checks (with regexp) for valid hex and base64 encoding of digests
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
51
diff
changeset
|
249 return False |
|
75
a31de3c65877
Remove the use of "hmac.compare_digest()": there are no secrets to protect here
Franz Glasner <fzglas.hg@dom66.de>
parents:
73
diff
changeset
|
250 return given_digest == exd |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
251 |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
252 |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
253 def verify_digests_with_checklist(opts): |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
254 dest = opts.dest or sys.stdout |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
255 exit_code = 0 |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
256 if not opts.files or (len(opts.files) == 1 and opts.files[0] == '-'): |
|
164
a813094ae4f5
Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents:
122
diff
changeset
|
257 if util.PY2: |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
258 if sys.platform == "win32": |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
259 import os, msvcrt # noqa: E401 |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
260 msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY) |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
261 source = sys.stdin |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
262 else: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
263 source = sys.stdin.buffer |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
264 pl = get_parsed_digest_line_from_checklist(opts.checklist, opts, None) |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
265 if pl is None: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
266 exit_code = 1 |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
267 print("-: MISSING", file=dest) |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
268 else: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
269 tag, algo, cl_filename, cl_digest = pl |
|
122
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
270 computed_digest = digest.compute_digest_stream(algo, source) |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
271 if compare_digests_equal(computed_digest, cl_digest, algo): |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
272 res = "OK" |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
273 else: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
274 res = "FAILED" |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
275 exit_code = 1 |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
276 print("{}: {}: {}".format(tag, "-", res), file=dest) |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
277 else: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
278 for fn in opts.files: |
| 167 | 279 pl = get_parsed_digest_line_from_checklist( |
| 280 opts.checklist, opts, fn) | |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
281 if pl is None: |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
282 print("{}: MISSING".format(fn), file=dest) |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
283 exit_code = 1 |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
284 else: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
285 tag, algo, cl_filename, cl_digest = pl |
|
122
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
286 computed_digest = digest.compute_digest_file( |
|
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
287 algo, fn, use_mmap=opts.mmap) |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
288 if compare_digests_equal(computed_digest, cl_digest, algo): |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
289 res = "OK" |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
290 else: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
291 exit_code = 1 |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
292 res = "FAILED" |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
293 print("{}: {}: {}".format(tag, fn, res), file=dest) |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
294 return exit_code |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
295 |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
296 |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
297 def verify_digests_from_files(opts): |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
298 dest = opts.dest or sys.stdout |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
299 exit_code = 0 |
|
20
8f0241ed4a00
Do not append "-" to an empty FILE list any more but check explicitely for an empty list
Franz Glasner <fzglas.hg@dom66.de>
parents:
19
diff
changeset
|
300 if not opts.files or (len(opts.files) == 1 and opts.files[0] == '-'): |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
301 for checkline in sys.stdin: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
302 if not checkline: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
303 continue |
|
18
285848db0b52
When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents:
17
diff
changeset
|
304 r, fn, tag = handle_checkline(opts, checkline) |
|
83
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
305 if tag in ("SIZE", "TIMESTAMP"): |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
306 assert opts.allow_distinfo |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
307 continue |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
308 print("{}: {}: {}".format(tag, fn, r.upper()), file=dest) |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
309 if r != "ok" and exit_code == 0: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
310 exit_code = 1 |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
311 else: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
312 for fn in opts.files: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
313 with io.open(fn, "rt", encoding="utf-8") as checkfile: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
314 for checkline in checkfile: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
315 if not checkline: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
316 continue |
|
18
285848db0b52
When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents:
17
diff
changeset
|
317 r, fn, tag = handle_checkline(opts, checkline) |
|
83
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
318 if tag in ("SIZE", "TIMESTAMP"): |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
319 assert opts.allow_distinfo |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
320 continue |
|
47
5bec7a5d894a
Allow internal output redirection: print() always to explicitely given file objects
Franz Glasner <fzglas.hg@dom66.de>
parents:
45
diff
changeset
|
321 print("{}: {}: {}".format(tag, fn, r.upper()), file=dest) |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
322 if r != "ok" and exit_code == 0: |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
323 exit_code = 1 |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
324 return exit_code |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
325 |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
326 |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
327 def handle_checkline(opts, line): |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
328 """ |
|
18
285848db0b52
When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents:
17
diff
changeset
|
329 :return: a tuple with static "ok", "missing", or "failed", the filename and |
|
285848db0b52
When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents:
17
diff
changeset
|
330 the digest used |
|
285848db0b52
When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents:
17
diff
changeset
|
331 :rtype: tuple(str, str, str) |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
332 |
|
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
333 """ |
|
21
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
334 parts = parse_digest_line(opts, line) |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
335 if not parts: |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
336 raise ValueError( |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
337 "improperly formatted digest line: {}".format(line)) |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
338 tag, algo, fn, digest = parts |
|
83
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
339 if tag in ("SIZE", "TIMESTAMP"): |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
340 assert opts.allow_distinfo |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
341 return (None, None, tag) |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
342 try: |
|
122
1e5127028254
Move the real computation of digests from files and streams into dedicated submodule cutils.util.digest
Franz Glasner <fzglas.hg@dom66.de>
parents:
120
diff
changeset
|
343 d = digest.compute_digest_file(algo, fn, use_mmap=opts.mmap) |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
344 if compare_digests_equal(d, digest, algo): |
|
23
232063b73e45
Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents:
22
diff
changeset
|
345 return ("ok", fn, tag) |
|
232063b73e45
Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents:
22
diff
changeset
|
346 else: |
|
232063b73e45
Optimized reading of files by using mmap.
Franz Glasner <fzglas.hg@dom66.de>
parents:
22
diff
changeset
|
347 return ("failed", fn, tag) |
|
12
5e2c9123f93f
Implemented digest verification: -c or --check option
Franz Glasner <fzglas.hg@dom66.de>
parents:
11
diff
changeset
|
348 except EnvironmentError: |
|
18
285848db0b52
When verifying/checking digests: also print the digest tag used
Franz Glasner <fzglas.hg@dom66.de>
parents:
17
diff
changeset
|
349 return ("missing", fn, tag) |
|
5
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
350 |
|
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
351 |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
352 def get_parsed_digest_line_from_checklist(checklist, opts, filename): |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
353 if filename is None: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
354 filenames = ("-", "stdin", "", ) |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
355 else: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
356 filenames = ( |
|
118
12339ac2148d
Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents:
117
diff
changeset
|
357 util.normalize_filename(filename, strip_leading_dot_slash=True),) |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
358 with io.open(checklist, "rt", encoding="utf-8") as clf: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
359 for checkline in clf: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
360 if not checkline: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
361 continue |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
362 parts = parse_digest_line(opts, checkline) |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
363 if not parts: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
364 raise ValueError( |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
365 "improperly formatted digest line: {}".format(checkline)) |
|
83
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
366 if parts[0] in ("SIZE", "TIMESTAMP"): |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
367 assert opts.allow_distinfo |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
368 continue |
| 167 | 369 fn = util.normalize_filename( |
| 370 parts[2], strip_leading_dot_slash=True) | |
|
22
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
371 if fn in filenames: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
372 return parts |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
373 else: |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
374 return None |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
375 |
|
6bdfc5ad4656
Implemented OpenBSD's -C (aka --checklist) option for shasum
Franz Glasner <fzglas.hg@dom66.de>
parents:
21
diff
changeset
|
376 |
|
21
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
377 def parse_digest_line(opts, line): |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
378 """Parse a `line` of a digest file and return its parts. |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
379 |
|
83
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
380 This is rather strict. But if `opts.allow_distinfo` is `True` then |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
381 some additional keywords ``SIZE`` and ``TIMESTAMP``are recignized |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
382 and returned. The caller is responsible to handle them. |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
383 |
|
21
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
384 :return: a tuple of the normalized algorithm tag, the algorithm |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
385 constructor, the filename and the hex digest; |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
386 if `line` cannot be parsed successfully `None` is returned |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
387 :rtype: tuple(str, obj, str, str) or None |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
388 |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
389 Handles coreutils and BSD-style file formats. |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
390 |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
391 """ |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
392 # determine checkfile format (BSD or coreutils) |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
393 # BSD? |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
394 mo = re.search(r"\A(\S+)\s*\((.*)\)\s*=\s*(.+)\n?\Z", line) |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
395 if mo: |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
396 # (tag, algorithm, filename, digest) |
|
83
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
397 if opts.allow_distinfo: |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
398 if mo.group(1) == "SIZE": |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
399 return ("SIZE", None, None, mo.group(3)) |
|
21
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
400 return (mo.group(1), |
|
118
12339ac2148d
Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents:
117
diff
changeset
|
401 util.algotag2algotype(mo.group(1)), |
|
21
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
402 mo.group(2), |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
403 mo.group(3)) |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
404 else: |
|
83
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
405 if opts.allow_distinfo: |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
406 mo = re.search(r"\ATIMESTAMP\s*=\s*([0-9]+)\s*\n\Z", line) |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
407 if mo: |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
408 return ("TIMESTAMP", None, None, mo.group(1)) |
|
05e2bf4796fd
Add an option "--checklist-allow-distinfo" to allow FreeBSD "distinfo" formatted files as checkfiles.
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
79
diff
changeset
|
409 |
|
21
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
410 # coreutils? |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
411 mo = re.search(r"([^\ ]+) [\*\ ]?(.+)\n?\Z", line) |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
412 if mo: |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
413 # (tag, algorithm, filename, digest) |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
414 return (opts.algorithm[1], |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
415 opts.algorithm[0], |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
416 mo.group(2), |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
417 mo.group(1)) |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
418 else: |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
419 return None |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
420 |
|
f2d634270e1c
Refactor: parse a line of a digest file within a dedicated funcion
Franz Glasner <fzglas.hg@dom66.de>
parents:
20
diff
changeset
|
421 |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
422 def out_bsd(dest, digest, filename, digestname, binary, use_base64): |
|
9
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
423 """BSD format output, also :command:`openssl dgst` and |
|
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
424 :command:`b2sum --tag" format output |
|
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
425 |
|
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
426 """ |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
427 if use_base64: |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
428 digest = base64.b64encode(digest).decode("ascii") |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
429 else: |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
430 digest = binascii.hexlify(digest).decode("ascii") |
|
5
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
431 if filename is None: |
|
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
432 print(digest, file=dest) |
|
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
433 else: |
|
19
2f9e702e3f7a
Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents:
18
diff
changeset
|
434 print("{} ({}) = {}".format(digestname, |
|
118
12339ac2148d
Move some functions into cutils.util (i.e. algorithms and their aliases)
Franz Glasner <fzglas.hg@dom66.de>
parents:
117
diff
changeset
|
435 util.normalize_filename(filename), |
|
19
2f9e702e3f7a
Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents:
18
diff
changeset
|
436 digest), |
|
5
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
437 file=dest) |
|
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
438 |
|
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
439 |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
440 def out_std(dest, digest, filename, digestname, binary, use_base64): |
|
9
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
441 """Coreutils format (:command:`shasum` et al.) |
|
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
442 |
|
81f28bf89c26
Some more output selection options and documentation
Franz Glasner <fzglas.hg@dom66.de>
parents:
8
diff
changeset
|
443 """ |
|
51
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
444 if use_base64: |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
445 digest = base64.b64encode(digest).decode("ascii") |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
446 else: |
|
58d5a0b6e5b3
Implement the OpenBSD variant (with --base64) to encode digests in base64, not hexadecimal
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
47
diff
changeset
|
447 digest = binascii.hexlify(digest).decode("ascii") |
|
19
2f9e702e3f7a
Convert backslashes in filenames to forward slashes when creating digests
Franz Glasner <fzglas.hg@dom66.de>
parents:
18
diff
changeset
|
448 print("{} {}{}".format( |
| 167 | 449 digest, |
| 450 '*' if binary else ' ', | |
| 451 '-' if filename is None else util.normalize_filename(filename)), | |
|
5
bbcb225640de
Handle standard and BSD-style output formats
Franz Glasner <fzglas.hg@dom66.de>
parents:
4
diff
changeset
|
452 file=dest) |
|
2
5510a39a2d04
Basic hashing with proper binary stdin/stdout support for Py2, Py3 and Windows
Franz Glasner <fzglas.hg@dom66.de>
parents:
1
diff
changeset
|
453 |
| 1 | 454 |
| 455 if __name__ == "__main__": | |
| 456 sys.exit(main()) |