Python/apps/py-cutils: cutils/treesum.py annotate

annotate cutils/treesum.py @ 270:42f4ca423ab3

treesum: REFACTOR: Major refactoring of computing digests: - Digests that are created by reading symlink targets are created as if the result of readllink() is the content of a regulare file: no special tagging is done now. - All digest values have their canonical algorithm name prepended. - Normalize the order of computation

author	Franz Glasner <fzglas.hg@dom66.de>
date	Tue, 18 Feb 2025 16:07:28 +0100
parents	fc002983253c
children	6fe88de236cb

rev	line source
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1 # -- coding: utf-8 --
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	3 # :Copyright: (c) 2020-2025 Franz Glasner
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	4 # :License: BSD-3-Clause
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	5 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	6 r"""Generate and verify checksums for directory trees.
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	7
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	8 """
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	9
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	10 from __future__ import print_function, absolute_import
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	11
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	12
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	13 __all__ = []
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	14
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	15
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	16 import argparse
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	17 import base64
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	18 import binascii
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	19 import collections
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	20 import datetime
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	21 import errno
176 7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	22 import logging
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	23 import os
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	24 import re
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	25 import stat
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	26 import sys
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	27 import time
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	28
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	29 from . import (__version__, __revision__)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	30 from . import util
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	31 from .util import cm
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	32 from .util import digest
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	33 from .util import walk
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	34 from .util.crc32 import crc32
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	35
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	36
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	37 def main(argv=None):
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	38
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	39 def _populate_generate_arguments(gp):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	40 """Use to populate command aliases.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	41
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	42 This is because :class:`argparse.ArgumentParser` does not
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	43 support them for all supported Python versions.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	44
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	45 """
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	46 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	47 "--algorithm", "-a", action="store", type=util.argv2algo,
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	48 help="1 (aka sha1), 224, 256 (aka sha256), 384, 512 (aka sha512), "
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	49 "3 (alias for sha3-512), 3-224, 3-256, 3-384, 3-512, "
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	50 "blake2b, blake2b-256, blake2s, "
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	51 "blake2 (alias for blake2b), "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	52 "blake2-256 (alias for blake2b-256), "
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	53 "md5. "
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	54 "The default depends on availability in hashlib: "
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	55 "blake2b-256, sha256 or sha1.")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	56 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	57 "--append-output", action="store_true", dest="append_output",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	58 help="Append to the output file instead of overwriting it.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	59 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	60 "--base64", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	61 help="Output checksums in base64 notation, not hexadecimal "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	62 "(OpenBSD).")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	63 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	64 "--comment", action="append", default=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	65 help="Put given comment COMMENT into the output as \"COMMENT\". "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	66 "Can be given more than once.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	67 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	68 "--follow-directory-symlinks", "-l", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	69 const="follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	70 default=FollowSymlinkConfig(False, False, True),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	71 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	72 help="""Follow symbolic links to directories when walking a
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	73 directory tree. Augments --physical and -p.""")
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	74 gp.add_argument(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	75 "--follow-file-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	76 const="follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	77 default=FollowSymlinkConfig(False, False, True),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	78 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	79 help="""Follow symbolic links to files when walking a
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	80 directory tree. Augments --physical.""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	81 gp.add_argument(
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	82 "--full-mode", action="store_true", dest="metadata_full_mode",
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	83 help="Consider all mode bits as returned from stat(2) when "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	84 "computing directory digests. "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	85 "Note that mode bits on symbolic links itself are not "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	86 "considered.")
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	87 gp.add_argument(
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	88 "--generator", choices=("normal", "full", "none"),
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	89 default="normal",
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	90 help="""Put a `GENERATOR' line into the output.
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	91 `full' prints full Python and OS/platform version information,
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	92 `normal' prints just whether Python 2 or Python 3 is used, and `none'
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	93 suppresses the output completely. The default is `normal'.""")
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	94 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	95 "--logical", "-L", action=SymlinkAction, dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	96 const=FollowSymlinkConfig(True, True, True),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	97 help="""Follow symbolic links everywhere: on command line
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	98 arguments and -- while walking -- directory and file symbolic links.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	99 Overwrites any other symlink related options
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	100 (--physical,-p, no-follow-directory-symlinks, no-follow-file-symlinks,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	101 et al.).
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	102 """)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	103 gp.add_argument(
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	104 "--minimal", nargs="?", const="", default=None, metavar="TAG",
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	105 help="Produce minimal output only. If a TAG is given and not "
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	106 "empty use it as the leading \"ROOT (<TAG>)\" output.")
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	107 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	108 "--mmap", action="store_true", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	109 help="Use mmap if available. Default is to determine "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	110 "automatically from the filesize.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	111 gp.add_argument(
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	112 "--mode", action="store_true", dest="metadata_mode",
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	113 help="Consider the permission bits of stat(2) using S_IMODE (i.e. "
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	114 "all bits without the filetype bits) when "
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	115 "computing directory digests. Note that mode bits on "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	116 "symbolic links itself are not considered.")
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	117 gp.add_argument(
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	118 "--mtime", action="store_true", dest="metadata_mtime",
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	119 help="Consider the mtime of files (non-directories) when "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	120 "generating digests for directories. Digests for files are "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	121 "not affected.")
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	122 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	123 "--no-follow-directory-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	124 const="no-follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	125 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	126 help="""Do not follow symbolic links to directories when walking a
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	127 directory tree. Augments --logical.""")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	128 gp.add_argument(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	129 "--no-follow-file-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	130 const="no-follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	131 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	132 help="""Dont follow symbolic links to files when walking a
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	133 directory tree. Augments --logical and -p.""")
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	134 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	135 "--no-mmap", action="store_false", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	136 help="Dont use mmap.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	137 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	138 "--output", "-o", action="store", metavar="OUTPUT",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	139 help="Put the checksum into given file. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	140 "If not given or if it is given as `-' then stdout is used.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	141 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	142 "--physical", "-P", action=SymlinkAction, dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	143 const=FollowSymlinkConfig(False, False, False),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	144 help="""Do not follow any symbolic links whether they are given
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	145 on the command line or when walking the directory tree.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	146 Overwrites any other symlink related options
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	147 (--logical, -p, follow-directory-symlinks, follow-file-symlinks, et al.).
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	148 This is the default.""")
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	149 gp.add_argument(
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	150 "-p", action=SymlinkAction, dest="follow_symlinks",
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	151 const=FollowSymlinkConfig(False, False, True),
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	152 help="""Do not follow any symbolic links to directories,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	153 whether they are given on the command line or when walking the directory tree,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	154 but follow symbolic links to files.
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	155 Overwrites any other symlink related options
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	156 (--logical, --physical, follow-directory-symlinks, no-follow-file-symlinks,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	157 et al.).
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	158 This is the default.""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	159 gp.add_argument(
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	160 "--print-size", action="store_true",
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	161 help="""Print the size of a file or the accumulated sizes of
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	162 directory content into the output also.
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	163 The size is not considered when computing digests. For symbolic links
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	164 the size is not printed also.""")
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	165 gp.add_argument(
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	166 "--size-only", action="store_true",
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	167 help="""Print only the size of files and for each directory its
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	168 accumulated directory size. Digests are not computed.""")
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	169 gp.add_argument(
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	170 "--utf8", "--utf-8", action="store_true",
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	171 help="""Encode all file paths using UTF-8 instead of
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	172 the filesystem encoding. Add some error tag into the path if it cannot
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	173 representated in Unicode cleanly.""")
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	174 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	175 "directories", nargs="*", metavar="DIRECTORY")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	176
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	177 def _populate_info_arguments(ip):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	178 ip.add_argument(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	179 "--last", action="store_true", dest="print_only_last_block",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	180 help="Print only the last block of every given input file")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	181 ip.add_argument(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	182 "digest_files", nargs="+", metavar="TREESUM-DIGEST-FILE")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	183
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	184 parser = argparse.ArgumentParser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	185 description="Generate and verify checksums for directory trees.",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	186 fromfile_prefix_chars='@',
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	187 add_help=False)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	188
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	189 #
153 3505406ef9f3 Comment Franz Glasner <fzglas.hg@dom66.de> parents: 152 diff changeset	190 # Global options for all sub-commands.
3505406ef9f3 Comment Franz Glasner <fzglas.hg@dom66.de> parents: 152 diff changeset	191 # In a group because this allows a customized title.
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	192 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	193 gparser = parser.add_argument_group(title="Global Options")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	194 gparser.add_argument(
191 1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	195 "--debug", action="store_true",
1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	196 help="Activate debug logging to stderr")
1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	197 gparser.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	198 "-v", "--version", action="version",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	199 version="%s (rv:%s)" % (__version__, __revision__),
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	200 help="Show program's version number and exit")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	201 gparser.add_argument(
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	202 "-h", "--help", action="help",
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	203 help="Show this help message and exit")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	204
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	205 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	206 # Subcommands
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	207 #
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	208 subparsers = parser.add_subparsers(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	209 dest="subcommand",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	210 title="Commands",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	211 description="This tool uses subcommands. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	212 "To see detailed help for a specific subcommand use "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	213 "the -h/--help option after the subcommand name. "
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	214 "A list of valid commands and their short descriptions "
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	215 "is listed below:",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	216 metavar="COMMAND")
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	217
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	218 genparser = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	219 "generate",
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	220 help="Generate checksums for directory trees",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	221 description="Generate checksums for directory trees.")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	222 _populate_generate_arguments(genparser)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	223 # And an alias for "generate"
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	224 genparser2 = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	225 "gen",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	226 help="Alias for \"generate\"",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	227 description="Generate checksums for directory trees. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	228 "This is an alias to \"generate\".")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	229 _populate_generate_arguments(genparser2)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	230
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	231 infoparser = subparsers.add_parser(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	232 "info",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	233 help="Print some information from given treesum digest file",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	234 description="""Print some informations from given treesum digest files
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	235 to stdout."""
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	236 )
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	237 _populate_info_arguments(infoparser)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	238
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	239 hparser = subparsers.add_parser(
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	240 "help",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	241 help="Show this help message or a subcommand's help and exit",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	242 description="Show this help message or a subcommand's help and exit.")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	243 hparser.add_argument("help_command", nargs='?', metavar="COMMAND")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	244
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	245 vparser = subparsers.add_parser(
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	246 "version",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	247 help="Show the program's version number and exit",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	248 description="Show the program's version number and exit.")
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	249
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	250 # Parse leniently to just check for "version" and/or help
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	251 opts, _dummy = parser.parse_known_args(args=argv)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	252
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	253 if opts.subcommand == "version":
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	254 print("%s (rv:%s)" % (__version__, __revision__),
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	255 file=sys.stdout)
174 fc1055878775 Use "return 0" instead of "sys.exit(0)" when printing version and help# Franz Glasner <fzglas.hg@dom66.de> parents: 173 diff changeset	256 return 0
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	257 if opts.subcommand == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	258 if not opts.help_command:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	259 parser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	260 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	261 if opts.help_command == "generate":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	262 genparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	263 elif opts.help_command == "gen":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	264 genparser2.print_help()
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	265 elif opts.help_command == "info":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	266 infoparser.print_help()
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	267 elif opts.help_command == "version":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	268 vparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	269 elif opts.help_command == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	270 hparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	271 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	272 parser.print_help()
174 fc1055878775 Use "return 0" instead of "sys.exit(0)" when printing version and help# Franz Glasner <fzglas.hg@dom66.de> parents: 173 diff changeset	273 return 0
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	274
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	275 # Reparse strictly
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	276 opts = parser.parse_args(args=argv)
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	277
176 7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	278 # Minimal logging -- just for debugging - not for more "normal" use
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	279 logging.basicConfig(
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	280 level=logging.DEBUG if opts.debug else logging.WARNING,
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	281 stream=sys.stderr,
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	282 format="[%(asctime)s][%(levelname)s][%(process)d:%(name)s] %(message)s"
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	283 )
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	284 logging.captureWarnings(True)
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	285
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	286 return treesum(opts)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	287
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	288
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	289 FollowSymlinkConfig = collections.namedtuple("FollowSymlinkConfig",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	290 ["command_line",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	291 "directory",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	292 "file"])
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	293
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	294
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	295 class SymlinkAction(argparse.Action):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	296
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	297 """`type' is fixed here.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	298 `dest' is a tuple with three items:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	299
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	300 1. follow symlinks on the command line
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	301 2. follow directory symlinks while walking
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	302 3. follow file symlinks while walking (not yet implemented)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	303
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	304 """
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	305
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	306 def __init__(self, args, *kwargs):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	307 if "nargs" in kwargs:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	308 raise ValueError("`nargs' not allowed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	309 if "type" in kwargs:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	310 raise ValueError("`type' not allowed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	311 c = kwargs.get("const", None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	312 if c is None:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	313 raise ValueError("a const value is needed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	314 if (not isinstance(c, FollowSymlinkConfig)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	315 and c not in ("follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	316 "no-follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	317 "follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	318 "no-follow-file-symlinks")):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	319 raise ValueError(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	320 "invalid value for the `const' configuration value")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	321 default = kwargs.get("default", None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	322 if (default is not None
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	323 and not isinstance(default, FollowSymlinkConfig)):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	324 raise TypeError("invalid type for `default'")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	325 kwargs["nargs"] = 0
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	326 super(SymlinkAction, self).__init__(args, *kwargs)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	327
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	328 def __call__(self, parser, namespace, values, option_string=None):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	329 curval = getattr(namespace, self.dest, None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	330 if curval is None:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	331 curval = FollowSymlinkConfig(False, False, True)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	332 if isinstance(self.const, FollowSymlinkConfig):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	333 curval = self.const
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	334 else:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	335 if self.const == "follow-directory-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	336 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	337 curval.command_line, True, curval.file)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	338 elif self.const == "no-follow-directory-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	339 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	340 curval.command_line, False, curval.file)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	341 elif self.const == "follow-file-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	342 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	343 curval.command_line, curval.directory, True)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	344 elif self.const == "no-follow-file-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	345 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	346 curval.command_line, curval.directory, False)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	347 else:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	348 assert False, "Implementation error: not yet implemented"
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	349
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	350 # Not following symlinks to files is not yet supported: reset to True
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	351 # if not curval.file:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	352 # curval = FollowSymlinkConfig(
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	353 # curval.command_line, curval.directory, True)
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	354 # logging.warning("Coercing options to `follow-file-symlinks'")
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	355 setattr(namespace, self.dest, curval)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	356
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	357
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	358 def gen_generate_opts(directories=[],
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	359 algorithm=util.default_algotag(),
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	360 append_output=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	361 base64=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	362 comment=[],
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	363 follow_symlinks=FollowSymlinkConfig(False, False, False),
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	364 full_mode=False,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	365 generator="normal",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	366 logical=None,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	367 minimal=None,
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	368 mode=False,
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	369 mmap=None,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	370 mtime=False,
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	371 output=None,
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	372 print_size=False,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	373 size_only=False,
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	374 utf8=False):
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	375 if not isinstance(follow_symlinks, FollowSymlinkConfig):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	376 raise TypeError("`follow_symlinks' must be a FollowSymlinkConfig")
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	377 # Not following symlinks to files is not yet supported: reset to True
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	378 # if not follow_symlinks.file:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	379 # follow_symlinks = follow_symlinks._make([follow_symlinks.command_line,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	380 # follow_symlinks.directory,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	381 # True])
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	382 # logging.warning("Coercing to follow-symlinks-file")
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	383 opts = argparse.Namespace(
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	384 directories=directories,
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	385 algorithm=util.argv2algo(algorithm),
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	386 append_output=append_output,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	387 base64=base64,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	388 comment=comment,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	389 follow_symlinks=follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	390 generator=generator,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	391 logical=logical,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	392 minimal=minimal,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	393 mmap=mmap,
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	394 metadata_full_mode=full_mode,
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	395 metadata_mode=mode,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	396 metadata_mtime=mtime,
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	397 output=output,
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	398 print_size=print_size,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	399 size_only=size_only,
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	400 utf8=utf8)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	401 return opts
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	402
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	403
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	404 def gen_info_opts(digest_files=[], last=False):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	405 opts = argparse.Namespace(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	406 digest_files=digest_files,
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	407 print_only_last_block=last)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	408 return opts
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	409
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	410
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	411 def treesum(opts):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	412 # XXX TBD: opts.check and opts.checklist (as in shasum.py)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	413 if opts.subcommand in ("generate", "gen"):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	414 return generate_treesum(opts)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	415 elif opts.subcommand == "info":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	416 return print_treesum_digestfile_infos(opts)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	417 else:
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	418 raise RuntimeError(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	419 "command `{}' not yet handled".format(opts.subcommand))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	420
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	421
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	422 def generate_treesum(opts):
149 f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	423 # Provide defaults
f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	424 if not opts.algorithm:
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	425 opts.algorithm = util.argv2algo(util.default_algotag())
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	426 if not opts.directories:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	427 opts.directories.append(".")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	428
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	429 if opts.output is None or opts.output == "-":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	430 if hasattr(sys.stdout, "buffer"):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	431 out_cm = cm.nullcontext(sys.stdout.buffer)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	432 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	433 out_cm = cm.nullcontext(sys.stdout)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	434 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	435 if opts.append_output:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	436 out_cm = open(opts.output, "ab")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	437 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	438 out_cm = open(opts.output, "wb")
179 53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	439 out_cm = CRC32Output(out_cm)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	440
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	441 with out_cm as outfp:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	442 writer = TreesumWriter(outfp)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	443 for d in opts.directories:
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	444 V1DirectoryTreesumGenerator(
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	445 opts.algorithm, opts.mmap, opts.base64,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	446 opts.follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	447 opts.generator,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	448 opts.metadata_mode,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	449 opts.metadata_full_mode,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	450 opts.metadata_mtime,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	451 opts.size_only,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	452 opts.print_size,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	453 opts.utf8,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	454 minimal=opts.minimal).generate(
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	455 writer, d, comment=opts.comment)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	456
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	457
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	458 class V1DirectoryTreesumGenerator(object):
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	459
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	460 def __init__(self, algorithm, use_mmap, use_base64,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	461 follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	462 with_generator,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	463 with_metadata_mode, with_metadata_full_mode,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	464 with_metadata_mtime, size_only, print_size, utf8_mode,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	465 minimal=None,):
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	466 super(V1DirectoryTreesumGenerator, self).__init__()
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	467 self._algorithm = algorithm
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	468 self._use_mmap = use_mmap
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	469 self._use_base64 = use_base64
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	470 self._follow_symlinks = follow_symlinks
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	471 self._with_generator = with_generator
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	472 self._with_metadata_mode = with_metadata_mode
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	473 self._with_metadata_full_mode = with_metadata_full_mode
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	474 self._with_metadata_mtime = with_metadata_mtime
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	475 self._size_only = size_only
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	476 self._print_size = print_size
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	477 self._utf8_mode = utf8_mode
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	478 self._minimal = minimal
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	479
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	480 def generate(self, writer, root, comment=None):
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	481 """
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	482
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	483 :param outfp: a binary file with a "write()" and a "flush()" method
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	484
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	485 """
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	486 self._writer = writer
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	487 self._writer.start("1")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	488 self._writer.write_fsencoding(util.n(walk.getfsencoding().upper()))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	489 self._writer.flush()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	490
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	491 if self._with_generator == "none":
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	492 pass # do nothing
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	493 elif self._with_generator == "normal":
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	494 self._writer.write_generator("PY2" if util.PY2 else "PY3")
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	495 elif self._with_generator == "full":
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	496 import platform
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	497 info = "%s %s, %s" % (platform.python_implementation(),
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	498 platform.python_version(),
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	499 platform.platform())
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	500 self._writer.write_generator(info)
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	501 else:
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	502 raise NotImplementedError(
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	503 "not implemented: %s" % (self._with_generator,))
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	504
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	505 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	506 # Note: Given non-default flags that are relevant for
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	507 # directory traversal.
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	508 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	509 flags = []
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	510 if self._with_metadata_full_mode:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	511 flags.append("with-metadata-fullmode")
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	512 elif self._with_metadata_mode:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	513 flags.append("with-metadata-mode")
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	514 if self._with_metadata_mtime:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	515 flags.append("with-metadata-mtime")
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	516 flags.append("follow-symlinks-commandline"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	517 if self._follow_symlinks.command_line
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	518 else "no-follow-symlinks-commandline")
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	519 flags.append("follow-symlinks-directory"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	520 if self._follow_symlinks.directory
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	521 else "no-follow-symlinks-directory")
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	522 flags.append("follow-symlinks-file"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	523 if self._follow_symlinks.file
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	524 else "no-follow-symlinks-file")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	525 if self._size_only:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	526 flags.append("size-only")
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	527 flags.append("utf8-encoding" if self._utf8_mode else "fs-encoding")
211 5bb0b25f8e99 FIX: FLAGS output: wrong "else" for "utf-8-mode" and "print-size" Franz Glasner <fzglas.hg@dom66.de> parents: 210 diff changeset	528 if self._print_size:
5bb0b25f8e99 FIX: FLAGS output: wrong "else" for "utf-8-mode" and "print-size" Franz Glasner <fzglas.hg@dom66.de> parents: 210 diff changeset	529 flags.append("print-size")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	530 self._writer.write_flags(flags)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	531
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	532 if self._minimal is None:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	533 # Write execution timestamps in POSIX epoch and ISO format
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	534 ts = int(time.time())
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	535 self._writer.write_timestamp(ts)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	536 ts = (datetime.datetime.utcfromtimestamp(ts)).isoformat("T")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	537 self._writer.write_isotimestamp(ts)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	538
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	539 if comment:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	540 for line in comment:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	541 self._writer.write_comment(line)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	542
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	543 if self._minimal is not None:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	544 self._writer.write_root(
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	545 (walk.WalkDirEntry.alt_u8(self._minimal)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	546 if self._minimal else b""))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	547 else:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	548 self._writer.write_root(walk.WalkDirEntry.alt_u8(root))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	549 self._writer.flush()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	550
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	551 if not self._follow_symlinks.command_line and os.path.islink(root):
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	552 linktgt = walk.WalkDirEntry.from_readlink(os.readlink(root))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	553 linkdgst = self._algorithm[0]()
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	554 linkdgst.update(linktgt.fspath)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	555 dir_dgst = self._algorithm[0]()
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	556 dir_dgst.update(b"2:L@,")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	557 dir_dgst.update(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	558 util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	559 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	560 len(self._algorithm[1]), util.b(self._algorithm[1]),
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	561 len(linkdgst.digest()), linkdgst.digest()))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	562 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	563 self._writer.write_size(b"./@/", 0)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	564 else:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	565 self._writer.write_file_digest(
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	566 self._algorithm[1],
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	567 b"./@/",
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	568 dir_dgst.digest(),
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	569 self._use_base64)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	570 self._writer.flush()
268 8aadffaaad5f treesum: refactor: remove early return and use an "else" branch instead Franz Glasner <fzglas.hg@dom66.de> parents: 267 diff changeset	571 else:
8aadffaaad5f treesum: refactor: remove early return and use an "else" branch instead Franz Glasner <fzglas.hg@dom66.de> parents: 267 diff changeset	572 self._generate(os.path.normpath(root), tuple())
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	573 self._writer.finish()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	574
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	575 def _generate(self, root, top):
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	576 logging.debug("Handling %s/%r", root, top)
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	577 path = os.path.join(root, *top) if top else root
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	578 try:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	579 with walk.ScanDir(path) as dirscan:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	580 fsobjects = list(dirscan)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	581 except OSError as e:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	582 if self._utf8_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	583 opath = walk.WalkDirEntry.alt_u8(path)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	584 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	585 opath = walk.WalkDirEntry.alt_fs(path)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	586 if e.errno == errno.ENOTDIR:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	587 # object exists but is not a directory
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	588 errmsg = b"not a directory"
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	589 elif e.errno in (errno.EACCES, errno.EPERM,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	590 getattr(errno, "ENOTCAPABLE", errno.EACCES)):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	591 # no permissions
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	592 errmsg = (
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	593 b"access denied / no permissions / missing capabilities")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	594 elif e.errno == errno.ENOENT:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	595 # given object does not exist
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	596 errmsg = b"no such file or directory"
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	597 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	598 raise
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	599 self._writer.write_error(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	600 b"`%s': %s", opath, errmsg))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	601 opath = join_output_path(top, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	602 if opath:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	603 if self._utf8_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	604 opath = walk.WalkDirEntry.alt_u8(opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	605 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	606 opath = walk.WalkDirEntry.alt_fs(opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	607 if self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	608 self._writer.write_size(opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	609 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	610 self._writer.write_file_digest(self._algorithm[1], opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	611 self._writer.flush()
269 fc002983253c treesum: also return the algorithm used by directory digests. Franz Glasner <fzglas.hg@dom66.de> parents: 268 diff changeset	612 return (None, None, None)
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	613 if self._utf8_mode:
221 ca9d5a0dc9bb Rename WalkDirEntry.sort_key and WalkDirEntry.alt_sort_key Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	614 fsobjects.sort(key=walk.WalkDirEntry.sort_key_u8)
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	615 else:
221 ca9d5a0dc9bb Rename WalkDirEntry.sort_key and WalkDirEntry.alt_sort_key Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	616 fsobjects.sort(key=walk.WalkDirEntry.sort_key_fs)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	617 dir_dgst = self._algorithm[0]()
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	618 dir_size = 0
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	619 dir_tainted = False
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	620 for fso in fsobjects:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	621 if fso.is_dir:
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	622 if fso.is_symlink and not self._follow_symlinks.directory:
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	623 linktgt = walk.WalkDirEntry.from_readlink(
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	624 os.readlink(fso.path))
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	625 # linktgt = util.fsencode(os.readlink(fso.path)))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	626 linkdgst = self._algorithm[0]()
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	627 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	628 if linktgt.u8path is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	629 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	630 linkdgst.update(linktgt.alt_u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	631 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	632 linkdgst.update(linktgt.u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	633 if fso.u8name is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	634 dir_tainted = True
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	635 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	636 b"2:@/,%d:%s,",
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	637 len(fso.alt_u8name),
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	638 fso.alt_u8name))
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	639 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	640 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	641 b"2:@/,%d:%s,", len(fso.u8name), fso.u8name))
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	642 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	643 if linktgt.fspath is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	644 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	645 linkdgst.update(linktgt.alt_fspath)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	646 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	647 linkdgst.update(linktgt.fspath)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	648 if fso.fsname is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	649 dir_tainted = True
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	650 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	651 b"2:@/,%d:%s,",
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	652 len(fso.alt_fsname),
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	653 fso.alt_fsname))
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	654 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	655 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	656 b"2:@/,%d:%s,", len(fso.fsname), fso.fsname))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	657 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	658 # - no mtime and no mode for symlinks
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	659 # - also does not count for dir_size
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	660 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	661 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	662 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	663 len(self._algorithm[1]), util.b(self._algorithm[1]),
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	664 len(linkdgst.digest()), linkdgst.digest()))
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	665 opath = join_output_path(top, fso.name)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	666 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	667 opath = walk.WalkDirEntry.alt_u8(opath)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	668 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	669 opath = walk.WalkDirEntry.alt_fs(opath)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	670 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	671 self._writer.write_size(
218 dee891ed2307 FIX: Output of symlinks was not converted property but was written in bytes repr Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	672 util.interpolate_bytes(b"%s/./@/", opath),
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	673 0)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	674 else:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	675 self._writer.write_file_digest(
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	676 self._algorithm[1],
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	677 util.interpolate_bytes(b"%s/./@/", opath),
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	678 linkdgst.digest(),
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	679 self._use_base64)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	680 else:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	681 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	682 # Follow the symlink to dir or handle a "real" directory
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	683 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	684
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	685 # Get subdir data from recursing into it
269 fc002983253c treesum: also return the algorithm used by directory digests. Franz Glasner <fzglas.hg@dom66.de> parents: 268 diff changeset	686 sub_dir_algo, sub_dir_dgst, sub_dir_size = self._generate(
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	687 root, top + (fso.name, ))
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	688
269 fc002983253c treesum: also return the algorithm used by directory digests. Franz Glasner <fzglas.hg@dom66.de> parents: 268 diff changeset	689 if (sub_dir_algo is None or sub_dir_dgst is None
fc002983253c treesum: also return the algorithm used by directory digests. Franz Glasner <fzglas.hg@dom66.de> parents: 268 diff changeset	690 or sub_dir_size is None):
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	691 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	692 # This should not happen:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	693 # - top-level directories are handled above
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	694 # - other filesystem objects should also have been
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	695 # handled already
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	696 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	697 assert False
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	698
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	699 dir_size += sub_dir_size
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	700 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	701 if fso.u8name is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	702 dir_tainted = True
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	703 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	704 b"1:/,%d:%s,",
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	705 len(fso.alt_u8name),
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	706 fso.alt_u8name))
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	707 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	708 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	709 b"1:/,%d:%s,", len(fso.u8name), fso.u8name))
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	710 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	711 if fso.fsname is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	712 dir_tainted = True
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	713 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	714 b"1:/,%d:%s,",
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	715 len(fso.alt_fsname),
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	716 fso.alt_fsname))
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	717 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	718 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	719 b"1:/,%d:%s,", len(fso.fsname), fso.fsname))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	720 if self._with_metadata_full_mode:
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	721 modestr = util.b(normalized_mode_str(fso.stat.st_mode))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	722 dir_dgst.update(util.interpolate_bytes(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	723 b"8:fullmode,%d:%s,", len(modestr), modestr))
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	724 elif self._with_metadata_mode:
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	725 modestr = util.b(normalized_compatible_mode_str(
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	726 fso.stat.st_mode))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	727 dir_dgst.update(util.interpolate_bytes(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	728 b"4:mode,%d:%s,", len(modestr), modestr))
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	729 dir_dgst.update(util.interpolate_bytes(
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	730 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	731 len(sub_dir_algo), util.b(sub_dir_algo),
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	732 len(sub_dir_dgst), sub_dir_dgst))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	733 else:
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	734 if fso.is_symlink and not self._follow_symlinks.file:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	735 #
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	736 # Symbolic link to some filesystem object which is not
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	737 # determined to be a link to a directory.
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	738 #
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	739 linktgt = walk.WalkDirEntry.from_readlink(
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	740 os.readlink(fso.path))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	741 # linktgt = util.fsencode(os.readlink(fso.path)))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	742 linkdgst = self._algorithm[0]()
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	743 if self._utf8_mode:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	744 if linktgt.u8path is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	745 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	746 linkdgst.update(linktgt.alt_u8path)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	747 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	748 linkdgst.update(linktgt.u8path)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	749 if fso.u8name is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	750 dir_tainted = True
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	751 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	752 b"1:@,%d:%s,",
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	753 len(fso.alt_u8name),
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	754 fso.alt_u8name))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	755 else:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	756 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	757 b"1:@,%d:%s,", len(fso.u8name), fso.u8name))
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	758 else:
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	759 if linktgt.fspath is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	760 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	761 linkdgst.update(linktgt.alt_fspath)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	762 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	763 linkdgst.update(linktgt.fspath)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	764 if fso.fsname is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	765 dir_tainted = True
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	766 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	767 b"1:@,%d:%s,",
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	768 len(fso.alt_fsname),
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	769 fso.alt_fsname))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	770 else:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	771 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	772 b"1:@,%d:%s,", len(fso.fsname), fso.fsname))
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	773 #
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	774 # - no mtime and no mode for symlinks
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	775 # - also does not count for dir_size
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	776 #
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	777 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	778 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	779 len(self._algorithm[1]), util.b(self._algorithm[1]),
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	780 len(linkdgst.digest()), linkdgst.digest()))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	781 opath = join_output_path(top, fso.name)
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	782 if self._utf8_mode:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	783 opath = walk.WalkDirEntry.alt_u8(opath)
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	784 else:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	785 opath = walk.WalkDirEntry.alt_fs(opath)
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	786 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	787 self._writer.write_size(
218 dee891ed2307 FIX: Output of symlinks was not converted property but was written in bytes repr Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	788 util.interpolate_bytes(b"%s/./@", opath),
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	789 0)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	790 else:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	791 self._writer.write_file_digest(
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	792 self._algorithm[1],
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	793 util.interpolate_bytes(b"%s/./@", opath),
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	794 linkdgst.digest(),
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	795 self._use_base64)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	796 else:
223 61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	797 #
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	798 # Follow the symlink to file or handle a "real" file
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	799 #
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	800
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	801 if self._utf8_mode:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	802 if fso.u8name is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	803 dir_tainted = True
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	804 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	805 b"0:,%d:%s,",
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	806 len(fso.alt_u8name),
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	807 fso.alt_u8name))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	808 else:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	809 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	810 b"0:,%d:%s,", len(fso.u8name), fso.u8name))
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	811 else:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	812 if fso.fsname is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	813 dir_tainted = True
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	814 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	815 b"0:,%d:%s,",
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	816 len(fso.alt_fsname),
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	817 fso.alt_fsname))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	818 else:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	819 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	820 b"0:,%d:%s,", len(fso.fsname), fso.fsname))
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	821 opath = join_output_path(top, fso.name)
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	822 if self._utf8_mode:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	823 opath = walk.WalkDirEntry.alt_u8(opath)
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	824 else:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	825 opath = walk.WalkDirEntry.alt_fs(opath)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	826 if fso.stat is None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	827 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	828 # Error: most likely a broken symlink here
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	829 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	830 dir_tainted = True
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	831 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	832 b"5:errno,%d:%s,",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	833 len(str(fso.stat_errno)),
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	834 util.b(str(fso.stat_errno))))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	835 self._writer.write_error(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	836 b"errno %d: %s",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	837 fso.stat_errno,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	838 util.b(fso.stat_errstr, "utf-8")))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	839 logging.error(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	840 "Directory entry has symlink problems: %r",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	841 opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	842 if self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	843 self._writer.write_size(opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	844 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	845 self._writer.write_file_digest(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	846 self._algorithm[1], opath, None)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	847 else:
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	848 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	849 # Ok: File has normal stat info
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	850 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	851 # XXX FIXME: Handle special files (fifo, socket,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	852 # block or char devices, ...).
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	853 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	854 dir_size += fso.stat.st_size
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	855 if self._with_metadata_mtime:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	856 mtime = datetime.datetime.utcfromtimestamp(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	857 int(fso.stat.st_mtime))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	858 mtime = util.b(mtime.isoformat("T") + "Z")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	859 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	860 b"5:mtime,%d:%s,", len(mtime), mtime))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	861 if self._with_metadata_full_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	862 modestr = util.b(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	863 normalized_mode_str(fso.stat.st_mode))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	864 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	865 b"8:fullmode,%d:%s,", len(modestr), modestr))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	866 elif self._with_metadata_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	867 modestr = util.b(normalized_compatible_mode_str(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	868 fso.stat.st_mode))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	869 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	870 b"4:mode,%d:%s,", len(modestr), modestr))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	871 if not self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	872 dgst = digest.compute_digest_file(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	873 self._algorithm[0],
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	874 fso.path,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	875 use_mmap=self._use_mmap)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	876 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	877 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	878 len(self._algorithm[1]),
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	879 util.b(self._algorithm[1]),
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	880 len(dgst),
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	881 dgst))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	882 if self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	883 self._writer.write_size(opath, fso.stat.st_size)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	884 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	885 sz = fso.stat.st_size if self._print_size else None
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	886 self._writer.write_file_digest(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	887 self._algorithm[1], opath, dgst,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	888 use_base64=self._use_base64,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	889 size=sz)
267 b9aa65a30b4c treesum: optimize the use of flush() somewhat Franz Glasner <fzglas.hg@dom66.de> parents: 266 diff changeset	890 self._writer.flush()
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	891 opath = join_output_path(top, None)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	892 if opath:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	893 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	894 opath = walk.WalkDirEntry.alt_u8(opath)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	895 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	896 opath = walk.WalkDirEntry.alt_fs(opath)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	897 if dir_tainted:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	898 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	899 # IMPORTANT: Print errors BEFORE the associated digest or size
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	900 # line. Otherwise the "info" command has a problem.
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	901 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	902 self._writer.write_error(b"directory is tainted")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	903 logging.error("Directory has filename and/or symlink problems: %r",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	904 opath)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	905 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	906 self._writer.write_size(opath, dir_size)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	907 else:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	908 sz = dir_size if self._print_size else None
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	909 self._writer.write_file_digest(
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	910 self._algorithm[1], opath, dir_dgst.digest(),
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	911 use_base64=self._use_base64, size=sz)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	912 self._writer.flush()
269 fc002983253c treesum: also return the algorithm used by directory digests. Franz Glasner <fzglas.hg@dom66.de> parents: 268 diff changeset	913 return (self._algorithm[1], dir_dgst.digest(), dir_size)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	914
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	915
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	916 def join_output_path(top, name):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	917 if name is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	918 # a path for a directory is to be computed
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	919 if top:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	920 if isinstance(top[0], bytes):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	921 return b"/".join(top) + b"/"
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	922 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	923 return u"/".join(top) + u"/"
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	924 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	925 return b""
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	926 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	927 # a path for a normal file is to be computed
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	928 if top:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	929 if isinstance(name, bytes):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	930 return b"/".join(top) + b"/" + name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	931 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	932 return u"/".join(top) + u"/" + name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	933 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	934 return name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	935
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	936
179 53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	937 class CRC32Output(object):
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	938
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	939 """Wrapper for a minimal binary file contextmanager that calculates
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	940 the CRC32 of the written bytes on the fly.
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	941
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	942 Also acts as context manager proxy for the given context manager.
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	943
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	944 """
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	945
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	946 __slots__ = ("_fp_cm", "_fp", "_crc32")
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	947
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	948 def __init__(self, fp_cm):
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	949 self._fp_cm = fp_cm
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	950 self._fp = None
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	951 self.resetdigest()
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	952
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	953 def __enter__(self):
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	954 assert self._fp is None
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	955 self._fp = self._fp_cm.__enter__()
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	956 return self
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	957
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	958 def __exit__(self, *args):
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	959 rv = self._fp_cm.__exit__(*args)
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	960 self._fp = None
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	961 return rv
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	962
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	963 def write(self, what):
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	964 self._fp.write(what)
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	965 self._crc32.update(what)
179 53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	966
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	967 def flush(self):
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	968 self._fp.flush()
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	969
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	970 def resetdigest(self):
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	971 """Reset the current CRC digest"""
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	972 self._crc32 = crc32()
179 53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	973
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	974 def hexcrcdigest(self):
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	975 """
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	976
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	977 :rtype: str
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	978
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	979 """
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	980 return self._crc32.hexdigest()
179 53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	981
53614a724bf0 Also write a (standard) CRC-32 checksum for each block of output Franz Glasner <fzglas.hg@dom66.de> parents: 178 diff changeset	982
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	983 def normalized_compatible_mode_str(mode):
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	984 # XXX FIXME: Windows and "executable"
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	985 modebits = stat.S_IMODE(mode)
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	986 modestr = "%o" % (modebits,)
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	987 if not modestr.startswith("0"):
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	988 modestr = "0" + modestr
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	989 return modestr
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	990
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	991
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	992 def normalized_mode_str(mode):
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	993 modestr = "%o" % (mode,)
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	994 if not modestr.startswith("0"):
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	995 modestr = "0" + modestr
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	996 return modestr
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	997
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	998
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	999 class TreesumWriter(object):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1000
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1001 """Writer to write treesum digest files in a format similar to BSD
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1002 digest files.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1003
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1004 Wraps an output file pointer for a binary file.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1005
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1006 Provides high-level methods to write data lines.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1007
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1008 Also holds the current CRC for a block.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1009
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1010 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1011
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1012 LS = util.b(os.linesep)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1013
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1014 def __init__(self, outfp):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1015 self._outfp = outfp
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1016 self._reset_crc()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1017
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1018 def _reset_crc(self):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1019 self._crc = crc32()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1020
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1021 def start(self, version):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1022 """Begin a new block, reset the current CRC and write the VERSION
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1023 tag.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1024
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1025 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1026 self._reset_crc()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1027 self.write(b"VERSION = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1028 self.writeln(util.b(version))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1029
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1030 def write_comment(self, comment):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1031 self.write(b"COMMENT (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1032 self.write(util.b(comment, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1033 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1034
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1035 def write_generator(self, generator):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1036 self.write(b"GENERATOR (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1037 self.write(util.b(generator, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1038 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1039
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1040 def write_error(self, error):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1041 self.write(b"ERROR (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1042 self.write(util.b(error, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1043 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1044
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1045 def write_fsencoding(self, encoding):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1046 self.write(b"FSENCODING = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1047 self.writeln(util.b(encoding))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1048
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1049 def write_flags(self, flags):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1050 self.write(b"FLAGS = ")
262 c3d6599c1b5e FIX: treesum: bytearray needs not to be handled Franz Glasner <fzglas.hg@dom66.de> parents: 261 diff changeset	1051 if isinstance(flags, (str, bytes)):
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1052 self.writeln(util.b(flags))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1053 else:
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1054 flags.sort()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1055 self.writeln(util.b(",".join(flags)))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1056
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1057 def write_timestamp(self, ts):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1058 self.write(b"TIMESTAMP = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1059 self.writeln(util.b(str(ts)))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1060
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1061 def write_isotimestamp(self, ts):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1062 self.write(b"ISOTIMESTAMP = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1063 self.writeln(util.b(ts))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1064
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1065 def write_root(self, root):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1066 assert isinstance(root, bytes)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1067 self.write(b"ROOT (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1068 self.write(root)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1069 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1070
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1071 def write_size(self, filename, sz):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1072 assert isinstance(filename, bytes)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1073 self.write(b"SIZE (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1074 self.write(filename)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1075 self.write(b")")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1076 if sz is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1077 self.write(b" = ")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1078 self.write(util.b(str(sz)))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1079 self.writeln(b"")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1080
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1081 def write_file_digest(self, algorithm, filename, digest,
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1082 use_base64=False, size=None):
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1083 if digest is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1084 digest = (base64.b64encode(digest)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1085 if use_base64
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1086 else binascii.hexlify(digest))
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1087 if filename != b"./@/":
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1088 filename = util.normalize_filename(filename, True)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1089 self.write(util.b(algorithm))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1090 self.write(b" (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1091 self.write(filename)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1092 self.write(b")")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1093 if digest is not None or size is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1094 self.write(b" = ")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1095 if digest is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1096 self.write(digest)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1097 if size is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1098 self.write(b",")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1099 self.write(util.b(str(size)))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1100 self.writeln(b"")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1101
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1102 def finish(self):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1103 """Finish a block and write the current CRC"""
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1104 crc = self._crc.hexdigest()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1105 self.write(b"CRC32 = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1106 self.writeln(util.b(crc))
267 b9aa65a30b4c treesum: optimize the use of flush() somewhat Franz Glasner <fzglas.hg@dom66.de> parents: 266 diff changeset	1107 self.flush()
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1108
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1109 def writeln(self, line):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1110 """Write the bytes `line` into the output file and update the CRC
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1111 accordingly.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1112
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1113 :param bytes line: The line to write to (without line ending)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1114
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1115 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1116 self.write(line)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1117 self.write(self.LS)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1118
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1119 def write(self, data):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1120 """Write `data` into the output file and update the CRC accordingly.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1121
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1122 :param bytes data: The data to write to and to update the CRC with
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1123
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1124 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1125 if data:
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1126 self._outfp.write(data)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1127 self._crc.update(data)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1128
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1129 def flush(self):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1130 self._outfp.flush()
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1131
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1132
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1133 class TreesumReader(object):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1134
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1135 """Reader to read and/or verify treesum digest files.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1136
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1137 Supports the iterator and context manager protocol.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1138
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1139 """
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1140
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1141 PATTERN0 = re.compile(br"\A[ \t]*\r?\n\Z") # empty lines
198 c1e875ba4bdc Put the effective filesystem encoding into the treesum digest file using FSENCODING = <encoding> Franz Glasner <fzglas.hg@dom66.de> parents: 197 diff changeset	1142 PATTERN1 = re.compile(br"\A(VERSION\|FSENCODING\|FLAGS\|TIMESTAMP\|ISOTIMESTAMP\|CRC32)[ \t]=[ \t]([^ \t]+)[ \t]*\r?\n\Z") # noqa: E501 line too long
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	1143 PATTERN2 = re.compile(br"\A(ROOT\|COMMENT\|ERROR\|GENERATOR)[ \t]\((.)\)[ \t]*\r?\n\Z") # noqa: E501 line too long
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1144 PATTERN3 = re.compile(br"\ASIZE[ \t]\((.)\)([ \t]=[ \t](\d+))?[ \t]*\r?\n\Z") # noqa: E501 line too long
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1145 PATTERN4 = re.compile(br"\A([A-Za-z0-9_-]+)[ \t]\((.)\)([ \t]=[ \t]([A-Za-z0-9=+/]+)(,(\d+))?)?[ \t]*\r?\n\Z") # noqa: E501 line too long
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1146
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1147 def __init__(self, _fp, _filename, _own_fp):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1148 self._fp = _fp
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1149 self._own_fp = _own_fp
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1150 self._filename = _filename
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1151 self._line_no = 0
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1152 self._reset_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1153 self._expect_crc = None # NOTE: tristate: None is different from False
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1154 self._current_algo_name = self._current_algo_digest_size = None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1155
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1156 @classmethod
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1157 def from_path(cls_, path):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1158 """Open file at `path` and return a reader that owns the file object"""
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1159 return cls_(open(path, "rb"), path, True)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1160
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1161 @classmethod
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1162 def from_binary_buffer(cls_, binary_fp, filename):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1163 return cls_(binary_fp, filename, False)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1164
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1165 def __enter__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1166 return self
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1167
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1168 def __exit__(self, *args):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1169 self.close()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1170
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1171 def close(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1172 if self._fp is not None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1173 try:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1174 if self._own_fp:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1175 self._fp.close()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1176 finally:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1177 self._fp = None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1178
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1179 def __iter__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1180 return self
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1181
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1182 def __next__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1183 rec = self.read_record()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1184 if rec is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1185 raise StopIteration()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1186 return rec
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1187
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1188 if util.PY2:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1189 next = __next__
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1190
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1191 def all_records(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1192 """Iterator over all remaining records"""
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1193 while True:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1194 rec = self.read_record()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1195 if rec is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1196 return
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1197 yield rec
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1198
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1199 def read_record(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1200 """Read and parse the "next" line.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1201
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1202 :returns: `None` at EOF or the parsed contents of the line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1203 :rtype: tuple or None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1204
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1205 """
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1206 # Loop to skip empty lines
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1207 while True:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1208 line = self._get_next_line()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1209 if not line:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1210 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1211 # Skip for empty files at the very beginning.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1212 # Check only after the first VERSION line.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1213 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1214 if self._expect_crc is not None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1215 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1216 logging.warning("CRC32 is missing at EOF")
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1217 return None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1218 if not self.PATTERN0.search(line):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1219 break
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1220 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1221 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1222 # At the beginning transparently skip an eventually embedded signify
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1223 # signature
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1224 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1225 if self._line_no == 1:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1226 if line.startswith(b"untrusted comment: "):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1227 line = self._get_next_line()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1228 if not line.endswith(b"\n"):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1229 raise binascii.Error("No valid signify signature value")
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1230 # Try to decode for an early error check
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1231 base64.b64decode(line[:-1])
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1232 mo = self.PATTERN1.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1233 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1234 if mo.group(1) == b"VERSION":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1235 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1236 logging.warning("CRC32 missing before line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1237 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1238 self._reset_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1239 self._expect_crc = True
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1240 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1241 return ("VERSION", util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1242 if mo.group(1) == b"CRC32":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1243 # TODO: check
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1244 if self._expect_crc is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1245 logging.warning("Lone CRC32 before VERSION in line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1246 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1247 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1248 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1249 if (self._hex_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1250 != mo.group(2).decode("latin1").upper()):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1251 logging.warning(
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1252 "CRC32 mismatch in line %d:"
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1253 " expected: %s, given: %s",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1254 self._line_no,
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1255 self._hex_crc(),
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1256 mo.group(2).decode("latin1").upper())
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1257 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1258 logging.warning("CRC32 before VERSION in line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1259 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1260 # Do not update the CRC here but reset the state
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1261 self._expect_crc = False
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1262 return ("CRC32", util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1263 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1264 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1265 return (util.n(mo.group(1)), util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1266 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1267 mo = self.PATTERN2.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1268 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1269 self._update_crc(line)
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	1270 if mo.group(1) in (b"COMMENT", b"ERROR", b"GENERATOR"):
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1271 return (util.u(mo.group(1)), util.u(mo.group(2), "utf-8"))
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1272 elif mo.group(1) == b"ROOT":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1273 return ("ROOT", mo.group(2))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1274 assert False, line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1275 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1276 mo = self.PATTERN3.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1277 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1278 self._update_crc(line)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1279 if mo.group(2):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1280 return ("SIZE", mo.group(1),
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1281 int(util.n(mo.group(3)), 10))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1282 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1283 return ("SIZE", mo.group(1), None)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1284 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1285 mo = self.PATTERN4.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1286 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1287 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1288 algo_name = util.n(mo.group(1))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1289 if mo.group(3):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1290 if (len(mo.group(4)) ==
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1291 2 * self._get_digest_size(algo_name)):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1292 # hex
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1293 digest = binascii.unhexlify(mo.group(4))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1294 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1295 # base64
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1296 digest = base64.b64decode(mo.group(4))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1297 if mo.group(5):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1298 size = int(util.n(mo.group(6)), 10)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1299 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1300 size = None
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1301 return (algo_name, mo.group(2), digest, size)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1302 else:
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1303 return (algo_name, mo.group(2), None, None)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1304 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1305 assert False, line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1306 return line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1307
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1308 def _get_next_line(self):
189 959c6d37b014 Extend the max line length to read to 4096, which is something along PATH_MAX onn Linux Franz Glasner <fzglas.hg@dom66.de> parents: 188 diff changeset	1309 line = self._fp.readline(4096) # along PATH_MAX on Linux
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1310 if line:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1311 self._line_no += 1
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1312 return line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1313
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1314 def _reset_crc(self):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1315 self._crc32 = crc32()
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1316
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1317 def _update_crc(self, data):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1318 self._crc32.update(data)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1319
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1320 def _hex_crc(self):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1321 return self._crc32.hexdigest()
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1322
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1323 def _get_digest_size(self, algo_name):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1324 if self._current_algo_name == algo_name:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1325 return self._current_algo_digest_size
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1326 h = util.algotag2algotype(algo_name)()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1327 self._current_algo_name = algo_name
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1328 self._current_algo_digest_size = h.digest_size
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1329 return self._current_algo_digest_size
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1330
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1331
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1332 def print_treesum_digestfile_infos(opts):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1333 print_infos_for_digestfile(opts.digest_files, opts.print_only_last_block)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1334
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1335
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1336 def print_infos_for_digestfile(digest_files, print_only_last_block=True):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1337 for fn in digest_files:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1338 if fn == "-":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1339 if util.PY2:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1340 reader = TreesumReader.from_binary_buffer(sys.stdin)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1341 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1342 reader = TreesumReader.from_binary_buffer(sys.stdin.buffer)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1343 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1344 reader = TreesumReader.from_path(fn)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1345
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1346 with reader:
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1347 root = generator = flags = fsencoding = algorithm = digest \
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1348 = size = None
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1349 errors = set()
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1350 comments = []
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1351 in_block = False
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1352 block_no = 0
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1353 for record in reader:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1354 if record[0] == "VERSION":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1355 assert record[1] == "1"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1356 # start a new block
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1357 in_block = True
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1358 block_no += 1
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1359 root = flags = algorithm = digest = size = None
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1360 comments = []
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1361 elif record[0] == "GENERATOR":
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1362 generator = record[1]
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1363 elif record[0] == "FSENCODING":
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1364 fsencoding = record[1]
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1365 elif record[0] == "FLAGS":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1366 flags = record[1]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1367 elif record[0] == "ROOT":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1368 root = record[1]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1369 elif record[0] == "COMMENT":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1370 comments.append(record[1])
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1371 elif record[0] == "ERROR":
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1372 errors.add(record[1])
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1373 elif record[0] in ("TIMESTAMP", "ISOTIMESTAMP"):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1374 pass
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1375 elif record[0] == "CRC32":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1376 pass
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1377 # in_block = False
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1378 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1379 if not in_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1380 continue
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1381 # digest line or size line
193 fb36e71f6ba8 Change: path indicators for symlinks: ./@ -> ./@/ and /./@ -> /./@/ Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	1382 if not record[1] or record[1] == b"./@/":
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1383 if record[0] == "SIZE":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1384 algorithm = "SIZE"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1385 digest = None
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1386 size = record[2]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1387 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1388 algorithm = record[0]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1389 digest = record[2]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1390 size = record[3]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1391 if not print_only_last_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1392 print_block_data(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1393 block_no,
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1394 root, generator, fsencoding, flags, comments,
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1395 errors, algorithm, digest, size)
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1396 root = generator = flags = fsencoding = algorithm \
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1397 = digest = size = None
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1398 errors = set()
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1399 comments = []
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1400 in_block = False
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1401 if print_only_last_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1402 if not in_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1403 if digest is not None or size is not None:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1404 print_block_data(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1405 block_no,
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1406 root, generator, fsencoding, flags, comments, errors,
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1407 algorithm, digest, size)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1408 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1409 logging.warning("missing block end")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1410
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1411
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1412 def print_block_data(block_no, tag, generator, fsencoding, flags, comments,
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1413 errors, algorithm, digest, size):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1414 digeststr = util.n(binascii.hexlify(digest)) if digest else "<no digest>"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1415 sizestr = str(size) if size is not None else "<no size>"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1416 print("BLOCK No %d:" % (block_no,))
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1417 print(" Tag:", tag)
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1418 print(" FS-Encoding:", fsencoding)
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1419 if generator:
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1420 print(" Generator:", generator)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1421 print(" Flags:", flags if flags else "<none>")
220 8db78850d800 "treesum info": print comments only if not empty Franz Glasner <fzglas.hg@dom66.de> parents: 219 diff changeset	1422 if comments:
8db78850d800 "treesum info": print comments only if not empty Franz Glasner <fzglas.hg@dom66.de> parents: 219 diff changeset	1423 print(" Comments:", comments)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1424 print(" Algorithm:", algorithm)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1425 if algorithm != "SIZE":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1426 print(" Digest:", digeststr)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1427 print(" Size:", sizestr)
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1428 print(" Errors:", errors if errors else "<none>")
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1429
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1430
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1431 if __name__ == "__main__":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1432 sys.exit(main())

Mercurial > hgrepos > Python > apps > py-cutils

annotate cutils/treesum.py @ 270:42f4ca423ab3