Python/apps/py-cutils: cutils/treesum.py annotate

annotate cutils/treesum.py @ 322:0cabc5439505

treesum: Remove CRC32Output completely

author	Franz Glasner <fzglas.hg@dom66.de>
date	Wed, 26 Mar 2025 13:00:40 +0100
parents	49ecfff4f319
children	48430941c18c

rev	line source
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1 # -- coding: utf-8 --
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	3 # :Copyright: (c) 2020-2025 Franz Glasner
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	4 # :License: BSD-3-Clause
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	5 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	6 r"""Generate and verify checksums for directory trees.
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	7
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	8 """
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	9
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	10 from __future__ import print_function, absolute_import
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	11
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	12
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	13 __all__ = []
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	14
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	15
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	16 import argparse
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	17 import base64
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	18 import binascii
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	19 import collections
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	20 import datetime
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	21 import errno
176 7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	22 import logging
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	23 import os
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	24 import re
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	25 import stat
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	26 import sys
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	27 import time
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	28
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	29 from . import (__version__, __revision__)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	30 from . import util
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	31 from .util import cm
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	32 from .util import digest
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	33 from .util import fnmatch
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	34 from .util import walk
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	35 from .util.crc32 import crc32
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	36
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	37
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	38 def main(argv=None):
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	39
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	40 def _populate_generate_arguments(gp):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	41 """Use to populate command aliases.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	42
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	43 This is because :class:`argparse.ArgumentParser` does not
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	44 support them for all supported Python versions.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	45
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	46 """
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	47 gp.add_argument(
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	48 "--accept-treesum", "-A", action=PatternMatchAction,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	49 kind="accept-treesum",
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	50 dest="fnmatch_filters", metavar="PATTERN",
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	51 help="""Accept an existing treesum file PATTERN for a directory
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	52 checksum.
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	53 Implicitly this also acts as `--exclude' option.
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	54 Can be given more than once.""")
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	55 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	56 "--algorithm", "-a", action="store", type=util.argv2algo,
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	57 help="1 (aka sha1), 224, 256 (aka sha256), 384, 512 (aka sha512), "
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	58 "3 (alias for sha3-512), 3-224, 3-256, 3-384, 3-512, "
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	59 "blake2b, blake2b-256, blake2s, "
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	60 "blake2 (alias for blake2b), "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	61 "blake2-256 (alias for blake2b-256), "
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	62 "md5. "
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	63 "The default depends on availability in hashlib: "
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	64 "blake2b-256, sha256 or sha1.")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	65 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	66 "--append-output", action="store_true", dest="append_output",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	67 help="Append to the output file instead of overwriting it.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	68 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	69 "--base64", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	70 help="Output checksums in base64 notation, not hexadecimal "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	71 "(OpenBSD).")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	72 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	73 "--comment", action="append", default=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	74 help="Put given comment COMMENT into the output as \"COMMENT\". "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	75 "Can be given more than once.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	76 gp.add_argument(
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	77 "--exclude", "-X", action=PatternMatchAction, kind="exclude",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	78 dest="fnmatch_filters", metavar="PATTERN",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	79 help="""Exclude names matching the given PATTERN.
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	80 For help on PATTERN use \"help patterns\".
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	81 Can be given more than once.""")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	82 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	83 "--follow-directory-symlinks", "-l", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	84 const="follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	85 default=FollowSymlinkConfig(False, False, True),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	86 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	87 help="""Follow symbolic links to directories when walking a
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	88 directory tree. Augments --physical, --half and -p.""")
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	89 gp.add_argument(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	90 "--follow-file-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	91 const="follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	92 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	93 help="""Follow symbolic links to files when walking a
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	94 directory tree. Augments --physical and --half.""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	95 gp.add_argument(
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	96 "--full-mode", action="store_true", dest="metadata_full_mode",
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	97 help="Consider all mode bits as returned from stat(2) when "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	98 "computing directory digests. "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	99 "Note that mode bits on symbolic links itself are not "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	100 "considered.")
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	101 gp.add_argument(
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	102 "--generator", choices=("normal", "full", "none"),
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	103 default="normal",
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	104 help="""Put a `GENERATOR' line into the output.
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	105 `full' prints full Python and OS/platform version information,
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	106 `normal' prints just whether Python 2 or Python 3 is used, and `none'
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	107 suppresses the output completely. The default is `normal'.""")
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	108 gp.add_argument(
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	109 "--half", "-H", action=SymlinkAction, dest="follow_symlinks",
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	110 const=FollowSymlinkConfig(True, False, False),
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	111 help="""Follow symbolic links given on the command line but do
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	112 not follow symlinks while traversing the directory tree.
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	113 Overwrites any other symlink related options
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	114 (--physical, --logical, -p, --no-follow-directory-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	115 --no-follow-file-symlinks, et al.).""")
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	116 gp.add_argument(
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	117 "--include", "-I", action=PatternMatchAction, kind="include",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	118 dest="fnmatch_filters", metavar="PATTERN",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	119 help="""Include names matching the given PATTERN.
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	120 For help on PATTERN use \"help patterns\".
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	121 Can be given more than once.""")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	122 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	123 "--logical", "-L", action=SymlinkAction, dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	124 const=FollowSymlinkConfig(True, True, True),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	125 help="""Follow symbolic links everywhere: on command line
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	126 arguments and -- while walking -- directory and file symbolic links.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	127 Overwrites any other symlink related options
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	128 (--physical, --half, -p, --no-follow-directory-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	129 --no-follow-file-symlinks, et al.).""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	130 gp.add_argument(
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	131 "--minimal", nargs="?", const="", default=None, metavar="TAG",
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	132 help="Produce minimal output only. If a TAG is given and not "
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	133 "empty use it as the leading \"ROOT (<TAG>)\" output.")
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	134 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	135 "--mmap", action="store_true", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	136 help="Use mmap if available. Default is to determine "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	137 "automatically from the filesize.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	138 gp.add_argument(
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	139 "--mode", action="store_true", dest="metadata_mode",
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	140 help="Consider the permission bits of stat(2) using S_IMODE (i.e. "
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	141 "all bits without the filetype bits) when "
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	142 "computing directory digests. Note that mode bits on "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	143 "symbolic links itself are not considered.")
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	144 gp.add_argument(
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	145 "--mtime", action="store_true", dest="metadata_mtime",
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	146 help="Consider the mtime of files (non-directories) when "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	147 "generating digests for directories. Digests for files are "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	148 "not affected.")
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	149 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	150 "--no-follow-directory-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	151 const="no-follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	152 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	153 help="""Do not follow symbolic links to directories when walking a
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	154 directory tree. Augments --logical.""")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	155 gp.add_argument(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	156 "--no-follow-file-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	157 const="no-follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	158 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	159 help="""Dont follow symbolic links to files when walking a
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	160 directory tree. Augments --logical and -p.""")
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	161 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	162 "--no-mmap", action="store_false", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	163 help="Dont use mmap.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	164 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	165 "--output", "-o", action="store", metavar="OUTPUT",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	166 help="Put the checksum into given file. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	167 "If not given or if it is given as `-' then stdout is used.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	168 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	169 "--physical", "-P", action=SymlinkAction, dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	170 const=FollowSymlinkConfig(False, False, False),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	171 help="""Do not follow any symbolic links whether they are given
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	172 on the command line or when walking the directory tree.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	173 Overwrites any other symlink related options
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	174 (--logical, --half, -p, --follow-directory-symlinks, --follow-file-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	175 et al.).""")
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	176 gp.add_argument(
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	177 "-p", action=SymlinkAction, dest="follow_symlinks",
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	178 const=FollowSymlinkConfig(False, False, True),
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	179 help="""Do not follow any symbolic links to directories,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	180 whether they are given on the command line or when walking the directory tree,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	181 but follow symbolic links to files.
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	182 Overwrites any other symlink related options
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	183 (--logical, --half, --physical, --follow-directory-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	184 --no-follow-file-symlinks, et al.).
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	185 This is the default.""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	186 gp.add_argument(
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	187 "--print-size", action="store_true",
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	188 help="""Print the size of a file or the accumulated sizes of
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	189 directory content into the output also.
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	190 The size is not considered when computing digests. For symbolic links
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	191 the size is not printed also.""")
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	192 gp.add_argument(
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	193 "--size-only", action="store_true",
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	194 help="""Print only the size of files and for each directory its
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	195 accumulated directory size. Digests are not computed.""")
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	196 gp.add_argument(
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	197 "--utf8", "--utf-8", action="store_true",
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	198 help="""Encode all file paths using UTF-8 instead of
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	199 the filesystem encoding. Add some error tag into the path if it cannot
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	200 representated in Unicode cleanly.""")
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	201 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	202 "directories", nargs="*", metavar="DIRECTORY")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	203
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	204 def _populate_info_arguments(ip):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	205 ip.add_argument(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	206 "--last", action="store_true", dest="print_only_last_block",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	207 help="Print only the last block of every given input file")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	208 ip.add_argument(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	209 "digest_files", nargs="+", metavar="TREESUM-DIGEST-FILE")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	210
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	211 parser = argparse.ArgumentParser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	212 description="Generate and verify checksums for directory trees.",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	213 fromfile_prefix_chars='@',
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	214 add_help=False)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	215
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	216 #
153 3505406ef9f3 Comment Franz Glasner <fzglas.hg@dom66.de> parents: 152 diff changeset	217 # Global options for all sub-commands.
3505406ef9f3 Comment Franz Glasner <fzglas.hg@dom66.de> parents: 152 diff changeset	218 # In a group because this allows a customized title.
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	219 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	220 gparser = parser.add_argument_group(title="Global Options")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	221 gparser.add_argument(
191 1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	222 "--debug", action="store_true",
1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	223 help="Activate debug logging to stderr")
1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	224 gparser.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	225 "-v", "--version", action="version",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	226 version="%s (rv:%s)" % (__version__, __revision__),
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	227 help="Show program's version number and exit")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	228 gparser.add_argument(
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	229 "-h", "--help", action="help",
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	230 help="Show this help message and exit")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	231
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	232 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	233 # Subcommands
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	234 #
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	235 subparsers = parser.add_subparsers(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	236 dest="subcommand",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	237 title="Commands",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	238 description="This tool uses subcommands. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	239 "To see detailed help for a specific subcommand use "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	240 "the -h/--help option after the subcommand name. "
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	241 "A list of valid commands and their short descriptions "
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	242 "is listed below:",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	243 metavar="COMMAND")
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	244
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	245 genparser = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	246 "generate",
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	247 help="Generate checksums for directory trees",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	248 description="Generate checksums for directory trees.")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	249 _populate_generate_arguments(genparser)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	250 # And an alias for "generate"
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	251 genparser2 = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	252 "gen",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	253 help="Alias for \"generate\"",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	254 description="Generate checksums for directory trees. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	255 "This is an alias to \"generate\".")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	256 _populate_generate_arguments(genparser2)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	257
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	258 infoparser = subparsers.add_parser(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	259 "info",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	260 help="Print some information from given treesum digest file",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	261 description="""Print some informations from given treesum digest files
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	262 to stdout."""
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	263 )
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	264 _populate_info_arguments(infoparser)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	265
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	266 hparser = subparsers.add_parser(
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	267 "help",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	268 help="Show this help message or a subcommand's help and exit",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	269 description="Show this help message or a subcommand's help and exit.")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	270 hparser.add_argument("help_command", nargs='?', metavar="COMMAND")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	271
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	272 patparser = subparsers.add_parser(
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	273 "patterns",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	274 help="Show the help for PATTERNs and exit",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	275 description=fnmatch.HELP_DESCRIPTION,
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	276 formatter_class=argparse.RawDescriptionHelpFormatter)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	277
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	278 vparser = subparsers.add_parser(
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	279 "version",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	280 help="Show the program's version number and exit",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	281 description="Show the program's version number and exit.")
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	282
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	283 # Parse leniently to just check for "version" and/or help
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	284 opts, _dummy = parser.parse_known_args(args=argv)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	285
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	286 if opts.subcommand == "version":
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	287 print("%s (rv:%s)" % (__version__, __revision__),
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	288 file=sys.stdout)
174 fc1055878775 Use "return 0" instead of "sys.exit(0)" when printing version and help# Franz Glasner <fzglas.hg@dom66.de> parents: 173 diff changeset	289 return 0
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	290 if opts.subcommand == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	291 if not opts.help_command:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	292 parser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	293 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	294 if opts.help_command == "generate":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	295 genparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	296 elif opts.help_command == "gen":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	297 genparser2.print_help()
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	298 elif opts.help_command == "info":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	299 infoparser.print_help()
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	300 elif opts.help_command == "version":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	301 vparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	302 elif opts.help_command == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	303 hparser.print_help()
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	304 elif opts.help_command == "patterns":
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	305 patparser.print_help()
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	306 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	307 parser.print_help()
174 fc1055878775 Use "return 0" instead of "sys.exit(0)" when printing version and help# Franz Glasner <fzglas.hg@dom66.de> parents: 173 diff changeset	308 return 0
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	309
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	310 # Reparse strictly
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	311 opts = parser.parse_args(args=argv)
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	312
176 7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	313 # Minimal logging -- just for debugging - not for more "normal" use
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	314 logging.basicConfig(
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	315 level=logging.DEBUG if opts.debug else logging.WARNING,
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	316 stream=sys.stderr,
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	317 format="[%(asctime)s][%(levelname)s][%(process)d:%(name)s] %(message)s"
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	318 )
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	319 logging.captureWarnings(True)
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	320
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	321 return treesum(opts, patparser=patparser)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	322
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	323
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	324 FollowSymlinkConfig = collections.namedtuple("FollowSymlinkConfig",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	325 ["command_line",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	326 "directory",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	327 "file"])
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	328
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	329
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	330 class SymlinkAction(argparse.Action):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	331
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	332 """`type' is fixed here.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	333 `dest' is a tuple with three items:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	334
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	335 1. follow symlinks on the command line
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	336 2. follow directory symlinks while walking
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	337 3. follow file symlinks while walking (not yet implemented)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	338
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	339 """
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	340
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	341 def __init__(self, args, *kwargs):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	342 if "nargs" in kwargs:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	343 raise ValueError("`nargs' not allowed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	344 if "type" in kwargs:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	345 raise ValueError("`type' not allowed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	346 c = kwargs.get("const", None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	347 if c is None:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	348 raise ValueError("a const value is needed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	349 if (not isinstance(c, FollowSymlinkConfig)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	350 and c not in ("follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	351 "no-follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	352 "follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	353 "no-follow-file-symlinks")):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	354 raise ValueError(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	355 "invalid value for the `const' configuration value")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	356 default = kwargs.get("default", None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	357 if (default is not None
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	358 and not isinstance(default, FollowSymlinkConfig)):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	359 raise TypeError("invalid type for `default'")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	360 kwargs["nargs"] = 0
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	361 super(SymlinkAction, self).__init__(args, *kwargs)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	362
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	363 def __call__(self, parser, namespace, values, option_string=None):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	364 curval = getattr(namespace, self.dest, None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	365 if curval is None:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	366 curval = FollowSymlinkConfig(False, False, True)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	367 if isinstance(self.const, FollowSymlinkConfig):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	368 curval = self.const
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	369 else:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	370 if self.const == "follow-directory-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	371 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	372 curval.command_line, True, curval.file)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	373 elif self.const == "no-follow-directory-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	374 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	375 curval.command_line, False, curval.file)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	376 elif self.const == "follow-file-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	377 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	378 curval.command_line, curval.directory, True)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	379 elif self.const == "no-follow-file-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	380 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	381 curval.command_line, curval.directory, False)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	382 else:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	383 assert False, "Implementation error: not yet implemented"
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	384
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	385 setattr(namespace, self.dest, curval)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	386
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	387
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	388 class PatternMatchAction(argparse.Action):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	389
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	390 def __init__(self, args, *kwargs):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	391 if "nargs" in kwargs:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	392 raise argparse.ArgumentError(None, "`nargs' not allowed")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	393 if "type" in kwargs:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	394 raise argparse.ArgumentError(None, "`type' not allowed")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	395 kwargs["nargs"] = 1
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	396
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	397 self.__kind = kwargs.pop("kind", None)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	398 if self.__kind is None:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	399 raise argparse.ArgumentError(None, "`kind' is required")
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	400 if self.__kind not in ("exclude", "include", "accept-treesum"):
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	401 raise argparse.ArgumentError(
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	402 None,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	403 "`kind' must be one of `include', `exclude' or"
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	404 " `accept-treesum'")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	405
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	406 super(PatternMatchAction, self).__init__(args, *kwargs)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	407
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	408 def __call__(self, parser, namespace, values, option_string=None):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	409 items = getattr(namespace, self.dest, None)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	410 if items is None:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	411 items = []
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	412 setattr(namespace, self.dest, items)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	413 for v in values:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	414 items.append((self.__kind, v))
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	415
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	416
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	417 def gen_generate_opts(directories=[],
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	418 algorithm=util.default_algotag(),
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	419 append_output=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	420 base64=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	421 comment=[],
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	422 fnmatch_filters=[],
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	423 follow_symlinks=FollowSymlinkConfig(False, False, False),
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	424 full_mode=False,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	425 generator="normal",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	426 logical=None,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	427 minimal=None,
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	428 mode=False,
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	429 mmap=None,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	430 mtime=False,
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	431 output=None,
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	432 print_size=False,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	433 size_only=False,
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	434 utf8=False):
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	435 if not isinstance(follow_symlinks, FollowSymlinkConfig):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	436 raise TypeError("`follow_symlinks' must be a FollowSymlinkConfig")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	437 if not isinstance(fnmatch_filters, (list, tuple, type(None))):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	438 raise TypeError("`fnmatch_filters' must be a sequence (list, tuple)")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	439 if fnmatch_filters:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	440 for f in fnmatch_filters:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	441 if not isinstance(f, (tuple, list)):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	442 raise TypeError(
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	443 "items in `fnmatch_filters' must be tuples or lists")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	444 if f[0] not in ("exclude", "include"):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	445 raise ValueError(
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	446 "every kind of every item in `fnmatch_filters' must be"
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	447 " \"include\", \"exclude\" or \"accept-treesum\""
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	448 )
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	449
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	450 # Not following symlinks to files is not yet supported: reset to True
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	451 # if not follow_symlinks.file:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	452 # follow_symlinks = follow_symlinks._make([follow_symlinks.command_line,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	453 # follow_symlinks.directory,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	454 # True])
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	455 # logging.warning("Coercing to follow-symlinks-file")
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	456 opts = argparse.Namespace(
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	457 directories=directories,
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	458 algorithm=util.argv2algo(algorithm),
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	459 append_output=append_output,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	460 base64=base64,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	461 comment=comment,
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	462 fnmatch_filters=fnmatch_filters,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	463 follow_symlinks=follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	464 generator=generator,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	465 logical=logical,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	466 minimal=minimal,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	467 mmap=mmap,
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	468 metadata_full_mode=full_mode,
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	469 metadata_mode=mode,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	470 metadata_mtime=mtime,
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	471 output=output,
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	472 print_size=print_size,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	473 size_only=size_only,
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	474 utf8=utf8)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	475 return opts
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	476
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	477
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	478 def gen_info_opts(digest_files=[], last=False):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	479 opts = argparse.Namespace(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	480 digest_files=digest_files,
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	481 print_only_last_block=last)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	482 return opts
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	483
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	484
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	485 def treesum(opts, patparser=None):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	486 # XXX TBD: opts.check and opts.checklist (as in shasum.py)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	487 if opts.subcommand in ("generate", "gen"):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	488 return generate_treesum(opts)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	489 elif opts.subcommand == "info":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	490 return print_treesum_digestfile_infos(opts)
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	491 elif opts.subcommand == "patterns":
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	492 patparser.print_help()
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	493 else:
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	494 raise RuntimeError(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	495 "command `{}' not yet handled".format(opts.subcommand))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	496
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	497
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	498 def generate_treesum(opts):
149 f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	499 # Provide defaults
f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	500 if not opts.algorithm:
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	501 opts.algorithm = util.argv2algo(util.default_algotag())
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	502 if not opts.directories:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	503 opts.directories.append(".")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	504
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	505 if opts.output is None or opts.output == "-":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	506 if hasattr(sys.stdout, "buffer"):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	507 out_cm = cm.nullcontext(sys.stdout.buffer)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	508 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	509 out_cm = cm.nullcontext(sys.stdout)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	510 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	511 if opts.append_output:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	512 out_cm = open(opts.output, "ab")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	513 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	514 out_cm = open(opts.output, "wb")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	515
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	516 fnmatcher = fnmatch.FnMatcher.build_from_commandline_patterns(
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	517 opts.fnmatch_filters)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	518
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	519 with out_cm as outfp:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	520 writer = TreesumWriter(outfp)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	521 for d in opts.directories:
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	522 V1DirectoryTreesumGenerator(
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	523 opts.algorithm, opts.mmap, opts.base64,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	524 opts.follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	525 opts.generator,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	526 opts.metadata_mode,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	527 opts.metadata_full_mode,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	528 opts.metadata_mtime,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	529 opts.size_only,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	530 opts.print_size,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	531 opts.utf8,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	532 minimal=opts.minimal,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	533 fnmatcher=fnmatcher).generate(
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	534 writer, d, comment=opts.comment)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	535
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	536
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	537 class V1DirectoryTreesumGenerator(object):
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	538
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	539 def __init__(self, algorithm, use_mmap, use_base64,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	540 follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	541 with_generator,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	542 with_metadata_mode, with_metadata_full_mode,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	543 with_metadata_mtime, size_only, print_size, utf8_mode,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	544 minimal=None,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	545 fnmatcher=None):
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	546 super(V1DirectoryTreesumGenerator, self).__init__()
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	547 self._algorithm = algorithm
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	548 self._use_mmap = use_mmap
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	549 self._use_base64 = use_base64
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	550 self._follow_symlinks = follow_symlinks
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	551 self._with_generator = with_generator
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	552 self._with_metadata_mode = with_metadata_mode
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	553 self._with_metadata_full_mode = with_metadata_full_mode
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	554 self._with_metadata_mtime = with_metadata_mtime
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	555 self._size_only = size_only
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	556 self._print_size = print_size
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	557 self._utf8_mode = utf8_mode
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	558 self._minimal = minimal
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	559 self._fnmatcher = fnmatcher
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	560
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	561 def generate(self, writer, root, comment=None):
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	562 """
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	563
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	564 :param outfp: a binary file with a "write()" and a "flush()" method
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	565
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	566 """
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	567 self._writer = writer
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	568 self._writer.start("1")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	569 self._writer.write_fsencoding(util.n(walk.getfsencoding().upper()))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	570 self._writer.flush()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	571
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	572 if self._with_generator == "none":
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	573 pass # do nothing
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	574 elif self._with_generator == "normal":
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	575 self._writer.write_generator("PY2" if util.PY2 else "PY3")
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	576 elif self._with_generator == "full":
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	577 import platform
295 4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	578 info = ("treesum %s (rv:%s), %s %s, %s"
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	579 % (__version__,
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	580 __revision__,
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	581 platform.python_implementation(),
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	582 platform.python_version(),
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	583 platform.platform()))
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	584 self._writer.write_generator(info)
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	585 else:
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	586 raise NotImplementedError(
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	587 "not implemented: %s" % (self._with_generator,))
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	588
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	589 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	590 # Note: Given non-default flags that are relevant for
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	591 # directory traversal.
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	592 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	593 flags = []
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	594 if self._with_metadata_full_mode:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	595 flags.append("with-metadata-fullmode")
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	596 elif self._with_metadata_mode:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	597 flags.append("with-metadata-mode")
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	598 if self._with_metadata_mtime:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	599 flags.append("with-metadata-mtime")
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	600 flags.append("follow-symlinks-commandline"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	601 if self._follow_symlinks.command_line
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	602 else "no-follow-symlinks-commandline")
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	603 flags.append("follow-symlinks-directory"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	604 if self._follow_symlinks.directory
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	605 else "no-follow-symlinks-directory")
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	606 flags.append("follow-symlinks-file"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	607 if self._follow_symlinks.file
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	608 else "no-follow-symlinks-file")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	609 if self._size_only:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	610 flags.append("size-only")
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	611 flags.append("utf8-encoding" if self._utf8_mode else "fs-encoding")
211 5bb0b25f8e99 FIX: FLAGS output: wrong "else" for "utf-8-mode" and "print-size" Franz Glasner <fzglas.hg@dom66.de> parents: 210 diff changeset	612 if self._print_size:
5bb0b25f8e99 FIX: FLAGS output: wrong "else" for "utf-8-mode" and "print-size" Franz Glasner <fzglas.hg@dom66.de> parents: 210 diff changeset	613 flags.append("print-size")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	614 self._writer.write_flags(flags)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	615
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	616 if self._minimal is None:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	617 # Write execution timestamps in POSIX epoch and ISO format
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	618 ts = int(time.time())
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	619 self._writer.write_timestamp(ts)
291 cc2deeb5f2e6 treesum: Indicate that the ISO timestamp is UTC Franz Glasner <fzglas.hg@dom66.de> parents: 290 diff changeset	620 ts = (datetime.datetime.utcfromtimestamp(ts)).isoformat("T") + "Z"
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	621 self._writer.write_isotimestamp(ts)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	622
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	623 if comment:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	624 for line in comment:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	625 self._writer.write_comment(line)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	626
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	627 for action, kind, pattern in self._fnmatcher.definitions():
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	628 self._writer.write_fnmatch_pattern(action, kind, pattern)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	629
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	630 if self._minimal is not None:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	631 self._writer.write_root(
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	632 (walk.WalkDirEntry.alt_u8(self._minimal)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	633 if self._minimal else b""))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	634 else:
278 822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed. Franz Glasner <fzglas.hg@dom66.de> parents: 277 diff changeset	635 self._writer.write_root(walk.WalkDirEntry.alt_u8(
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed. Franz Glasner <fzglas.hg@dom66.de> parents: 277 diff changeset	636 util.normalize_filename(root, True)))
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	637
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	638 self._writer.flush()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	639
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	640 if not self._follow_symlinks.command_line and os.path.islink(root):
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	641 linktgt = walk.WalkDirEntry.from_readlink(os.readlink(root))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	642 linkdgst = self._algorithm[0]()
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	643 linkdgst.update(linktgt.fspath)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	644 dir_dgst = self._algorithm[0]()
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	645 dir_dgst.update(b"2:L@,")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	646 dir_dgst.update(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	647 util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	648 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	649 len(self._algorithm[1]), util.b(self._algorithm[1]),
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	650 len(linkdgst.digest()), linkdgst.digest()))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	651 if self._size_only:
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	652 self._writer.write_size(b"./@/", "")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	653 else:
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	654 sz = "" if self._print_size else None
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	655 self._writer.write_file_digest(
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	656 self._algorithm[1],
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	657 b"./@/",
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	658 dir_dgst.digest(),
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	659 self._use_base64,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	660 size=sz)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	661 self._writer.flush()
268 8aadffaaad5f treesum: refactor: remove early return and use an "else" branch instead Franz Glasner <fzglas.hg@dom66.de> parents: 267 diff changeset	662 else:
8aadffaaad5f treesum: refactor: remove early return and use an "else" branch instead Franz Glasner <fzglas.hg@dom66.de> parents: 267 diff changeset	663 self._generate(os.path.normpath(root), tuple())
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	664 self._writer.finish()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	665
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	666 def _generate(self, root, top):
301 d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	667 if top:
d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	668 logging.debug("Recursing into directory: %s/%r", root, top)
d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	669 else:
d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	670 logging.debug("Handling root directory: %s", root)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	671 path = os.path.join(root, *top) if top else root
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	672 # Determine also the path to be used for directory filtering
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	673 fpath = join_output_path(top, None) if top else ""
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	674 if self._fnmatcher:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	675 if not self._fnmatcher.shall_visit(fpath):
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	676 logging.debug("Skipping directory: %s", fpath)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	677 return (None, None, None, None)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	678 try:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	679 with walk.ScanDir(path) as dirscan:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	680 fsobjects = list(dirscan)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	681 except OSError as e:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	682 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	683 # NOTE: Sync the error handler code with this method's
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	684 # code below before returning!
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	685 #
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	686 if e.errno == errno.ENOTDIR:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	687 # object exists but is not a directory
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	688 errmsg = b"not a directory"
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	689 elif e.errno in (errno.EACCES, errno.EPERM,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	690 getattr(errno, "ENOTCAPABLE", errno.EACCES)):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	691 # no permissions
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	692 errmsg = (
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	693 b"access denied / no permissions / missing capabilities")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	694 elif e.errno == errno.ENOENT:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	695 # given object does not exist
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	696 errmsg = b"no such file or directory"
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	697 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	698 raise
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	699 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	700 opath = walk.WalkDirEntry.alt_u8(path)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	701 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	702 opath = walk.WalkDirEntry.alt_fs(path)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	703 self._writer.write_error(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	704 b"`%s': %s", opath, errmsg))
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	705 # Reuse from top
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	706 opath = join_output_path(top, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	707 if opath:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	708 if self._utf8_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	709 opath = walk.WalkDirEntry.alt_u8(opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	710 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	711 opath = walk.WalkDirEntry.alt_fs(opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	712 if self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	713 self._writer.write_size(opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	714 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	715 self._writer.write_file_digest(self._algorithm[1], opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	716 self._writer.flush()
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	717 return (e.errno, None, None, None)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	718
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	719 # Check whether to accept existing treesum digest files
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	720 if self._fnmatcher:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	721 for fso in fsobjects:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	722 fpath = join_output_path(top, fso.name)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	723 if self._fnmatcher.shall_accept_treesum(fpath):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	724 # Yes we have hit a treesum digest file
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	725 logging.debug("Accepting existing treesum from: %s", fpath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	726 collector = DigestSizeCollector()
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	727 try:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	728 collector.collect_from_file(os.path.join(root, fpath))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	729 except OSError as e:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	730 eno = e.errno
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	731 emsg = e.strerror
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	732 except Exception as e:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	733 # XXX FIXME: other EIO, EBADF, EFAULT
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	734 eno = errno.ESRCH
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	735 emsg = str(e)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	736 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	737 eno = 0
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	738 emsg = None
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	739 if self._utf8_mode:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	740 fpath = walk.WalkDirEntry.alt_u8(fpath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	741 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	742 fpath = walk.WalkDirEntry.alt_fs(fpath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	743 opath = join_output_path(top, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	744 if self._utf8_mode:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	745 opath = walk.WalkDirEntry.alt_u8(opath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	746 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	747 opath = walk.WalkDirEntry.alt_fs(opath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	748 if eno == 0:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	749 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	750 # treesum file could be read.
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	751 # Now check whether the infos we got from it are
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	752 # compatible with our current requirements
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	753 # (digest, size).
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	754 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	755 if self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	756 if collector.size is None:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	757 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	758 # This is a severe error here: just the size
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	759 # is required, but we have not got one.
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	760 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	761 self._writer.write_error(util.b(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	762 util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	763 b"No size in treesum-file `%s' while"
aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	764 b" requiring it",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	765 fpath),
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	766 "utf-8"))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	767 self._writer.write_size(opath, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	768 return (errno.ESRCH, None, None, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	769 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	770 if self._print_size:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	771 if collector.size is None:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	772 #
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	773 # XXX FIXME: Is this a severe error
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	774 # here? Currently: no
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	775 #
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	776 self._writer.write_error(util.b(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	777 util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	778 b"No size in treesum-file `%s'",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	779 fpath),
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	780 "utf-8"))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	781 sz = ""
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	782 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	783 sz = collector.size
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	784 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	785 sz = None
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	786 if collector.digest is None:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	787 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	788 # This is really a severe error. Most probably
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	789 # the treesum file was created with
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	790 # "--size-only" and contains no digest.
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	791 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	792 self._writer.write_error(util.b(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	793 util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	794 b"No digest in treesum-file `%s' while"
aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	795 b" it is required",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	796 fpath),
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	797 "utf-8"))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	798 self._writer.write_file_digest(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	799 collector.algorithm or "MD5",
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	800 opath,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	801 None,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	802 use_base64=self._use_base64,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	803 size=sz)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	804 return (errno.ESRCH, None, None, None)
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	805 # We got all required infos without errors
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	806 self._writer.write_accept_treesum_file(fpath)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	807 if self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	808 self._writer.write_size(opath, collector.size)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	809 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	810 self._writer.write_file_digest(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	811 collector.algorithm, opath, collector.digest,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	812 use_base64=self._use_base64, size=sz)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	813 return (0,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	814 collector.algorithm,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	815 collector.digest,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	816 collector.size)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	817 else:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	818 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	819 # treesum file could not be read
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	820 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	821 self._writer.write_error(util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	822 b"Cannot read treesum-file `%s' for directory"
aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	823 b"`%s': %s",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	824 fpath,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	825 opath,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	826 util.b(emsg, "utf-8")))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	827 if self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	828 self._writer.write_size(opath, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	829 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	830 self._writer.write_file_digest(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	831 self._algorithm[1], opath, None,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	832 use_base64=self._use_base64, size=None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	833 return (eno, None, None, None)
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	834 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	835 # No treesum file: just process normally with digesting
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	836 #
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	837 if self._utf8_mode:
221 ca9d5a0dc9bb Rename WalkDirEntry.sort_key and WalkDirEntry.alt_sort_key Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	838 fsobjects.sort(key=walk.WalkDirEntry.sort_key_u8)
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	839 else:
221 ca9d5a0dc9bb Rename WalkDirEntry.sort_key and WalkDirEntry.alt_sort_key Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	840 fsobjects.sort(key=walk.WalkDirEntry.sort_key_fs)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	841 dir_dgst = self._algorithm[0]()
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	842 dir_size = 0
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	843 dir_tainted = False
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	844 for fso in fsobjects:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	845 # Determine the effective name to be used for digesting
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	846 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	847 if fso.u8name is None:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	848 dir_tainted = True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	849 effective_fso_name = fso.alt_u8name
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	850 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	851 effective_fso_name = fso.u8name
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	852 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	853 if fso.fsname is None:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	854 dir_tainted = True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	855 effective_fso_name = fso.alt_fsname
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	856 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	857 effective_fso_name = fso.fsname
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	858 # Determine the path (mostly its prefix) that is to be printed
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	859 opath = join_output_path(top, fso.name)
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	860 # Determine the path to be used for filename filtering
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	861 fpath = opath
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	862 if self._fnmatcher:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	863 if not self._fnmatcher.shall_visit(fpath):
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	864 logging.debug("Skipping: %s", fpath)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	865 continue
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	866 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	867 opath = walk.WalkDirEntry.alt_u8(opath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	868 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	869 opath = walk.WalkDirEntry.alt_fs(opath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	870 if fso.is_special:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	871 # Determine the tag character
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	872 if fso.is_chr:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	873 special_tag = b':'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	874 elif fso.is_blk:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	875 special_tag = b';'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	876 elif fso.is_fifo:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	877 special_tag = b'\|'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	878 elif fso.is_socket:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	879 special_tag = b'='
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	880 elif fso.is_door:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	881 special_tag = b'>'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	882 elif fso.is_whiteout:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	883 special_tag = b'%'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	884 elif fso.is_eventport:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	885 special_tag = b'+'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	886 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	887 assert False, "unknown special filesystem object"
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	888 assert fso.stat is not None # because .is_special is True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	889 if fso.is_symlink and not self._follow_symlinks.file:
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	890 linktgt = walk.WalkDirEntry.from_readlink(
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	891 os.readlink(fso.path))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	892 linkdgst = self._algorithm[0]()
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	893 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	894 if linktgt.u8path is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	895 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	896 linkdgst.update(linktgt.alt_u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	897 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	898 linkdgst.update(linktgt.u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	899 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	900 if linktgt.fspath is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	901 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	902 linkdgst.update(linktgt.alt_fspath)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	903 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	904 linkdgst.update(linktgt.fspath)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	905 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	906 b"2:@%s,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	907 special_tag,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	908 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	909 effective_fso_name))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	910 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	911 b"%d:%s,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	912 len(self._algorithm[1]), util.b(self._algorithm[1]),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	913 len(linkdgst.digest()), linkdgst.digest()))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	914 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	915 # - no mtime and no mode for symlinks
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	916 # - also does not count for dir_size
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	917 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	918 if self._size_only:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	919 self._writer.write_size(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	920 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	921 b"%s/./@%s", opath, special_tag),
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	922 "")
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	923 else:
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	924 sz = "" if self._print_size else None
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	925 self._writer.write_file_digest(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	926 self._algorithm[1],
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	927 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	928 b"%s/./@%s", opath, special_tag),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	929 linkdgst.digest(),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	930 self._use_base64,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	931 size=sz)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	932 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	933 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	934 # Follow the symlink to special file and/or handle a
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	935 # special file
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	936 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	937 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	938 b"1:%s,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	939 special_tag,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	940 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	941 effective_fso_name))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	942 # no important size here but a mode
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	943 if self._with_metadata_mtime:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	944 mtime = datetime.datetime.utcfromtimestamp(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	945 int(fso.stat.st_mtime))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	946 mtime = util.b(mtime.isoformat("T") + "Z")
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	947 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	948 b"5:mtime,%d:%s,", len(mtime), mtime))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	949 if self._with_metadata_full_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	950 modestr = util.b(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	951 normalized_mode_str(fso.stat.st_mode))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	952 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	953 b"8:fullmode,%d:%s,", len(modestr), modestr))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	954 elif self._with_metadata_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	955 modestr = util.b(normalized_compatible_mode_str(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	956 fso.stat.st_mode))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	957 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	958 b"4:mode,%d:%s,", len(modestr), modestr))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	959 if self._size_only:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	960 self._writer.write_size(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	961 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	962 b"%s/./%s", opath, special_tag),
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	963 "")
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	964 else:
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	965 sz = "" if self._print_size else None
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	966 self._writer.write_file_digest(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	967 self._algorithm[1],
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	968 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	969 b"%s/./%s", opath, special_tag),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	970 b"",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	971 self._use_base64,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	972 size=sz)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	973 elif fso.is_dir:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	974 assert fso.stat is not None # because .is_dir is True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	975 if fso.is_symlink and not self._follow_symlinks.directory:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	976 linktgt = walk.WalkDirEntry.from_readlink(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	977 os.readlink(fso.path))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	978 linkdgst = self._algorithm[0]()
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	979 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	980 if linktgt.u8path is None:
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	981 dir_tainted = True
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	982 linkdgst.update(linktgt.alt_u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	983 else:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	984 linkdgst.update(linktgt.u8path)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	985 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	986 if linktgt.fspath is None:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	987 dir_tainted = True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	988 linkdgst.update(linktgt.alt_fspath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	989 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	990 linkdgst.update(linktgt.fspath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	991 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	992 b"2:@/,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	993 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	994 effective_fso_name))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	995 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	996 # - no mtime and no mode for symlinks
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	997 # - also does not count for dir_size
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	998 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	999 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1000 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1001 len(self._algorithm[1]), util.b(self._algorithm[1]),
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1002 len(linkdgst.digest()), linkdgst.digest()))
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1003 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1004 self._writer.write_size(
218 dee891ed2307 FIX: Output of symlinks was not converted property but was written in bytes repr Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	1005 util.interpolate_bytes(b"%s/./@/", opath),
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	1006 "")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1007 else:
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	1008 sz = "" if self._print_size else None
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1009 self._writer.write_file_digest(
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1010 self._algorithm[1],
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1011 util.interpolate_bytes(b"%s/./@/", opath),
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1012 linkdgst.digest(),
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1013 self._use_base64)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1014 else:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1015 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1016 # Follow the symlink to dir or handle a "real" directory
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1017 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1018
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1019 # Get subdir data from recursing into it
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1020 sub_dir_errno, sub_dir_algo, sub_dir_dgst, sub_dir_size = \
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1021 self._generate(root, top + (fso.name, ))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1022
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1023 #
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1024 # Check first whether the directory was selected to be
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1025 # excluded
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1026 #
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1027 if sub_dir_errno is None:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1028 # Yes -- skipped
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1029 continue
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1030 if sub_dir_errno == 0:
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1031 if sub_dir_size is None:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1032 if self._print_size or self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1033 dir_tainted = True
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1034 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1035 dir_size += (sub_dir_size or 0)
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1036 else:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1037 dir_tainted = True
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1038 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1039 b"1:/,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1040 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1041 effective_fso_name))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1042 if self._with_metadata_full_mode:
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1043 modestr = util.b(normalized_mode_str(fso.stat.st_mode))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1044 dir_dgst.update(util.interpolate_bytes(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1045 b"8:fullmode,%d:%s,", len(modestr), modestr))
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1046 elif self._with_metadata_mode:
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1047 modestr = util.b(normalized_compatible_mode_str(
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1048 fso.stat.st_mode))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1049 dir_dgst.update(util.interpolate_bytes(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1050 b"4:mode,%d:%s,", len(modestr), modestr))
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1051 if sub_dir_errno == 0:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1052 dir_dgst.update(util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1053 b"%d:%s,%d:%s,",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1054 len(sub_dir_algo), util.b(sub_dir_algo),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1055 len(sub_dir_dgst), sub_dir_dgst))
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1056 else:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1057 # NOTE: error message is already printed here
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1058 dir_dgst.update(util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1059 b"5:errno,%d:%s",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1060 len(str(sub_dir_errno)),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1061 util.b(str(sub_dir_errno))))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1062 else:
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1063 if fso.is_symlink and not self._follow_symlinks.file:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1064 #
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1065 # Symbolic link to some filesystem object which is not
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1066 # determined to be a link to a directory or some other
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1067 # special file (socket, FIFO, et al.).
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1068 #
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1069 linktgt = walk.WalkDirEntry.from_readlink(
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1070 os.readlink(fso.path))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1071 linkdgst = self._algorithm[0]()
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1072 if self._utf8_mode:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1073 if linktgt.u8path is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1074 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1075 linkdgst.update(linktgt.alt_u8path)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1076 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1077 linkdgst.update(linktgt.u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1078 else:
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1079 if linktgt.fspath is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1080 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1081 linkdgst.update(linktgt.alt_fspath)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1082 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1083 linkdgst.update(linktgt.fspath)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1084 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1085 b"1:@,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1086 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1087 effective_fso_name))
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1088 #
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1089 # - no mtime and no mode for symlinks
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1090 # - also does not count for dir_size
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1091 #
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1092 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1093 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1094 len(self._algorithm[1]), util.b(self._algorithm[1]),
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1095 len(linkdgst.digest()), linkdgst.digest()))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1096 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1097 self._writer.write_size(
218 dee891ed2307 FIX: Output of symlinks was not converted property but was written in bytes repr Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	1098 util.interpolate_bytes(b"%s/./@", opath),
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	1099 "")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1100 else:
290 3c3f8151f36a treesum: FIX: Output of empty sizes in Python 3: print a realy empty string instead of "b''" Franz Glasner <fzglas.hg@dom66.de> parents: 278 diff changeset	1101 sz = "" if self._print_size else None
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1102 self._writer.write_file_digest(
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1103 self._algorithm[1],
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1104 util.interpolate_bytes(b"%s/./@", opath),
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1105 linkdgst.digest(),
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1106 self._use_base64,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1107 size=sz)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1108 else:
223 61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1109 #
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1110 # Follow the symlink to file or handle a "real" file
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1111 #
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1112
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1113 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1114 b"0:,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1115 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1116 effective_fso_name))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1117 if fso.stat is None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1118 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1119 # Error: most likely a broken symlink here
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1120 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1121 dir_tainted = True
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1122 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1123 b"5:errno,%d:%s,",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1124 len(str(fso.stat_errno)),
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1125 util.b(str(fso.stat_errno))))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1126 self._writer.write_error(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1127 b"errno %d: %s",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1128 fso.stat_errno,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1129 util.b(fso.stat_errstr, "utf-8")))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1130 logging.error(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1131 "Directory entry has symlink problems: %r",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1132 opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1133 if self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1134 self._writer.write_size(opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1135 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1136 self._writer.write_file_digest(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1137 self._algorithm[1], opath, None)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1138 else:
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1139 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1140 # Ok: File has normal stat info
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1141 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1142 # XXX FIXME: Handle special files (fifo, socket,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1143 # block or char devices, ...).
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1144 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1145 dir_size += fso.stat.st_size
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1146 if self._with_metadata_mtime:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1147 mtime = datetime.datetime.utcfromtimestamp(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1148 int(fso.stat.st_mtime))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1149 mtime = util.b(mtime.isoformat("T") + "Z")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1150 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1151 b"5:mtime,%d:%s,", len(mtime), mtime))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1152 if self._with_metadata_full_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1153 modestr = util.b(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1154 normalized_mode_str(fso.stat.st_mode))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1155 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1156 b"8:fullmode,%d:%s,", len(modestr), modestr))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1157 elif self._with_metadata_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1158 modestr = util.b(normalized_compatible_mode_str(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1159 fso.stat.st_mode))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1160 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1161 b"4:mode,%d:%s,", len(modestr), modestr))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1162 if self._size_only:
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1163 #
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1164 # size can be printed here because .stat is
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1165 # available
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1166 #
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1167 self._writer.write_size(opath, fso.stat.st_size)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1168 else:
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1169 try:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1170 dgst = digest.compute_digest_file(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1171 self._algorithm[0],
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1172 fso.path,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1173 use_mmap=self._use_mmap)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1174 except OSError as e:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1175 dir_tainted = True
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1176 self._writer.write_error(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1177 util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1178 b"`%s': errno %d: %s",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1179 opath,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1180 e.errno,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1181 util.b(e.strerror, "utf-8")))
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1182 sz = (fso.stat.st_size if self._print_size
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1183 else None)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1184 self._writer.write_file_digest(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1185 self._algorithm[1], opath, None,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1186 size=sz)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1187 else:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1188 dir_dgst.update(util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1189 b"%d:%s,%d:%s,",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1190 len(self._algorithm[1]),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1191 util.b(self._algorithm[1]),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1192 len(dgst),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1193 dgst))
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1194 sz = (fso.stat.st_size if self._print_size
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1195 else None)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1196 self._writer.write_file_digest(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1197 self._algorithm[1], opath, dgst,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1198 use_base64=self._use_base64,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1199 size=sz)
267 b9aa65a30b4c treesum: optimize the use of flush() somewhat Franz Glasner <fzglas.hg@dom66.de> parents: 266 diff changeset	1200 self._writer.flush()
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1201 opath = join_output_path(top, None)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1202 if opath:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1203 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1204 opath = walk.WalkDirEntry.alt_u8(opath)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1205 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1206 opath = walk.WalkDirEntry.alt_fs(opath)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1207 if dir_tainted:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1208 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1209 # IMPORTANT: Print errors BEFORE the associated digest or size
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1210 # line. Otherwise the "info" command has a problem.
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1211 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1212 self._writer.write_error(b"directory is tainted")
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1213 logging.error("Directory has problems: %r", opath)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1214 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1215 self._writer.write_size(opath, dir_size)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1216 else:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1217 sz = dir_size if self._print_size else None
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1218 self._writer.write_file_digest(
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1219 self._algorithm[1], opath, dir_dgst.digest(),
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1220 use_base64=self._use_base64, size=sz)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1221 self._writer.flush()
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1222 return (0, self._algorithm[1], dir_dgst.digest(), dir_size)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1223
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1224
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1225 def join_output_path(top, name):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1226 if name is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1227 # a path for a directory is to be computed
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1228 if top:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1229 if isinstance(top[0], bytes):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1230 return b"/".join(top) + b"/"
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1231 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1232 return u"/".join(top) + u"/"
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1233 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1234 return b""
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1235 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1236 # a path for a normal file is to be computed
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1237 if top:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1238 if isinstance(name, bytes):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1239 return b"/".join(top) + b"/" + name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1240 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1241 return u"/".join(top) + u"/" + name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1242 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1243 return name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1244
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1245
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1246 def normalized_compatible_mode_str(mode):
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1247 # XXX FIXME: Windows and "executable"
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1248 modebits = stat.S_IMODE(mode)
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1249 modestr = "%o" % (modebits,)
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1250 if not modestr.startswith("0"):
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1251 modestr = "0" + modestr
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1252 return modestr
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1253
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1254
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1255 def normalized_mode_str(mode):
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1256 modestr = "%o" % (mode,)
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1257 if not modestr.startswith("0"):
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1258 modestr = "0" + modestr
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1259 return modestr
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1260
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1261
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1262 class TreesumWriter(object):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1263
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1264 """Writer to write treesum digest files in a format similar to BSD
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1265 digest files.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1266
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1267 Wraps an output file pointer for a binary file.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1268
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1269 Provides high-level methods to write data lines.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1270
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1271 Also holds the current CRC for a block.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1272
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1273 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1274
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1275 LS = util.b(os.linesep)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1276
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1277 def __init__(self, outfp):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1278 self._outfp = outfp
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1279 self._reset_crc()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1280
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1281 def _reset_crc(self):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1282 self._crc = crc32()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1283
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1284 def start(self, version):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1285 """Begin a new block, reset the current CRC and write the VERSION
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1286 tag.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1287
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1288 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1289 self._reset_crc()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1290 self.write(b"VERSION = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1291 self.writeln(util.b(version))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1292
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1293 def write_comment(self, comment):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1294 self.write(b"COMMENT (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1295 self.write(util.b(comment, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1296 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1297
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1298 def write_generator(self, generator):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1299 self.write(b"GENERATOR (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1300 self.write(util.b(generator, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1301 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1302
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1303 def write_error(self, error):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1304 self.write(b"ERROR (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1305 self.write(util.b(error, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1306 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1307
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1308 def write_fsencoding(self, encoding):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1309 self.write(b"FSENCODING = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1310 self.writeln(util.b(encoding))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1311
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1312 def write_fnmatch_pattern(self, action, kind, pattern):
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1313 self.write(b"FNMATCH (")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1314 self.write(util.b(action))
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1315 self.write(b": ")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1316 self.write(util.b(kind))
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1317 self.write(b":")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1318 self.write(util.b(pattern, "utf-8"))
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1319 self.writeln(b")")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1320
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1321 def write_flags(self, flags):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1322 self.write(b"FLAGS = ")
262 c3d6599c1b5e FIX: treesum: bytearray needs not to be handled Franz Glasner <fzglas.hg@dom66.de> parents: 261 diff changeset	1323 if isinstance(flags, (str, bytes)):
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1324 self.writeln(util.b(flags))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1325 else:
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1326 flags.sort()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1327 self.writeln(util.b(",".join(flags)))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1328
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1329 def write_timestamp(self, ts):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1330 self.write(b"TIMESTAMP = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1331 self.writeln(util.b(str(ts)))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1332
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1333 def write_isotimestamp(self, ts):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1334 self.write(b"ISOTIMESTAMP = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1335 self.writeln(util.b(ts))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1336
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1337 def write_root(self, root):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1338 assert isinstance(root, bytes)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1339 self.write(b"ROOT (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1340 self.write(root)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1341 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1342
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1343 def write_size(self, filename, sz):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1344 assert isinstance(filename, bytes)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1345 self.write(b"SIZE (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1346 self.write(filename)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1347 self.write(b")")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1348 if sz is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1349 self.write(b" = ")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1350 self.write(util.b(str(sz)))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1351 self.writeln(b"")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1352
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1353 def write_accept_treesum_file(self, filename):
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1354 assert isinstance(filename, bytes)
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1355 self.write(b"ACCEPT-TREESUM (")
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1356 self.write(filename)
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1357 self.writeln(b")")
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1358
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1359 def write_file_digest(self, algorithm, filename, digest,
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1360 use_base64=False, size=None):
314 1fbe9904b188 treesum: assert also in .write_accept_treesum() that the given filename is of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 313 diff changeset	1361 assert isinstance(filename, bytes)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1362 if digest is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1363 digest = (base64.b64encode(digest)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1364 if use_base64
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1365 else binascii.hexlify(digest))
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1366 self.write(util.b(algorithm))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1367 self.write(b" (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1368 self.write(filename)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1369 self.write(b")")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1370 if digest is not None or size is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1371 self.write(b" = ")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1372 if digest is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1373 self.write(digest)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1374 if size is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1375 self.write(b",")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1376 self.write(util.b(str(size)))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1377 self.writeln(b"")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1378
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1379 def finish(self):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1380 """Finish a block and write the current CRC"""
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1381 crc = self._crc.hexdigest()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1382 self.write(b"CRC32 = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1383 self.writeln(util.b(crc))
267 b9aa65a30b4c treesum: optimize the use of flush() somewhat Franz Glasner <fzglas.hg@dom66.de> parents: 266 diff changeset	1384 self.flush()
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1385
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1386 def writeln(self, line):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1387 """Write the bytes `line` into the output file and update the CRC
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1388 accordingly.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1389
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1390 :param bytes line: The line to write to (without line ending)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1391
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1392 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1393 self.write(line)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1394 self.write(self.LS)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1395
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1396 def write(self, data):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1397 """Write `data` into the output file and update the CRC accordingly.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1398
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1399 :param bytes data: The data to write to and to update the CRC with
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1400
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1401 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1402 if data:
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1403 self._outfp.write(data)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1404 self._crc.update(data)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1405
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1406 def flush(self):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1407 self._outfp.flush()
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1408
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1409
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1410 class TreesumReader(object):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1411
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1412 """Reader to read and/or verify treesum digest files.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1413
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1414 Supports the iterator and context manager protocol.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1415
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1416 """
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1417
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1418 PATTERN0 = re.compile(br"\A[ \t]*\r?\n\Z") # empty lines
198 c1e875ba4bdc Put the effective filesystem encoding into the treesum digest file using FSENCODING = <encoding> Franz Glasner <fzglas.hg@dom66.de> parents: 197 diff changeset	1419 PATTERN1 = re.compile(br"\A(VERSION\|FSENCODING\|FLAGS\|TIMESTAMP\|ISOTIMESTAMP\|CRC32)[ \t]=[ \t]([^ \t]+)[ \t]*\r?\n\Z") # noqa: E501 line too long
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1420 PATTERN2 = re.compile(br"\A(ROOT\|COMMENT\|ERROR\|GENERATOR\|FNMATCH\|ACCEPT-TREESUM)[ \t]\((.)\)[ \t]*\r?\n\Z") # noqa: E501 line too long
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1421 PATTERN3 = re.compile(br"\ASIZE[ \t]\((.)\)([ \t]=[ \t](\d+))?[ \t]*\r?\n\Z") # noqa: E501 line too long
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1422 PATTERN4 = re.compile(br"\A([A-Za-z0-9_-]+)[ \t]\((.)\)([ \t]=[ \t]([A-Za-z0-9=+/]+)?(,(\d+)?)?)?[ \t]*\r?\n\Z") # noqa: E501 line too long
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1423
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1424 def __init__(self, _fp, _filename, _own_fp):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1425 self._fp = _fp
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1426 self._own_fp = _own_fp
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1427 self._filename = _filename
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1428 self._line_no = 0
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1429 self._reset_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1430 self._expect_crc = None # NOTE: tristate: None is different from False
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1431 self._current_algo_name = self._current_algo_digest_size = None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1432
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1433 @classmethod
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1434 def from_path(cls_, path):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1435 """Open file at `path` and return a reader that owns the file object"""
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1436 return cls_(open(path, "rb"), path, True)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1437
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1438 @classmethod
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1439 def from_binary_buffer(cls_, binary_fp, filename):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1440 return cls_(binary_fp, filename, False)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1441
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1442 def __enter__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1443 return self
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1444
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1445 def __exit__(self, *args):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1446 self.close()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1447
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1448 def close(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1449 if self._fp is not None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1450 try:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1451 if self._own_fp:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1452 self._fp.close()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1453 finally:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1454 self._fp = None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1455
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1456 def __iter__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1457 return self
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1458
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1459 def __next__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1460 rec = self.read_record()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1461 if rec is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1462 raise StopIteration()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1463 return rec
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1464
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1465 if util.PY2:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1466 next = __next__
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1467
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1468 def all_records(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1469 """Iterator over all remaining records"""
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1470 while True:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1471 rec = self.read_record()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1472 if rec is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1473 return
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1474 yield rec
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1475
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1476 def read_record(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1477 """Read and parse the "next" line.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1478
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1479 :returns: `None` at EOF or the parsed contents of the line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1480 :rtype: tuple or None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1481
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1482 """
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1483 # Loop to skip empty lines
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1484 while True:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1485 line = self._get_next_line()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1486 if not line:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1487 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1488 # Skip for empty files at the very beginning.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1489 # Check only after the first VERSION line.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1490 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1491 if self._expect_crc is not None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1492 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1493 logging.warning("CRC32 is missing at EOF")
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1494 return None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1495 if not self.PATTERN0.search(line):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1496 break
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1497 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1498 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1499 # At the beginning transparently skip an eventually embedded signify
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1500 # signature
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1501 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1502 if self._line_no == 1:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1503 if line.startswith(b"untrusted comment: "):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1504 line = self._get_next_line()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1505 if not line.endswith(b"\n"):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1506 raise binascii.Error("No valid signify signature value")
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1507 # Try to decode for an early error check
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1508 base64.b64decode(line[:-1])
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1509 mo = self.PATTERN1.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1510 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1511 if mo.group(1) == b"VERSION":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1512 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1513 logging.warning("CRC32 missing before line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1514 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1515 self._reset_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1516 self._expect_crc = True
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1517 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1518 return ("VERSION", util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1519 if mo.group(1) == b"CRC32":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1520 # TODO: check
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1521 if self._expect_crc is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1522 logging.warning("Lone CRC32 before VERSION in line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1523 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1524 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1525 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1526 if (self._hex_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1527 != mo.group(2).decode("latin1").upper()):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1528 logging.warning(
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1529 "CRC32 mismatch in line %d:"
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1530 " expected: %s, given: %s",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1531 self._line_no,
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1532 self._hex_crc(),
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1533 mo.group(2).decode("latin1").upper())
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1534 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1535 logging.warning("CRC32 before VERSION in line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1536 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1537 # Do not update the CRC here but reset the state
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1538 self._expect_crc = False
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1539 return ("CRC32", util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1540 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1541 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1542 return (util.n(mo.group(1)), util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1543 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1544 mo = self.PATTERN2.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1545 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1546 self._update_crc(line)
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1547 if mo.group(1) in (b"COMMENT", b"ERROR", b"GENERATOR",
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1548 b"FNMATCH"):
316 3f1f1c2e9e5f treesum: FIX: returned tag should be a native string Franz Glasner <fzglas.hg@dom66.de> parents: 315 diff changeset	1549 return (util.n(mo.group(1)), util.u(mo.group(2), "utf-8"))
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1550 elif mo.group(1) == b"ROOT":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1551 return ("ROOT", mo.group(2))
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1552 elif mo.group(1) == b"ACCEPT-TREESUM":
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1553 return ("ACCEPT-TREESUM", mo.group(2))
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1554 assert False, line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1555 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1556 mo = self.PATTERN3.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1557 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1558 self._update_crc(line)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1559 if mo.group(2):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1560 return ("SIZE", mo.group(1),
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1561 int(util.n(mo.group(3)), 10))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1562 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1563 return ("SIZE", mo.group(1), None)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1564 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1565 mo = self.PATTERN4.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1566 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1567 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1568 algo_name = util.n(mo.group(1))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1569 if mo.group(3):
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1570 if mo.group(4):
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1571 if (len(mo.group(4)) ==
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1572 2 * self._get_digest_size(algo_name)):
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1573 # hex
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1574 digest = binascii.unhexlify(mo.group(4))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1575 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1576 # base64
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1577 digest = base64.b64decode(mo.group(4))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1578 else:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1579 digest = None
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1580 if mo.group(5):
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1581 if mo.group(6):
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1582 size = int(util.n(mo.group(6)), 10)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1583 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1584 size = None
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1585 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1586 size = None
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1587 return (algo_name, mo.group(2), digest, size)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1588 else:
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1589 return (algo_name, mo.group(2), None, None)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1590 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1591 assert False, line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1592 return line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1593
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1594 def _get_next_line(self):
189 959c6d37b014 Extend the max line length to read to 4096, which is something along PATH_MAX onn Linux Franz Glasner <fzglas.hg@dom66.de> parents: 188 diff changeset	1595 line = self._fp.readline(4096) # along PATH_MAX on Linux
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1596 if line:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1597 self._line_no += 1
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1598 return line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1599
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1600 def _reset_crc(self):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1601 self._crc32 = crc32()
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1602
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1603 def _update_crc(self, data):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1604 self._crc32.update(data)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1605
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1606 def _hex_crc(self):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1607 return self._crc32.hexdigest()
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1608
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1609 def _get_digest_size(self, algo_name):
307 64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1610 """Get the `digest_size` from algorithm specifier `algo_name`.
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1611
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1612 Cache this on the assumption, that algorithms do not change very
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1613 often. Do this because the `digest_size` can only be given by a
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1614 digest instance.
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1615
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1616 """
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1617 if self._current_algo_name == algo_name:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1618 return self._current_algo_digest_size
307 64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1619 sz = util.algotag2digest_size(algo_name)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1620 self._current_algo_name = algo_name
307 64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1621 self._current_algo_digest_size = sz
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1622 return self._current_algo_digest_size
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1623
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1624
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1625 def print_treesum_digestfile_infos(opts):
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1626 get_infos_from_digestfile(
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1627 opts.digest_files,
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1628 print_block_data,
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1629 opts.print_only_last_block)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1630
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1631
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1632 def get_infos_from_digestfile(digest_files, block_handler,
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1633 only_last_block=True):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1634 for fn in digest_files:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1635 if fn == "-":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1636 if util.PY2:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1637 reader = TreesumReader.from_binary_buffer(sys.stdin)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1638 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1639 reader = TreesumReader.from_binary_buffer(sys.stdin.buffer)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1640 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1641 reader = TreesumReader.from_path(fn)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1642
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1643 with reader:
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1644 root = generator = flags = fsencoding = algorithm = digest \
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1645 = size = None
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1646 errors = set()
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1647 comments = []
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1648 fnmatch_filters = []
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1649 in_block = False
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1650 block_no = 0
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1651 for record in reader:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1652 if record[0] == "VERSION":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1653 assert record[1] == "1"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1654 # start a new block
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1655 in_block = True
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1656 block_no += 1
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1657 root = flags = algorithm = digest = size = None
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1658 comments = []
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1659 elif record[0] == "GENERATOR":
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1660 generator = record[1]
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1661 elif record[0] == "FSENCODING":
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1662 fsencoding = record[1]
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1663 elif record[0] == "FLAGS":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1664 flags = record[1]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1665 elif record[0] == "ROOT":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1666 root = record[1]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1667 elif record[0] == "COMMENT":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1668 comments.append(record[1])
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1669 elif record[0] == "ERROR":
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1670 errors.add(record[1])
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1671 elif record[0] == "FNMATCH":
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1672 fnmatch_filters.append(record[1])
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1673 elif record[0] in ("TIMESTAMP", "ISOTIMESTAMP"):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1674 pass
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1675 elif record[0] == "ACCEPT-TREESUM":
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1676 pass
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1677 elif record[0] == "CRC32":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1678 pass
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1679 # in_block = False
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1680 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1681 if not in_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1682 continue
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1683 # digest line or size line
193 fb36e71f6ba8 Change: path indicators for symlinks: ./@ -> ./@/ and /./@ -> /./@/ Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	1684 if not record[1] or record[1] == b"./@/":
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1685 if record[0] == "SIZE":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1686 algorithm = "SIZE"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1687 digest = None
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1688 size = record[2]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1689 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1690 algorithm = record[0]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1691 digest = record[2]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1692 size = record[3]
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1693 if not only_last_block:
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1694 block_handler(
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1695 block_no,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1696 root, generator, fsencoding, flags,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1697 fnmatch_filters,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1698 comments, errors, algorithm, digest, size)
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1699 root = generator = flags = fsencoding = algorithm \
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1700 = digest = size = None
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1701 errors = set()
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1702 comments = []
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1703 in_block = False
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1704 if only_last_block:
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1705 if not in_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1706 if digest is not None or size is not None:
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1707 block_handler(
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1708 block_no,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1709 root, generator, fsencoding, flags, fnmatch_filters,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1710 comments, errors, algorithm, digest, size)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1711 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1712 logging.warning("missing block end")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1713
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1714
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1715 def print_block_data(block_no, tag, generator, fsencoding, flags,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1716 fnmatch_filters, comments, errors,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1717 algorithm, digest, size):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1718 digeststr = util.n(binascii.hexlify(digest)) if digest else "<no digest>"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1719 sizestr = str(size) if size is not None else "<no size>"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1720 print("BLOCK No %d:" % (block_no,))
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1721 print(" Tag:", tag)
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1722 print(" FS-Encoding:", fsencoding)
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1723 if generator:
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1724 print(" Generator:", generator)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1725 print(" Flags:", flags if flags else "<none>")
220 8db78850d800 "treesum info": print comments only if not empty Franz Glasner <fzglas.hg@dom66.de> parents: 219 diff changeset	1726 if comments:
8db78850d800 "treesum info": print comments only if not empty Franz Glasner <fzglas.hg@dom66.de> parents: 219 diff changeset	1727 print(" Comments:", comments)
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1728 if fnmatch_filters:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1729 for f in fnmatch_filters:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1730 print(" FNMatch:", f)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1731 print(" Algorithm:", algorithm)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1732 if algorithm != "SIZE":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1733 print(" Digest:", digeststr)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1734 print(" Size:", sizestr)
303 73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1735 if errors:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1736 errorlist = list(errors)
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1737 errorlist.sort()
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1738 for idx, err in enumerate(errorlist):
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1739 if idx == 0:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1740 print(" Errors:", err)
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1741 else:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1742 print(" ", err)
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1743 else:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1744 print(" Errors: <none>")
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1745
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1746
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1747 class DigestSizeCollector(object):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1748
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1749 def __init__(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1750 self._algorithm = self._digest = self._size = None
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1751
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1752 def __call__(self, block_no, tag, generator, fsencoding, flags,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1753 fnmatch_filters, comments, errors,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1754 algorithm, digest, size):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1755 self._algorithm = algorithm
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1756 self._digest = digest
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1757 self._size = size
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1758
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1759 @property
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1760 def algorithm(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1761 return self._algorithm
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1762
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1763 @property
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1764 def digest(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1765 return self._digest
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1766
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1767 @property
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1768 def size(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1769 return self._size
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1770
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1771 def collect_from_file(self, digest_file):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1772 get_infos_from_digestfile([digest_file], self, True)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1773
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1774
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1775 if __name__ == "__main__":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1776 sys.exit(main())

Mercurial > hgrepos > Python > apps > py-cutils

annotate cutils/treesum.py @ 322:0cabc5439505